SDMX resources

SdmxResource extends HttpResource to handle SDMX 2.1 web service responses. It adds SDMX-specific error handling and XML header normalization on top of the standard HTTP resource capabilities.

Constructor parameters

In addition to all parameters inherited from HttpResource and FileResource, SdmxResource accepts:

  • normalize_sdmx_file_header (bool | None, default: True) — whether to normalize the SDMX file header after download (see below).

  • sdmx_parser_factory (SdmxParserFactory | None) — a factory for creating SDMX parsers. Defaults to SdmxParserFactory().

SDMX error handling

SDMX web services can return errors in the XML response body that differ from the HTTP status code. The method raise_for_sdmx_error(error) inspects the response content and raises specific exceptions:

  • NoResultFound — when the SDMX error code is NO_RESULT_FOUND (the query returned no data).

  • ResponseTooLarge — when the SDMX error code is RESPONSE_TOO_LARGE (the requested dataset exceeds the server’s size limit).

  • SdmxResourceError — for any other SDMX error code.

These exceptions can be caught by a BasePageResourceGroup or BasePartitionGroup to trigger a split or stop pagination.

Example: handling response-too-large in a partition group

from dbnomics_toolbox.fetcher_utils.processors.errors import EnqueueSubPartitions
from dbnomics_toolbox.fetcher_utils.resources.errors import ResponseTooLarge


class MyPartitionGroup(BasePartitionGroup):
    def _download_partition(self, partition, *, partition_depth, partition_num):
        try:
            super()._download_partition(
                partition,
                partition_depth=partition_depth,
                partition_num=partition_num,
            )
        except ResponseTooLarge:
            # Split the partition into smaller ones
            raise EnqueueSubPartitions(partition.split()) from None

SDMX header normalization

When normalize_sdmx_file_header=True (default), the _post_process_target_file method calls normalize_sdmx_file_header after the standard MIME type validation and file reformatting.

This normalization ensures that SDMX XML headers are consistent across different versions of the same file, reducing noise in version control diffs (e.g., the Prepared timestamp is normalized to a fixed date).

Download flow

  1. HttpResource fetches the URL and writes the response to the target file.

  2. FileResource validates the MIME type and reformats the file (JSON/XML).

  3. SdmxResource normalizes the SDMX file header.

If any of these steps fails, the resource is considered failed.