Resources

Resources represent data files that can be downloaded from a provider.

Single file download

Download a single file from a URL.

→ Use file resources such as HTTP resources for standard HTTP resources or SDMX resources when the file is served by an SDMX 2.1 web service (adds SDMX error handling and header normalization).

Couple of files download

Download several files that form a logical unit (e.g., a data file and its structure file). If any file fails, none are written to the target directory — the group is all-or-nothing.

→ Use Resource groups

Paginated download

Download resources from an HTTP API paginated with ?page=N where the total number of pages is unknown. The group enumerates pages starting from start_page_num and stops when the resource raises PageOutOfBounds.

There is no split: each page corresponds to exactly one resource, and failures are not retried as smaller sub-pages.

→ Use Page resource groups

Partitioned download

Download a large dataset by splitting it into partitions (e.g., filtered by time period, by region). Unlike pagination, partitions can be split dynamically: when a resource fails (e.g., ResponseTooLarge), it can raise EnqueueSubPartitions to replace 1 resource with N smaller ones.

→ Use Partition groups