Existing fetchers¶
To learn how to use dbnomics-toolbox, you may be interested by reading the source code of existing fetchers.
All the fetchers of DBnomics are open-source and available on GitLab. You can browse the source code of each fetcher, however not all the fetchers use dbnomics-toolbox.
Here is a selection of fetchers that use dbnomics-toolbox as a foundation, sorted by increasing complexity:
Fetcher |
Approach |
Best for learning… |
|---|---|---|
abc-fetcher (dummy) |
CSV files (mock) |
the simplest base pattern — JSON catalog, pandas conversion, |
Paginated JSON API |
how to consume a paginated API with |
|
SDMX 2.1 REST API |
how to extend the SDMX base classes ( |
|
SDMX 2.1 + rate limiting |
how to handle API quotas (20 req/min), configure exponential backoff, add provider-specific CLI arguments, and post-process XML/SDMX files. |
|
Website scraping |
how to scrape HTML with BeautifulSoup, build a category tree from a website structure, and extract CSV links when no API is available. |