Skip to main content

DCAT to Intake Catalog translation layer

Project description



This is an intake data source for DCAT catalogs.

These catalogs are a standardized format for describing metadata and access information for public datasets, as described here. Many Socrata and ESRI data portals publish data.json files in this format describing their catalogs. Two examples of thes can be found at

This project provides an opinionated way for users to load datasets from these catalogs into the scientific Python ecosystem. At the moment it loads CSVs into Pandas dataframes and GeoJSON files into GeoDataFrames, and ESRI Shapefiles into GeoDataFrames. Future formats could include plain JSON and Parquet.


intake >= 0.4.4
intake_geopandas >= 0.2.2
geopandas >= 0.5.0


intake-dcat is published on PyPI. You can install it by running the following in your terminal:

pip install intake-dcat

You can test the functionality by opening the example notebooks in the examples/ directory


The package can be imported using

from intake_dcat import DCATCatalog

Loading a catalog

You can load data from a DCAT catalog by providing the URL to the data.json file:

catalog = DCATCatalog('', name='geohub')

You can display the items in the catalog

for entry_id, entry in catalog.items():

If the catalog has too many entries to comfortably print all at once, you can narrow it by searching for a term (e.g. 'district'):

for entry_id, entry in'district').items():

Loading a dataset

Once you have identified a dataset, you can load it into a dataframe using read():

df =

This will automatically load that dataset into a Pandas dataframe, or a GeoDataFrame, depending on the source format.

Specifying catalogs

You can read a DCATCatalog directly in Python using a URL, as done above, but it is also possible to write a catalog file that itself contains DCATCatalog entries. This allows you to more easily specify DCAT catalogs for use in distribution and version control.

For instance, this YAML file creates entries for two open data catalogs:

  version: 1
  # Here we have two data sources for this catalog, which are themselves
  # DCAT catalogs, one for LA open data, and the other for LA GeoHub
    # We identify them as being loaded with the DCAT driver
    driver: dcat
    # Here we specify the args used to load the catalog
      # The URL to the catalog
      # An optional name for the catalog.
      name: la-open-data
    driver: dcat
      name: la_geohub
      # We can also specify a subset of the datasets in the catalog using an "items"
      # dictionary. If these are specified, only these datasets will be available in
      # the resulting catalog. They will be available under the more human-readable
      # name specified as the key.
        # So, this dataset will be available as "bikeways"

Command Line Interface

intake-dcat provides a small command line interface for some common operations. These are invoked using intake-dcat <subcommand> <options>

The mirror command

This command loads a manifest file that lists a set of DCAT entries, uploads them to a specified s3 bucket, and outputs a new catalog with identical entries pointing to the bucket.

An example manifest is given by

# Name of the LA open data portal
  # URL to the open data portal catalog
  # The s3 bucket to upload the data to
  bucket_uri: s3://my-bucket
  # A list of data resources to mirror
# Name of the LA GeoHub data portal
  # URL to the open data portal catalog
  # The s3 bucket to upload the data to
  bucket_uri: s3://my-bucket
  # A list of data resources to mirror

This can be mirrored using the command

intake-dcat mirror manifest.yml > new-catalog.yml

This command uses the boto3 library and assumes it can find AWS credentials. For more information see this documentation.

The create command

This command creates a new intake catalog from a DCAT catalog, and outputs it to standard out. An example command is given by

intake-dcat create > catalog.yml

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

intake-dcat-0.4.0.tar.gz (13.4 kB view hashes)

Uploaded Source

Built Distribution

intake_dcat-0.4.0-py3-none-any.whl (14.3 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page