Skip to main content

Statistics Netherlands opendata API client for Python

Project description

Statistics Netherlands opendata API client for Python

pypi tests

Retrieve data from the open data interface of Statistics Netherlands (Centraal Bureau voor de Statistiek) with Python. The data is identical in content to the tables which can be retrieved and downloaded from StatLine. CBS datasets are accessed via the CBS open data portal.

The documentation of this package is found at this page and on readthedocs.io.

R user? Use cbsodataR.

Installation

From PyPi

pip install cbsodata

Usage

Load the package with

>>> import cbsodata

Tables

Statistics Netherlands (CBS) has a large amount of public available data tables (more than 4000 at the moment of writing). Each table is identified by a unique identifier (Identifier).

>>> tables = cbsodata.get_table_list()
>>> print(tables[0])
{'Catalog': 'CBS',
 'ColumnCount': 18,
 'DefaultPresentation': '_la=nl&_si=&_gu=&_ed=LandVanUiteindelijkeZeggenschapUCI&_td=Perioden&graphType=line',
 'DefaultSelection': "$filter=((LandVanUiteindelijkeZeggenschapUCI eq '11111') or (LandVanUiteindelijkeZeggenschapUCI eq '22222')) and (Bedrijfsgrootte eq '10000') and (substringof('JJ',Perioden))&$select=LandVanUiteindelijkeZeggenschapUCI, Bedrijfsgrootte, Perioden, FiscaalJaarloonPerBaan_15",
 'ExplanatoryText': '',
 'Frequency': 'Perjaar',
 'GraphTypes': 'Table,Bar,Line',
 'ID': 0,
 'Identifier': '82010NED',
 'Language': 'nl',
 'MetaDataModified': '2014-02-04T02:00:00',
 'Modified': '2014-02-04T02:00:00',
 'OutputStatus': 'Regulier',
 'Period': '2008 t/m 2011',
 'ReasonDelivery': 'Actualisering',
 'RecordCount': 32,
 'SearchPriority': '2',
 'ShortDescription': '\nDeze tabel bevat informatie over banen en lonen bij bedrijven in Nederland, uitgesplitst naar het land van uiteindelijke zeggenschap van die bedrijven. Hierbij wordt onderscheid gemaakt tussen bedrijven onder Nederlandse zeggenschap en bedrijven onder buitenlandse zeggenschap. In de tabel zijn alleen de bedrijven met werknemers in loondienst meegenomen. De cijfers hebben betrekking op het totale aantal banen bij deze bedrijven en de samenstelling van die banen naar kenmerken van de werknemers (baanstatus, geslacht, leeftijd, herkomst en hoogte van het loon). Ook het gemiddelde fiscale jaarloon per baan is in de tabel te vinden. \n\nGegevens beschikbaar vanaf: 2008 \n\nStatus van de cijfers: \nDe cijfers in deze tabel zijn definitief.\n\nWijzigingen per 4 februari 2014\nDe cijfers van 2011 zijn toegevoegd.\n\nWanneer komen er nieuwe cijfers?\nDe cijfers over 2012 verschijnen in de eerste helft van 2015.\n',
 'ShortTitle': 'Zeggenschap bedrijven; banen, grootte',
 'Source': 'CBS.',
 'Summary': 'Banen en lonen van werknemers bij bedrijven in Nederland\nnaar land van uiteindelijke zeggenschap en bedrijfsgrootte',
 'SummaryAndLinks': 'Banen en lonen van werknemers bij bedrijven in Nederland<br />naar land van uiteindelijke zeggenschap en bedrijfsgrootte<br /><a href="http://opendata.cbs.nl/ODataApi/OData/82010NED">http://opendata.cbs.nl/ODataApi/OData/82010NED</a><br /><a href="http://opendata.cbs.nl/ODataFeed/OData/82010NED">http://opendata.cbs.nl/ODataFeed/OData/82010NED</a>',
 'Title': 'Zeggenschap bedrijven in Nederland; banen en lonen, bedrijfsgrootte',
 'Updated': '2014-02-04T02:00:00'}

Info

Get information about a table with the get_info function.

>>> info = cbsodata.get_info('82070ENG') # Returns a dict with info
>>> info['Title']
'Caribbean Netherlands; employed labour force characteristics 2012'
>>> info['Modified']
'2013-11-28T15:00:00'

Data

The function you are looking for!! The function get_data returns a list of dicts with the table data.

>>> data = cbsodata.get_data('82070ENG')
[{'CaribbeanNetherlands': 'Bonaire',
  'EmployedLabourForceInternatDef_1': 8837,
  'EmployedLabourForceNationalDef_2': 8559,
  'Gender': 'Total male and female',
  'ID': 0,
  'Periods': '2012',
  'PersonalCharacteristics': 'Total personal characteristics'},
 {'CaribbeanNetherlands': 'St. Eustatius',
  'EmployedLabourForceInternatDef_1': 2099,
  'EmployedLabourForceNationalDef_2': 1940,
  'Gender': 'Total male and female',
  'ID': 1,
  'Periods': '2012',
  'PersonalCharacteristics': 'Total personal characteristics'},
 {'CaribbeanNetherlands': 'Saba',
  'EmployedLabourForceInternatDef_1': 1045,
  'EmployedLabourForceNationalDef_2': 971,
  'Gender': 'Total male and female',
  'ID': 2,
  'Periods': '2012',
  'PersonalCharacteristics': 'Total personal characteristics'},
 # ...
]

The keyword argument dir can be used to download the data directly to your file system.

>>> data = cbsodata.get_data('82070ENG', dir="dir_to_save_data")

Catalogs (dataderden)

There are multiple ways to retrieve data from catalogs other than 'opendata.cbs.nl'. The code below shows 3 different ways to retrieve data from the catalog 'dataderden.cbs.nl' (known from Iv3).

On module level.

cbsodata.options.catalog_url = 'dataderden.cbs.nl'
# list tables
cbsodata.get_table_list()
# get dataset 47003NED
cbsodata.get_data('47003NED')

With context managers.

with cbsodata.catalog('dataderden.cbs.nl'):
    # list tables
    cbsodata.get_table_list()
    # get dataset 47003NED
    cbsodata.get_data('47003NED')

As a function argument.

# list tables
cbsodata.get_table_list(catalog_url='dataderden.cbs.nl')
# get dataset 47003NED
cbsodata.get_data('47003NED', catalog_url='dataderden.cbs.nl')

Pandas users

The package works well with Pandas. Convert the result easily into a pandas DataFrame with the code below.

>>> data = pandas.DataFrame(cbsodata.get_data('82070ENG'))
>>> data.head()

The list of tables can be turned into a pandas DataFrame as well.

>>> tables = pandas.DataFrame(cbsodata.get_table_list())
>>> tables.head()

Command Line Interface

This library ships with a Command Line Interface (CLI).

> cbsodata -h
usage: cbsodata [-h] [--version] [subcommand]

CBS Open Data: Command Line Interface

positional arguments:
  subcommand  the subcommand (one of 'data', 'info', 'list')

optional arguments:
  -h, --help  show this help message and exit
  --version   show the package version

Download data:

> cbsodata data 82010NED

Retrieve table information:

> cbsodata info 82010NED

Retrieve a list with all tables:

> cbsodata list

Export data

Use the flag -o to load data to a file (JSON lines).

> cbsodata data 82010NED -o table_82010NED.jl

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cbsodata-1.3.5.tar.gz (22.4 kB view details)

Uploaded Source

Built Distribution

cbsodata-1.3.5-py3-none-any.whl (12.5 kB view details)

Uploaded Python 3

File details

Details for the file cbsodata-1.3.5.tar.gz.

File metadata

  • Download URL: cbsodata-1.3.5.tar.gz
  • Upload date:
  • Size: 22.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.6

File hashes

Hashes for cbsodata-1.3.5.tar.gz
Algorithm Hash digest
SHA256 32e4f0ed6884794513a4666673c53fe0227219f409c370615938eff60bf146c8
MD5 97f2f98964d803b3949765616562bf91
BLAKE2b-256 20a2abc89c2ff94c39b2d56c7a3c344c8ac22b7c96ffc49908bacf98261fcd65

See more details on using hashes here.

File details

Details for the file cbsodata-1.3.5-py3-none-any.whl.

File metadata

  • Download URL: cbsodata-1.3.5-py3-none-any.whl
  • Upload date:
  • Size: 12.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.6

File hashes

Hashes for cbsodata-1.3.5-py3-none-any.whl
Algorithm Hash digest
SHA256 01e980914dabe7fefa11dd8bdad3672c520d4b0fd00b93439b3afb1958839aad
MD5 a8e8950e3f6bb183429541d3764c766d
BLAKE2b-256 858c70ff1ba83e76eeee7d095bca26850a796f1f79203c9912b51698512094e0

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page