Skip to main content

A suite of command-line tools for working with OCDS data

Project description

PyPI version Build Status Dependency Status Coverage Status

A suite of command-line tools for working with OCDS data.

pip install ocdskit
ocdskit --help

To install from source:

pip install --upgrade .

To see all commands available, run:

ocdskit --help

Most ocdskit tools accept JSON data from standard input. To report on a remote file:

curl <url> | ocdskit <command>

To report on a local file:

cat <path> | ocdskit <command>

For exploring JSON data, consider using jq.


Optional arguments for all commands are:

  • -h, --help show the help message and exit
  • --encoding ENCODING the file encoding
  • --pretty pretty print output


Reads record packages from standard input, collects packages and records, and prints one record package.

cat tests/fixtures/record-package_*.json | ocdskit combine-record-packages > out.json


Reads release packages from standard input, collects releases, and prints one release package.

cat tests/fixtures/release-package_*.json | ocdskit combine-release-packages > out.json


Reads release packages from standard input, merges the releases by OCID, and prints the compiled releases.

cat tests/fixtures/realdata/release-package-1.json | ocdskit compile > out.json


Indents JSON files by modifying the given files in-place.

Optional arguments:

  • -r, --recursive recursively indent JSON files
  • --indent INDENT indent level
ocdskit indent --recursive file1 path/to/directory file2


Generates a spreadsheet with all field paths from an OCDS schema.

cat path/to/release-schema.json | ocdskit mapping-sheet > mapping-sheet.csv


Reports details of a JSON Schema (open and closed codelists).

cat path/to/release-schema.json | ocdskit schema-report


For any required field, adds “minItems” if an array, “minProperties” if an object and “minLength” if a string and “enum”, “format” and “pattern” are not set.

cat path/to/release-schema.json | ocdskit schema-strict > out.json


Sets the enum in a JSON Schema to match the codes in the CSV files of closed codelists.

ocdskit set-closed-codelist-enums path/to/standard path/to/extension1 path/to/extension2


Load packages into a database.

Optional arguments:

  • --drop drop all tables before loading
  • --schema SCHEMA the release-schema.json to use
cat release_package.json | ocdskit tabulate sqlite:///data.db

For the format of database_url, see the SQLAlchemy documentation.


Reads JSON data from standard input, validates it against the schema, and prints errors.

Optional arguments:

  • --schema SCHEMA the schema to validate against
  • --check-urls check the HTTP status code if “format”: “uri”
  • --timeout TIMEOUT timeout (seconds) to GET a URL
cat tests/fixtures/* | ocdskit validate


Example 1

Download a list of release packages:

curl > release_packages.json

Transform it to a stream of release packages, and validate each:

jq -crM '.[]' release_packages.json | ocdskit validate --schema

Or, validate each with a local schema file:

jq -crM '.[]' release_packages.json | ocdskit validate --schema file:///path/to/release-package-schema.json

Transform it to a stream of compiled releases:

jq -crM '.[]' release_packages.json | ocdskit compile > compiled_releases.json

Find a compiled release with a given ocid (replace the ):

jq 'select(.ocid == "OCDS-87SD3T-AD-SF-DRM-063-2015")' compiled_releases.json

Measure indicators across release packages:

cat release_packages.json | ocdskit --encoding iso-8859-1 measure --currency MXN

Example 2

Download a list of record packages:

curl > record_packages.json

Combine it into a single record package:

jq -crM '.[]' record_packages.json | ocdskit combine-record-packages > record_package.json

If the file is too large for the OCDS Validator, you can break it into parts. First, transform the list into a stream:

jq -crM '.[]' record_packages.json > stream.json

Combine the first 10,000 items from the stream into a single record package:

head -n 10000 stream.json | ocdskit combine-record-packages > record_package-1.json

Then, combine the next 10,000 items from the stream into a single record package:

tail -n +10001 stream.json | head -n 10000 | ocdskit combine-record-packages > record_package-2.json

And so on:

tail -n +20001 stream.json | head -n 10000 | ocdskit combine-record-packages > record_package-3.json

Copyright (c) 2017 Open Contracting Partnership, released under the BSD license

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ocdskit, version 0.0.2
Filename, size File type Python version Upload date Hashes
Filename, size ocdskit-0.0.2.tar.gz (17.9 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page