cernopendata-client

About

cernopendata-client is a command-line tool to facilitate downloading files from the CERN Open Data portal. The tool enables to query datasets hosted on the CERN Open Data portal and to download and verify the individual data set files.

Installation

$ pip install cernopendata-client

Usage

The detailed information on how to install and use cernopendata-client can be found in cernopendata-client.readthedocs.io.

Development

1. Clone the repository:

   $ git clone https://github.com/cernopendata/cernopendata-client
   

2. Setup a virtual environment:

   $ python3 -m venv env
   $ source env/bin/activate
   

3. Install [cernopendata-client]{.title-ref} in editable mode:

   $ pip install -e '.[tests]'
   

Useful links

• CERN Open Data portal
• CERN Open Data user forum

Changelog

1.0.0 (2025-02-25)

⚠ BREAKING CHANGES

• python: Drops support for Python 3.6 and 3.7.

Build

• docker: upgrade to Ubuntu 24.04 and Python 3.12 (#143) (bac8200)
• python: add support for Python 3.13 (#146) (1714dbf)
• python: drop support for Python 3.6 and 3.7 (#143) (cb633e0)

Bug fixes

• searcher: adapt to the new metadata schema with file indices (#147) (bab9401), closes #148

Code refactoring

• docs: move from reST to Markdown (#145) (845b1bc)

Test suite

• metadater: adapt filtering test after CCID removal (#147) (4ff86a9)

Continuous integration

• actions: update GitHub actions due to Node 16 deprecation (#143) (a3a898e)
• actions: upgrade to Ubuntu 24.04 and Python 3.12 (#143) (925c0b5)
• commitlint: addition of commit message linter (#143) (e01fff8)
• jsonlint: add JSON linting (#150) (cab5fcb)
• markdownlint: add Markdown linting (#150) (45a2a5f)
• prettier: add Prettier code formatting checks (#150) (1020c7c)
• release-please: increment version number in Dockerfile (#151) (1bbfcd5)
• release-please: initial Release Please configuration (#143) (5724844)
• run-tests: add usage help and refactor options (#150) (b9368d8)
• run-tests: stop properly after running all tests (#145) (357a719)
• shfmt: add shfmt code formatting checks (#150) (e70291e)
• yamllint: add YAML linting (#150) (0d70783)

Documentation

• usage: add an example on how to retrieve container images (#141) (fef3295)

0.4.0 (2024-08-22)

• Adds support for Python 3.11 and 3.12.
• Improves get-metadata --output-field command by adding a new --filter option allowing to output only selected field values matching desired criteria.
• Changes container image base to Fedora 38 and slightly optimises image size.
• Drops support for Python 2.7.

0.3.0 (2022-03-16)

• Adds support for Python 3.10.
• Improves download-files command to resume interrupted downloads when using HTTP downloads with pycurl engine.
• Improves XRootD downloader by using vanilla XRootD package. Changes xrootdpyfs to xrootd download engine.
• Improves download-files command with a new option --download-engine to select pycurl or requests engines when downloading files over HTTP.
• Fixes minor issues with file index unwinding and output directory handling.

0.2.0 (2020-11-19)

• Adds new list-directory command to list content of EOS directories holding open data files.
• Adds support for Python 3.9.
• Improves download-files command to allow using XRootD protocol.
• Improves download-files command to allow using HTTPS protocol.
• Improves download-files command to optionally verify file integrity as soon as files are being downloaded.
• Improves get-file-locations command to optionally output file sizes and checksums.
• Improves output format colouring to better indicate notes and errors.
• Fixes minor issues and improves code coverage.

0.1.0 (2020-09-24)

• Enriches download-files command to optionally download only files matching certain name, regexp, or range count.
• Adds new verify-files command to verify number, size, and checksum of downloaded files.
• Improves get-metadata command with respect to outputting only certain desired metadata field values.
• Enriches user documentation.

0.0.1 (2020-09-09)

• Initial public release.