Cineon Format

This package contains a definition of a common data format to be used across Cineon repositories.

The CineonData format is a class that inherits from pydantic's BaseModel class, and as such the data validation is performed on class instantiation. If the data being ingested has a problem then class instantiation will fail. Simple. This is useful because downstream processing tasks can make valid assumptions about the data and therefore do not have to perform any explicit checks. This means that lots of downstream data-validation code can be deleted.

Prerequisites

To use the example script in the package you need to install uv, but you do not need uv in order to use the package in other Python code.

Usage

Add the cineon_format package into your project, with either:

poetry add git+https://github.com/cineon-ai/cineon_format.git
uv add git+https://github.com/cineon-ai/cineon_format.git

Then:

from cineon_format import CineonData

# Load from CSV
cineon_data = CineonData.from_csv("path/to/csv")

# Alternatively, load from JSON
cineon_data = CineonData.from_json("path/to/json")

Examples

Running the example script with either:

uv run scripts/example.py --csv=data/good.csv
uv run scripts/example.py --json=data/good.json

will load a file, convert that file into the CineonData format and then spit out a dictionary representation and a DataFrame representation to the terminal.

If you run the script with either:

uv run scripts/example.py --csv=data/bad.csv
uv run scripts/example.py --json=data/bad.json

you can see examples of data that does not conform to the format, and therefore errors out with helpful error messages printed to the terminal.

"Random" CineonData can be generated via the script:

uv run scripts/generate.py

This will print a summary of the CineonData to the terminal.