Skip to main content

Blue Core Data Models

The Blue Core Data Models are used in Blue Core API and in the Blue Core Workflows services.

🐳 Run Postgres with Docker

To run the Postgres with the Blue Core Database, run the following command from this directory:

docker run --name bluecore_db -e POSTGRES_USER=airflow -e POSTGRES_PASSWORD=airflow -v ./create-db.sql:/docker-entrypoint-initdb.d/create_database.sql -p 5432:5432 postgres:17


🛠️ Installing

  • Install via pip: pip install bluecore-models
  • Install via uv: uv add bluecore-models

🗄️ Database Management

The SQLAlchemy Object Relational Mapper (ORM) is used to create the Bluecore database models.

erDiagram
    ResourceBase ||--o{ Hub : "has"
    ResourceBase ||--o{ Instance : "has"
    ResourceBase ||--o{ Work : "has"
    ResourceBase ||--o{ OtherResource : "has"
    ResourceBase ||--o{ Profile : "has"
    ResourceBase ||--o{ ResourceBibframeClass : "has classes"
    ResourceBase ||--o{ Version : "has versions"
    ResourceBase ||--o{ BibframeOtherResources : "has other resources"

    Hub ||--o{ Work : "has"
    Work ||--o{ Instance : "has"
    Instance ||--o{ InstanceExport : "exported as"
    Version ||--o{ InstanceExport : "snapshot"
    
    BibframeClass ||--o{ ResourceBibframeClass : "classifies"
    
    OtherResource ||--o{ BibframeOtherResources : "links to"

    Profile ||--o{ ProfileRelation : "nests"
    Profile ||--o{ ProfileRelation : "is nested by"

Profile nesting

A Profile is a Sinopia profile: JSON-LD describing how to edit a kind of resource. Its data is stored unframed, because Sinopia Editor needs back the shape it sent.

A profile's data may name other profiles that it nests, with sinopia:hasResourceTemplateId. Those references are stored as profile URIs, and each one that resolves is recorded as a ProfileRelation row with a foreign key on both ends. The nesting is many-to-many — one profile is commonly nested by several others — so it cannot be a column on profiles.

Both foreign keys cascade, so deleting either end removes the edge, and a CHECK constraint stops a profile nesting itself. That means "is this profile nested by anything?" is derived rather than stored, and cannot go stale:

# Every top-level profile, as one SQL statement with an inlined NOT EXISTS.
session.scalars(select(Profile).where(~Profile.is_nested))

Profile.is_nested is deferred, so an ordinary select(Profile) does not carry the subquery. It is meant for filtering in SQL: reading it off an instance that did not select it issues a query, so it is an N+1 in a loop. Use Profile.children and Profile.parents to walk the relation itself.

References are expected to be profile URIs. A reference naming nothing stored records no row rather than raising — bluecore_api rejects those on save, where there is a cataloger to tell.

Works are linked to Instances by bf:instanceOf / bf:hasInstance, and to a Hub by bf:expressionOf (see BluecoreGraph._link).

Both ends of a link need a URI. A resource created in an editor arrives without one and is minted a Bluecore URI, but a blank node in a bulk-loaded record is an inline description of something the record merely refers to — LC catalog data often states bf:expressionOf against an anonymous bf:Hub — so it gets no record and no link (see BluecoreGraph._anonymous_description).

Instance exports

An InstanceExport records one export of an Instance to another system, such as FOLIO or Alma. It stores the Keycloak user who exported it, the Keycloak group it was exported on behalf of, when it happened, the Version exported, and the identifier the record received in the target system, if one is known.

Exports are recorded explicitly, not by an ORM event, since an export doesn't change the Instance:

from bluecore_models.utils.db import add_instance_export

add_instance_export(session, instance, target_identifier="in00000123")
session.commit()

The user, username and group are read from the CURRENT_USER_ID, CURRENT_USERNAME and CURRENT_GROUP context vars, which bluecore_api sets from the token. Callers outside HTTP, such as bluecore-workflows, pass keycloak_user_id, keycloak_username and keycloak_group instead. The version defaults to the Instance's latest. A missing user id or group raises ValueError, since an anonymous export isn't a useful audit record.

Exports outlive the Instance: both foreign keys are ON DELETE SET NULL, and instance_uri keeps a copy of the Instance's URI to identify it afterwards.

Database Migrations with Alembic

The Alembic database migration package is used to manage database changes with the Bluecore Data models.

To create a new migration, ensure that the Postgres database is available and then run:

  • uv run alembic revision --autogenerate -m "{short message describing change}

A new migration script will be created in the bluecore_store_migration directory. Be sure to add the new script to the repository with git.

Applying Migrations

To apply all of the migrations, run the following command:

  • uv run alembic upgrade head

🧹 Linter for Python

bluecore-models uses ruff

  • uv run ruff check

To auto-fix errors in both (where possible):

  • uv run ruff check --fix

Check formatting differences without changing files:

  • uv run ruff format --diff

Apply Ruff's code formatting:

  • uv run ruff format

🧪 Running Tests

The test suite is written using pytest and is executed via uv. All tests are located in the tests/ directory.

Run All Tests

uv run pytest

Run a specific test file

uv run pytest tests/test_models.py

Run a specific test function

uv run pytest tests/test_models.py -k test_updated_instance

Show output (prints/logs) during test execution

uv run pytest -s

💡 Make sure your virtual environment is activated and dependencies are installed with uv before running tests.


🔍 Checking a change to the save

A change to how save_graph and BluecoreGraph save a graph can be checked with a differential harness, scripts/diff_save.py. It loads the same sample of LC records with your branch and with the commit it started from, and compares what each wrote. docs/checking-save-changes.md says which changes need it and how to run it, and how to run the benchmark below side by side. What a save does with each resource is written down in docs/save-policy.md.


📊 Benchmarking save_graph

benchmarks/save_graph_bench.py persists a set of Bibframe graphs through the real save path (URI minting, resource save, linking, bf-class updates) and reports throughput (graphs/s, triples/s). With --profile it prints a cProfile hot-spot report, which is handy for finding where the save path spends its time.

It writes to a Postgres, so first start one — the Run Postgres with Docker command above works (it creates a bluecore database). Then point the benchmark at it with --database-url (or the DATABASE_URL env var); the benchmark creates the schema if it isn't there.

Run the benchmark (50 saves of the sample graphs)

uv run python benchmarks/save_graph_bench.py \
  --database-url postgresql+psycopg://airflow:airflow@localhost:5432/bluecore \
  --count 50

Print a cProfile hot-spot report

Add --profile:

uv run python benchmarks/save_graph_bench.py \
  --database-url postgresql+psycopg://airflow:airflow@localhost:5432/bluecore \
  --count 50 --profile

Other options:

  • --reset — TRUNCATE the resource tables first (don't point this at data you care about).
  • --input "<glob>" — RDF files to load (defaults to tests/data/*.jsonld); pass a directory of .rdf/.jsonld records for a larger, more representative run.

💡 Since the graph content doesn't change which code runs, reusing a few sample records many times (--count) is a fair, repeatable way to measure changes to the save path (e.g. before/after an optimization).


⬆️ Publishing to Pypi

To publish the bluecore-models to pypi, the following steps need to be taken.

  1. Update the version in pyproject.toml either in a feature branch PR or in a dedicated PR.
  2. After the PR is merged, create a tagged release using the same version (prepended with a v i.e. v0.4.2.
  3. Once the tagged release is saved, the Publish to PyPi Github Action should then publish the release to PyPi.

Metadata

Release files for bluecore-models 0.39.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for bluecore-models 0.39.1
File Size Uploaded Provenance Actions
bluecore_models-0.39.1.tar.gz 289.1 kB Without provenance Details for bluecore_models-0.39.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for bluecore-models 0.39.1
File Interpreter ABI Platform Actions
bluecore_models-0.39.1-py3-none-any.whl Python 3 none any Details for bluecore_models-0.39.1-py3-none-any.whl

Total release size: 385.6 kB

Release files / bluecore_models-0.39.1.tar.gz

Download URL bluecore_models-0.39.1.tar.gz
Size 289.1 kB
Tags Source
SHA-256 checksum
How to use checksums
c0a64ccca0cedbb626509abb5cae996bdf79f9c586e4ff1d0c3dae4e595187ea
BLAKE2b-256 checksum
How to use checksums
6889f70a6b3695ed4bda166608f25bd82175e18f8a3b2de8f6eb7a8169d8aa4c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.8 {"installer":{"name":"uv","version":"0.13.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / bluecore_models-0.39.1-py3-none-any.whl

Download URL bluecore_models-0.39.1-py3-none-any.whl
Size 96.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
41f2156c30825d953723398b157b02b7dfc81c1e5427e5976feaba0e6e1ec098
BLAKE2b-256 checksum
How to use checksums
435b170673da7eb71545a1459de1954e17013e77d592802c0961a53ded68d8f3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.8 {"installer":{"name":"uv","version":"0.13.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for bluecore-models 0.39.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.39.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page