omop-lite

A small container to get an OMOP CDM database running quickly, with support for PostgreSQL, SQL Server, and DuckDB.

Drop your data into data/, and run the container.

Configuration

You can configure the container or CLI using environment variables, or the equivalent CLI flag. A CLI flag always overrides its environment variable.

--fts-create/FTS_CREATE is not currently functional. For full-text and vector search, use the text-search Compose profile described in Text search OMOP.

Usage

CLI

Install the package, which provides the omop-lite command:

pip install omop-lite
omop-lite --help

Running omop-lite with no subcommand runs the full pipeline: it creates the schema (if needed), creates the tables, loads the data, and adds constraints. This is the same thing the Docker image and Helm chart run by default.

# Quick start with bundled synthetic data
omop-lite --synthetic

Commands

Besides the default pipeline, omop-lite has subcommands for running each step on its own - useful for custom workflows, or recovering partway through a failed run:

Every subcommand accepts the database connection options from the table above (--db-host, --db-port, --db-user, --db-password, --db-name, --schema-name, --dialect, --log-level). --omop_version is also accepted by every subcommand except test and drop, since those two don't touch version-specific SQL. load-data additionally accepts --synthetic, --synthetic-number, --data-dir, and --delimiter; drop additionally accepts --tables-only, --schema-only, and --confirm. Run omop-lite <command> --help to see a command's exact options.

For example, to set up a database step by step instead of running the full pipeline at once:

omop-lite test                    # check the connection first
omop-lite create-tables
omop-lite load-data --synthetic
omop-lite add-constraints

Or to reload data without recreating the schema:

omop-lite drop --tables-only --confirm
omop-lite create-tables
omop-lite load-data

Docker

docker run -v ./data:/data ghcr.io/health-informatics-uon/omop-lite

# docker-compose.yml
services:
  omop-lite:
    image: ghcr.io/health-informatics-uon/omop-lite
    volumes:
      - ./data:/data
    depends_on:
      - db

  db:
    image: postgres:latest
    environment:
      - POSTGRES_DB=omop
      - POSTGRES_PASSWORD=password
    ports:
      - "5432:5432"

Helm

To install using Helm:

# Add the Helm repository
helm install omop-lite oci://ghcr.io/health-informatics-uon/charts/omop-lite --version 0.2.2

The Helm chart deploys OMOP Lite as a Kubernetes Job that creates an OMOP CDM in a database. You can customise the installation using a values file:

# values.yaml
env:
  dbHost: postgres
  dbPort: "5432"
  dbUser: postgres
  dbPassword: postgres
  dbName: omop_helm
  dialect: postgresql
  schemaName: public
  synthetic: "false" 

Install with custom values:

helm install omop-lite omop-lite/omop-lite -f values.yaml

DuckDB

Unlike PostgreSQL/SQL Server, DuckDB is file-based rather than a server you connect to - running omop-lite with DIALECT=duckdb creates a single .duckdb file, pre-loaded with the OMOP CDM schema and either synthetic or your own data, which you can then open directly with the DuckDB CLI, Python, R, or a notebook.

docker run -v ./data:/data -e DIALECT=duckdb -e DB_NAME=/data/omop.duckdb -e SYNTHETIC=true ghcr.io/health-informatics-uon/omop-lite

or with the bundled compose file:

docker compose --profile duckdb up

DB_NAME is repurposed as the output file path for this dialect (see Configuration above) - point it at a path under a mounted volume so the file persists after the container exits. DB_HOST/DB_PORT/DB_USER/DB_PASSWORD are ignored.

DuckDB does not support adding foreign keys to an existing table (only primary keys can be added after creation), so foreign key constraints are skipped for this dialect - primary keys and indices are still created as normal.

Synthetic Data

If you need synthetic data, some is provided in the synthetic directory. It provides a small amount of data to load quickly. To load the synthetic data, run the container with the SYNTHETIC environment variable set to true.

• 100 is fake data
• 1000 is Synthea 1k data.
• 1001 is Synthea 1k data but with Specimen, Death, Device Exposure added in

Bring Your Own Data

You can provide your own data for loading into the tables by placing your files in the data/ directory. This should contain .csv files matching the data tables (DRUG_STRENGTH.csv, CONCEPT.csv, etc.).

To match the vocabulary files from Athena, this data should be tab-separated, but as a .csv file extension. You can override the delimiter with DELIMITER configuration.

Text search OMOP

Full-text search

Adding a tsvector column to the concept table and an index on that column makes full-text search queries on the concept table run much faster.

Vector search

Postgres does vector search too!

Enabling text search

To enable these features in omop-lite, you can use the text-search profile

docker compose --profile text-search up

To do this, you need to have text-search/embeddings.parquet, containing concept_ids and embeddings (an example file is provided). This uses pgvector to create an embeddings table.

Testing

If you're a developer and want to iterate on omop-lite quickly, there's a small subset of the vocabularies sufficient to build in synthetic/. If you wish to test the vector search, there are matching embeddings in embeddings/embeddings.parquet.