omop-lite

A small container to get an OMOP CDM database running quickly, with support for PostgreSQL, SQL Server, and DuckDB.

Drop your data into data/, and run the container.

Configuration

You can configure the container or CLI using the following environment variables:

• DB_HOST: The hostname of the database. Default is db.
• DB_PORT: The port number of the database. Default is 5432.
• DB_USER: The username for the database. Default is postgres.
• DB_PASSWORD: The password for the database. Default is password.
• DB_NAME: The name of the database. Default is omop. For the duckdb dialect, this is instead the path to the .duckdb file to create/use, e.g. /data/omop.duckdb.
• DIALECT: The type of database to use. Default is postgresql, but can also be mssql or duckdb.
• OMOP_VERSION: Version of the OMOP-CDM schema to load. Default is omop5_4, but can also be omop5_3.
• SCHEMA_NAME: The name of the schema to be created/used in the database. Default is public.
• DATA_DIR: The directory containing the data CSV files. Default is data.
• SYNTHETIC: Load synthetic data (boolean). Default is false
• SYNTHETIC_NUMBER: Size of synthetic data, 100 or 1000. Default is 100.
• DELIMITER: The delimiter used to separate data. Default is tab, can also be ,

Usage

CLI

pip install omop-lite python omop-lite --help

Docker

docker run -v ./data:/data ghcr.io/health-informatics-uon/omop-lite

# docker-compose.yml
services:
  omop-lite:
    image: ghcr.io/health-informatics-uon/omop-lite
    volumes:
      - ./data:/data
    depends_on:
      - db

  db:
    image: postgres:latest
    environment:
      - POSTGRES_DB=omop
      - POSTGRES_PASSWORD=password
    ports:
      - "5432:5432"

Helm

To install using Helm:

# Add the Helm repository
helm install omop-lite oci://ghcr.io/health-informatics-uon/charts/omop-lite --version 0.2.2

The Helm chart deploys OMOP Lite as a Kubernetes Job that creates an OMOP CDM in a database. You can customise the installation using a values file:

# values.yaml
env:
  dbHost: postgres
  dbPort: "5432"
  dbUser: postgres
  dbPassword: postgres
  dbName: omop_helm
  dialect: postgresql
  schemaName: public
  synthetic: "false" 

Install with custom values:

helm install omop-lite omop-lite/omop-lite -f values.yaml

DuckDB

Unlike PostgreSQL/SQL Server, DuckDB is file-based rather than a server you connect to - running omop-lite with DIALECT=duckdb creates a single .duckdb file, pre-loaded with the OMOP CDM schema and either synthetic or your own data, which you can then open directly with the DuckDB CLI, Python, R, or a notebook.

docker run -v ./data:/data -e DIALECT=duckdb -e DB_NAME=/data/omop.duckdb -e SYNTHETIC=true ghcr.io/health-informatics-uon/omop-lite

or with the bundled compose file:

docker compose --profile duckdb up

DB_NAME is repurposed as the output file path for this dialect (see Configuration above) - point it at a path under a mounted volume so the file persists after the container exits. DB_HOST/DB_PORT/DB_USER/DB_PASSWORD are ignored.

DuckDB does not support adding foreign keys to an existing table (only primary keys can be added after creation), so foreign key constraints are skipped for this dialect - primary keys and indices are still created as normal.

Synthetic Data

If you need synthetic data, some is provided in the synthetic directory. It provides a small amount of data to load quickly. To load the synthetic data, run the container with the SYNTHETIC environment variable set to true.

• 100 is fake data
• 1000 is Synthea 1k data.
• 1001 is Synthea 1k data but with Specimen, Death, Device Exposure added in

Bring Your Own Data

You can provide your own data for loading into the tables by placing your files in the data/ directory. This should contain .csv files matching the data tables (DRUG_STRENGTH.csv, CONCEPT.csv, etc.).

To match the vocabulary files from Athena, this data should be tab-separated, but as a .csv file extension. You can override the delimiter with DELIMITER configuration.

Text search OMOP

Full-text search

Adding a tsvector column to the concept table and an index on that column makes full-text search queries on the concept table run much faster.

Vector search

Postgres does vector search too!

Enabling text search

To enable these features in omop-lite, you can use the text-search profile

docker compose --profile text-search up

To do this, you need to have text-search/embeddings.parquet, containing concept_ids and embeddings (an example file is provided). This uses pgvector to create an embeddings table.

Testing

If you're a developer and want to iterate on omop-lite quickly, there's a small subset of the vocabularies sufficient to build in synthetic/. If you wish to test the vector search, there are matching embeddings in embeddings/embeddings.parquet.