pytrilogy 0.3.314

Declarative, typed query language that compiles to SQL.

Trilogy is a batteries-included data-productivity toolkit that accelerates SQL-based analytics with a typed, expressive language. It's great for humans - and even better for agents. Start with a single file and scale fast with a rich ecosystem, including UI and CLI tooling, public models to get started, rich python integration, and modern visuals and reporting.

Why Trilogy

SQL is the best way to work with data with but shows strain at scale and as it ages. Can we have the pros without the cons? We believe you can.

Trilogy adds a lightweight semantic layer to keep SQL fast through the full lifecycle of analytics - from exploration to production. It provides a full stack for interactive, visualization, and orchestration that can be adopted incrementally and without lock-in; start with checking types and asking agents questions; end with a more efficient and productive warehouse.

Headline features:

• No manual joins; no from clause
• Reusable models, types, and functions
• Safe refactoring across queries
• Supports all standard engines: BigQuery, DuckDB, Snowflake, Presto
• Easy to write - for humans and AI
• Built-in semantic layer without boilerplate or YAML

This repo contains pytrilogy, the reference implementation of the core language and CLI.

Install To try it out, include both the CLI and serve dependencies.

pip install pytrilogy[cli,serve]

or

uv tool install "pytrilogy[cli,serve]"

Docs and Website

[!TIP] Try it now: Open-source studio | Interactive demo | Documentation

Hello World

Trilogy includes a public model registry with fun datasets you can explore. Run the below to import, query, and explore one of these models directly.

# 1. Pull a public model (fetches all source .preql + setup.sql + trilogy.toml).
trilogy public fetch faa ./faa-demo
cd faa-demo

# Run a quick adhoc query (--import prepends the import for you — discover
# what's available with `trilogy explore flight.preql`)
trilogy run --import flight "select carrier.code, count(id) as flight_count order by flight_count desc;"

# Plot it
trilogy run --import flight "chart layer barh ( y_axis <- carrier.name, x_axis <- count(id) as flight_count ) order by flight_count desc limit 10;"

# 3. Add a derived datasource by grabbing the hosted snippet
trilogy file write reporting.preql --from-url https://raw.githubusercontent.com/trilogy-data/trilogy-public-models/refs/heads/main/examples/duckdb/faa/example.preql

# 4. Refresh — builds the managed asset declared in reporting.preql and tracks watermarks.
trilogy refresh reporting.preql

# 5. Launch the Studio UI against the live model (opens your browser) to explore + query
trilogy serve .

The snippet fetched in step 3 looks like this — copy/paste it into your editor if you'd rather author it by hand:

import flight as flight;

# derive reusable concepts
auto flight_date <- flight.dep_time::date;

# this can be properties or metrics
auto flight_count <- count(flight.id);

# datasources can be read from or written to
# use this to write to 
datasource daily_airplane_usage (
    flight_date,
    flight.aircraft.model.name,
    flight_count
)
grain(flight_date, flight.aircraft.model.name)
address daily_airplane_usage
;

Browse other available models with trilogy public list (filter with --engine duckdb or --tag benchmark). Every model in trilogy-public-models is pullable.

Principles

Versus SQL, Trilogy aims to:

Keep:

• Correctness
• Accessibility

Improve:

• Simplicity
• Refactoring and maintainability
• Reusability and composability
• Expressivness

Maintain:

• Acceptable performance

Backend Support

Backend	Status	Notes
BigQuery	Core	Full support
DuckDB	Core	Full support
Snowflake	Core	Full support
Sqlite	Core	Full support
SQL Server	Experimental	Limited testing
Presto	Experimental	Limited testing

Syntax Overview

Trilogy preql models are compositions of types, keys, and properties

Save the following code in a file named hello.preql

# semantic model is abstract from data

type word string; # types can be used to provide expressive metadata tags that propagate through dataflow

key sentence_id int;
property sentence_id.word_one string::word; # comments after a definition 
property sentence_id.word_two string::word; # are syntactic sugar for adding
property sentence_id.word_three string::word; # a description to it

# comments in other places are just comments

# define our datasource to bind the model to data
# for most work, you can import something already defined
# testing using query fixtures is a common pattern
datasource word_one(
    sentence: sentence_id,
    word:word_one
)
grain(sentence_id)
query '''
select 1 as sentence, 'Hello' as word
union all
select 2, 'Bonjour'
''';

datasource word_two(
    sentence: sentence_id,
    word:word_two
)
grain(sentence_id)
query '''
select 1 as sentence, 'World' as word
union all
select 2 as sentence, 'World'
''';

datasource word_three(
    sentence: sentence_id,
    word:word_three
)
grain(sentence_id)
query '''
select 1 as sentence, '!' as word
union all
select 2 as sentence, '!'
''';

def concat_with_space(x,y) -> x || ' ' || y;

# an actual select statement
# joins are automatically resolved between the 3 sources
with sentences as
select sentence_id, @concat_with_space(word_one, word_two) || word_three as text;

WHERE 
    sentences.sentence_id in (1,2)
SELECT
    sentences.text
;

Run it:

trilogy run hello.preql duckdb

Python SDK Intro

Use the python SDK to embed Trilogy in larger python workflows.

A BigQuery example, similar to the BigQuery quickstart:

from trilogy import Dialects, Environment

environment = Environment()

environment.parse('''
key name string;
key gender string;
key state string;
key year int;
key yearly_name_count int; int;

datasource usa_names(
    name:name,
    number:yearly_name_count,
    year:year,
    gender:gender,
    state:state
)
address `bigquery-public-data.usa_names.usa_1910_2013`;
''')

executor = Dialects.BIGQUERY.default_executor(environment=environment)

results = executor.execute_text('''
WHERE
    name = 'Elvis'
SELECT
    name,
    sum(yearly_name_count) -> name_count 
ORDER BY
    name_count desc
LIMIT 10;
''')

# multiple queries can result from one text batch
for row in results:
    # get results for first query
    answers = row.fetchall()
    for x in answers:
        print(x)

LLM Usage

Connect to your favorite provider and generate queries with confidence.

from trilogy import Environment, Dialects
from trilogy.ai import Provider, text_to_query
import os

executor = Dialects.DUCK_DB.default_executor(
    environment=Environment(working_path=Path(__file__).parent)
)

api_key = os.environ.get(OPENAI_API_KEY)
if not api_key:
    raise ValueError("OPENAI_API_KEY required for gpt generation")
# load a model
executor.parse_file("flight.preql")
# create tables in the DB if needed
executor.execute_file("setup.sql")
# generate a query
query = text_to_query(
    executor.environment,
    "number of flights by month in 2005",
    Provider.OPENAI,
    "gpt-5.5",
    api_key,
)

# print the generated trilogy query
print(query)
# run it
results = executor.execute_text(query)[-1].fetchall()
assert len(results) == 12

for row in results:
    # all monthly flights are between 5000 and 7000
    assert row[1] > 5000 and row[1] < 7000, row

CLI Usage

Trilogy can be run through a CLI tool, also named 'trilogy'.

Basic syntax:

trilogy run <cmd or path to trilogy file> <dialect>

With backend options:

trilogy run "key x int; datasource test_source(i:x) grain(x) address test; select x;" duckdb --path <path/to/database>

Format code:

trilogy fmt <path to trilogy file>

Browse and pull public models:

trilogy public list [--engine duckdb] [--tag benchmark]
trilogy public fetch <model-name> [<dir>] [--no-examples]

Fetches model source files, setup scripts, and a ready-to-use trilogy.toml from trilogy-public-models into a local directory so you can immediately refresh and serve it.

Managing workspace files from the CLI

trilogy file has shell-agnostic CRUD operations on the filesystem.

trilogy file list .                      # list entries (-r for recursive, -l for size)
trilogy file read reporting.preql        # dump contents to stdout
trilogy file write path --content "..."  # create/overwrite from a string
trilogy file write path --from-file src  # copy from a local file
trilogy file write path --from-url URL   # fetch from http(s):// or file:// URL
trilogy file delete path --recursive     # remove a file or directory
trilogy file move old.preql new.preql    # rename within a backend
trilogy file exists path                 # exit 0 if present, 1 otherwise

Backend Configuration

BigQuery:

• Uses applicationdefault authentication (TODO: support arbitrary credential paths)
• In Python, you can pass a custom client

DuckDB:

• --path - Optional database file path

Postgres:

• --host - Database host
• --port - Database port
• --username - Username
• --password - Password
• --database - Database name

Snowflake:

• --account - Snowflake account
• --username - Username
• --password - Password

Config Files

The CLI can pick up default configuration from a config file in the toml format. Detection will be recursive form parent directories of the current working directory, including the current working directory.

This can be used to set

• default engine and arguments
• parallelism for execute for the CLI
• any startup commands to run whenever creating an executor.

# Trilogy Configuration File
# Learn more at: https://github.com/trilogy-data/pytrilogy

[engine]
# Default dialect for execution
dialect = "duck_db"

# Parallelism level for directory execution
# parallelism = 2

# Connection parameters for the dialect; ${env:VAR} reads an environment variable
[engine.config]
# db_location = "local.duckdb"

# Startup scripts to run before execution
[setup]
# startup_trilogy = []
sql = ['setup/setup_dev.sql']

trilogy init [path] [dialect] scaffolds this file, alongside root/ (where trilogy ingest writes generated models by default), jobs/, and an example script. Passing a dialect (trilogy init . bigquery) pins engine.dialect and stubs out that dialect's connection parameters in [engine.config]. Init refuses to run over an existing trilogy.toml; --force overwrites it and leaves every other file untouched.

More Resources

• Interactive demo
• Public model repository - Great place for modeling examples
• Full documentation

Python API Integration

Root Imports

Are stable and should be sufficient for executing code from Trilogy as text.

from pytrilogy import Executor, Dialect

Authoring Imports

Are also stable, and should be used for cases which programatically generate Trilogy statements without text inputs or need to process/transform parsed code in more complicated ways.

from pytrilogy.authoring import Concept, Function, ...

Contributing

Clone repository and install requirements.txt and requirements-test.txt.

Please open an issue first to discuss what you would like to change, and then create a PR against that issue.

Similar Projects

Trilogy combines two aspects: a semantic layer and a query language. Examples of both are linked below:

Semantic layers - tools for defining a metadata layer above SQL/warehouse to enable higher level abstractions:

• MetricFlow
• Cube
• Zillion

Better SQL has been a popular space. We believe Trilogy takes a different approach than the following, but all are worth checking out. Please open PRs/comment for anything missed!

• Malloy
• Preql
• PRQL