Quail

Quail (QUery Aware Inference Layer) is an open-source, extensible execution engine for AI-SQL, being developed at Full Stack Data Lab at CMU.

AI-SQL is a variant of SQL with operators that let you write logic in natural language for an LLM to evaluate on every row.

SELECT r.id
FROM reviews r
WHERE AI.IF(PROMPT('Does this review discuss the ending?\n\n{0}', r.body))

Documentation | Quickstart | QUAIL-B

Install

uv pip install quail-engine

Requires Python 3.12 and a CUDA GPU.

Example

import pyarrow as pa
import quail

reviews = pa.table({
    "id": ["r1", "r2", "r3"],
    "body": [
        "A beautiful film with outstanding performances.",
        "Terrible pacing and a nonsensical plot.",
        "The cinematography was stunning, though the story dragged.",
    ],
})

config = quail.EngineConfig(model="qwen3-4b-fp8", device="h100-sxm")
with quail.Session(config=config) as session:
    session.register("reviews", quail.DocumentProvider.from_table(reviews, id_col="id"))
    result = session.sql("""
        SELECT r.id
        FROM reviews r
        WHERE AI.IF(PROMPT(
            'Does this review mention a positive aspect of the movie?\n\n{0}',
            r.body))
    """, dialect="bq").collect()
    print(result)

Quail Server

Quail Server is an optional HTTP server that runs on the machine with the GPU. You start it once, send queries to it with endpoint, and the query keeps running after the client disconnects. submit() returns once the server has saved the record, and get_run reads that record later.

uv pip install "quail-engine[server]"
quail-server

import pyarrow as pa
import quail

reviews = pa.table({
    "id": ["r1", "r2"],
    "body": ["The ending was excellent.", "I liked the soundtrack."],
})
sql = """
    SELECT r.id
    FROM reviews r
    WHERE AI.IF(PROMPT('Does this discuss the ending? {0}', r.body))
"""
config = quail.EngineConfig(model="qwen3-4b-fp8", device="h100-sxm")

with quail.Session(
    config=config,
    endpoint="http://127.0.0.1:8642",
) as session:
    session.register(
        "reviews",
        quail.DocumentProvider.from_table(reviews, id_col="id"),
    )
    run = session.sql(sql, dialect="bq").submit()
    for status in run.watch():
        print(status.phase["message"])
    table = run.result().collect()
    print(table)

See Quail Server.

Supported operators

Quail currently supports AI-powered filters, joins, and EXISTS / NOT EXISTS. We are actively adding more operators (AI.CLASSIFY, AI.EXTRACT, AI.MAP).

We support two AI-SQL dialects: Snowflake AI_FILTER and BigQuery AI.IF, plus a Python builder API.

Supported models and GPUs

1, 2, 4, or 8 GPUs per query. We are actively adding more models and hardware.

Development

uv run ruff check quail tests experiments tools
uv run python tools/check_long_strings.py
uv run vulture
uv run pytest -q

Contributing

See the contributing guide for how to propose and submit changes.