Skip to main content

NLQueries

nlqueries-core

CI PyPI Python License: BSL 1.1 Visitors

NLQueries Core is an open-source natural language to SQL engine — see nlqueries.com. It turns plain-English questions into validated SQL, builds a self-updating YAML knowledge base from your schema and query history, and exposes everything as an MCP server your AI assistant can call directly. It also answers questions from your documents (PDF, Word, Excel, Notion, Confluence) and can blend both in a single hybrid answer.


Features

Capability Description
Database connectors PostgreSQL, MySQL, Snowflake, BigQuery, Redshift, SQL Server / Azure SQL, DuckDB, SQLite — plus a generic SQLAlchemy connector for any other SQLAlchemy-reachable database, driven by a connection URL
Document connectors PDF, Word, Excel, Notion, Confluence — ask questions over ingested documents with citations
Query pipeline Filter, cluster, and parameterize query history into reusable QueryCapsule templates
Knowledge base Auto-generated YAML schema + capsule file, with coverage reporting via kb-stats
Multi-agent orchestration Routes each question to a SQL agent, document agent, or both in parallel (hybrid)
Conversational follow-ups Carries context across questions so a follow-up like "and by region?" resolves against the previous query — on by default in nlqueries query, reset with --new-session
Semantic cache Returns previously-answered similar questions in under 50 ms, no LLM or DB round-trip
Embedding daemon Keeps the embedding model resident in memory — ~10 ms per call instead of ~9 s
Feedback loop Rate an answer with feedback, review with feedback-stats, and promote-feedback the thumbs-up ones — their SQL is re-validated against the current schema and blended into later prompts as verified examples
Regression checks eval re-asks the agent's own mined capsules — and a golden question set too, with --golden — then checks the generated SQL parses and references only tables the agent knows
dbt import import-dbt merges dbt model and column descriptions into the knowledge base
LLM client Anthropic, OpenAI, Amazon Bedrock, or any LiteLLM-supported provider
MCP server Query execution and schema/knowledge lookup exposed as MCP tools for Claude, Cursor, etc.
CLI nlqueries (or the shorter nlq alias) — connect, build, query, and inspect from your terminal

See docs/architecture.md (or read online) for how these pieces fit together.


Quickstart

Prerequisite: Python 3.11–3.14.

Pulls the published nlqueries/core image from Docker Hub — no clone required, just the compose file:

curl -O https://raw.githubusercontent.com/nlqueries/nlqueries/main/docker-compose.yml

Create a .env file next to it with one LLM key, a Qdrant key and an MCP token. All three are required — compose refuses to start without QDRANT_API_KEY or NLQ_MCP_STATIC_TOKEN:

ANTHROPIC_API_KEY=sk-ant-...
# or OPENAI_API_KEY=sk-...
QDRANT_API_KEY=...         # any random secret: openssl rand -hex 32
NLQ_MCP_STATIC_TOKEN=...   # another one: openssl rand -hex 32

The MCP server is reachable over the network, so it requires that token: an MCP client connects to http://localhost:8080/sse and sends Authorization: Bearer <your NLQ_MCP_STATIC_TOKEN>. The token has full access. To give different people different access, see docs/mcp-authentication.md.

Then start the stack:

docker compose up

This pulls nlqueries/core:latest and starts it alongside Qdrant (:6333), with the MCP server on :8080. Run CLI commands against the running stack from a second terminal:

docker exec -it nlqueries-core nlqueries health

Option B — pip install

pip install nlqueries-core
export ANTHROPIC_API_KEY=sk-ant-...   # or OPENAI_API_KEY
nlqueries health

Optional extras for specific connectors:

pip install "nlqueries-core[mysql]"      # MySQL
pip install "nlqueries-core[snowflake]"  # Snowflake
pip install "nlqueries-core[bigquery]"   # Google BigQuery
pip install "nlqueries-core[redshift]"   # Amazon Redshift
pip install "nlqueries-core[mssql]"      # SQL Server / Azure SQL
pip install "nlqueries-core[duckdb]"     # DuckDB
pip install "nlqueries-core[docs]"       # PDF / Word / Excel ingestion
pip install "nlqueries-core[wiki]"       # Notion / Confluence sync

PostgreSQL and SQLite need no extra. The generic sqlalchemy connector needs whichever driver its URL names.

Option C — Clone and install from source

No Docker required — for contributing, or to run against unreleased changes:

git clone https://github.com/nlqueries/nlqueries.git
cd nlqueries
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\Activate.ps1
pip install -e ".[dev]"
export ANTHROPIC_API_KEY=sk-ant-...   # or OPENAI_API_KEY
nlqueries health

See CONTRIBUTING.md for linting and test commands.

First query

nlqueries connect postgres --host localhost --database mydb --user alice --password secret --alias dev
nlqueries process-history dev --days 30 --annotate
nlqueries export-kb dev
nlqueries query dev "How many orders shipped last month?"
nlqueries query dev "and how many were returned?"   # follow-up — keeps prior context

Follow-up context is on by default; pass --new-session to start fresh or --no-session to disable it for a one-off question.

Full walkthrough: docs/getting-started.md (or read online).


Documentation

Doc Covers
docs/getting-started.md (or read online) Step-by-step setup and your first query
docs/cli-reference.md (or read online) Every command and flag
docs/connectors.md (or read online) Database and document connector setup, per-connector notes and caveats
docs/configuration.md (or read online) Environment variables
docs/troubleshooting.md (or read online) Common warnings and errors explained
docs/qdrant-setup.md (or read online) Setting up Qdrant (required for embeddings, semantic cache, document search)
docs/mcp-authentication.md (or read online) Authenticating the MCP server — required to serve it over a network
docs/architecture.md (or read online) Module layout and request flow
docs/database-hardening.md (or read online) The read-only role to grant per engine, and exactly what each connector does and does not enforce
docs/dependency-locking.md (or read online) The hashed lock file the image builds from, the digest-pinned base image, and how to regenerate both
docs/contributing.md (or read online) What a new connector has to implement, and what to open an issue about first

Contributing

See CONTRIBUTING.md. All contributors must sign the CLA before a PR can be merged — see CONTRIBUTOR_LICENSE_AGREEMENT.md.


License

Business Source License 1.1 — converting to Apache 2.0 no later than June 4, 2030. The licence sets that date and also converts each version on the fourth anniversary of its own first publication, whichever comes first, so a version first published before June 4, 2026 converts on its own anniversary, ahead of the fixed date.

Until then the licence grants production use, including commercially and self-hosted, with one carve-out: you may not offer NLQueries Core to third parties as a hosted or managed service. Individual and non-commercial self-hosted deployments are unrestricted. LICENSE is the authority; this paragraph is a summary.

Metadata

Release files for nlqueries-core 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for nlqueries-core 0.3.0
File Size Uploaded
nlqueries_core-0.3.0.tar.gz 977.6 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for nlqueries-core 0.3.0
File Interpreter ABI Platform
nlqueries_core-0.3.0-py3-none-any.whl Python 3 none any Details

Total release size: 1.4 MB

Release files / nlqueries_core-0.3.0.tar.gz

Download URL nlqueries_core-0.3.0.tar.gz
Size 977.6 kB
Tags Source
SHA-256 checksum
How to use checksums
9c9e46211a13271ba6e050836abfeae4398013e99fa0809fdbd3e906b5f593b9
BLAKE2b-256 checksum
How to use checksums
629c58032f4d84eddc63c0d7b3dc4ad1f1dced07d1f953a4b1d7aeba40c7df0d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 2, 2026.

Transparency log

Release files / nlqueries_core-0.3.0-py3-none-any.whl

Download URL nlqueries_core-0.3.0-py3-none-any.whl
Size 405.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ff1d9e7e3fdbde207774bc9e644aac1d77b555cb765c68674eac9606b833f3bc
BLAKE2b-256 checksum
How to use checksums
b9ee7925dcc7266cb77152dfd14dd63723087727e07312f399cfac1557493153
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 2, 2026.

Transparency log

Release history Release notifications | RSS feed

This release

0.3.0 This release

2 release files

0.2.0

2 release files

0.1.0

2 release files

0.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page