PipeQL

Pipelined · Injection-Safe · Polyglot Query Language

📚 Docs & Live Playground · 💿 Install · 🧠 Syntax Reference · 🔌 SDK Usage · ⬇️ Download Binaries

---

PipeQL is a compiled query language that transpiles to parameterized SQL. You write clean, left-to-right pipelines; the compiler extracts every value into bind parameters at the AST level — making SQL injection mathematically impossible. One query, four databases: PostgreSQL, SQLite, DuckDB, and MySQL.

from orders
| join customers on orders.customer_id == customers.id
| filter orders.status == 'active' and orders.total >= $min
| group [region] (total = sum(orders.total), cnt = count(*))
| filter total > $threshold
| select [region, total, cnt]
| sort [total desc]
| take 10

→ PostgreSQL output:

SELECT region, SUM(orders.total) AS total, COUNT(*) AS cnt
FROM orders
INNER JOIN customers ON (orders.customer_id = customers.id)
WHERE ((orders.status = $1) AND (orders.total >= $2))
GROUP BY region HAVING (SUM(orders.total) > $3)
ORDER BY total DESC LIMIT 10;
-- Parameters: [$1='active', $2=min, $3=threshold]

Every string literal and $param is extracted into a positional bind array at the AST level. No string concatenation. No escaping. No injection.

---

Why PipeQL?

Problem	PipeQL Solution
🛡️ SQL injection	100% AST-level parameter isolation — impossible to inject
🔀 Dialect lock-in	Write once, compile to Postgres / SQLite / DuckDB / MySQL
↔️ Right-to-left SQL	Clean left-to-right pipeline: from → filter → select → sort
🐌 Slow template engines	Native Rust compiler, ~19µs per query
🌍 Language silos	One compiler, 5 SDKs: Rust, JS/TS, Python, C/C++, Go
🧩 Fragmented tooling	Built-in LSP, VS Code extension, tree-sitter grammar, CLI

---

💿 Install

Rust — CLI & Library

cargo install pipeql-cli          # CLI tool

# Cargo.toml
[dependencies]
pipeql-core = "1.1.6"

JavaScript / TypeScript

npm install @flaxmbot/pipeql

Python

pip install pipeql

Go

⚠️ Prerequisites: the Go binding uses CGO and needs the libpipeql_cffi shared library on your system. It is a library, not a command — use go get (inside a Go module), not go install.

# 1. Build the shared library
git clone https://github.com/Flaxmbot/PipeQL.git && cd PipeQL
cargo build --release -p pipeql-cffi

# 2. Install the shared library
# Linux:
sudo cp target/release/libpipeql_cffi.so /usr/local/lib/ && sudo ldconfig
# macOS:
sudo cp target/release/libpipeql_cffi.dylib /usr/local/lib/
# Windows: copy target/release/pipeql_cffi.dll next to your binary (or add to PATH)

# 3. Add the module — from INSIDE your Go project (any dir with a go.mod):
go get github.com/Flaxmbot/PipeQL/go@latest

#    Or pin a specific release:
#    go get github.com/Flaxmbot/PipeQL/go@v1.1.6

If you see go.mod file not found you are outside a Go module — run go mod init <yourmodule> first, or run the go get from a project that already has a go.mod.

C / C++

Build the shared library, then link against the header:

cargo build --release -p pipeql-cffi
# Header:  crates/pipeql-cffi/include/libpipeql.h
# Library: target/release/libpipeql_cffi.{so,dylib,dll}

gcc demo.c -I./crates/pipeql-cffi/include -L./target/release -lpipeql_cffi -o demo

C package managers: for distributing the C SDK, vcpkg (Microsoft; cross-platform, CMake-native, 2400+ ports) and Conan (decentralized, custom remotes, great for versioned dependency graphs) are the two leading choices. vcpkg is the easiest first step — create a port in the ports/ dir, and vcpkg install pipeql works on Windows/macOS/Linux out of the box.

Pre-built Binaries

Download from GitHub Releases:

Platform	CLI	Shared Library
Linux x64	pipeql-linux-x86_64	libpipeql_cffi.so
macOS x64	pipeql-macos-x86_64	libpipeql_cffi.dylib
Windows x64	pipeql-windows-x86_64.exe	pipeql_cffi.dll

Each release also ships: @flaxmbot/pipeql npm tarball, pipeql PyPI wheel + sdist, pipeql-core crate file, the C SDK header tarball, changelog, and the AI system prompt.

---

🧠 Syntax Reference

Keywords

All PipeQL reserved keywords:

Keyword	Purpose	Example
from	Source table for reads, updates, deletes	from users
into	Target table for inserts and upserts	into users
as	Alias a table or column	from users as u, select [name as n]
filter	Row filtering (WHERE / HAVING)	filter age >= 18
select	Choose output columns	select [id, name]
derive	Add computed columns	derive [total = price * qty]
join	Inner join	join orders on users.id == orders.uid
left	Left outer join modifier	left join orders on ...
right	Right outer join modifier	right join roles on ...
full	Full outer join modifier	full join archive on ...
inner	Explicit inner join modifier	inner join orders on ...
on	Join condition	join t on a.id == t.id
group	Group by with aggregates	group [region] (total = sum(amt))
sort	Order results	sort [created_at desc]
take	Limit rows	take 25
skip	Offset rows	skip 50
insert	Insert values	insert [name = $name]
update	Update values (requires filter)	update [name = $name]
update all	Update every row (explicit opt-in)	update all [plan = 'free']
delete	Delete rows (requires filter)	delete
delete all	Delete every row (explicit opt-in)	delete all
upsert	Insert-or-update values	upsert [id = $id, name = $n]
conflict	Conflict target columns for upsert	conflict [id]
do	Conflict action for upsert	do update [name = $n]
union	Combine result sets (distinct)	... | union ...
all	Include duplicates in union	... | union all ...
table	Create a table (DDL)	table users [...]
and	Logical AND	filter a == 1 and b == 2
or	Logical OR	filter a == 1 or b == 2
not	Logical NOT / negation	filter not active
in	Set membership test	filter id in (1, 2, 3)
is	Null check	filter name is null
null	Null literal	filter name is not null
true	Boolean true	filter active == true
false	Boolean false	filter active == false
asc	Sort ascending	sort [name asc]
desc	Sort descending	sort [created_at desc]

Pipeline Stages

Every PipeQL query starts with a source table and chains stages with |:

from <table> [as <alias>]
| <stage>
| <stage>
| ...

Stage	Syntax	SQL Equivalent
from	from users	FROM users
from (alias)	from users as u	FROM users u
filter	filter age >= 18 and active == true	WHERE age >= 18 AND active = TRUE
select	select [id, name, email]	SELECT id, name, email
select (alias)	select [full_name as name]	SELECT full_name AS name
select (star)	select [*]	SELECT *
derive	derive [total = price * qty]	SELECT *, (price * qty) AS total
join	join orders on users.id == orders.user_id	INNER JOIN orders ON ...
left join	left join orders on users.id == orders.uid	LEFT JOIN orders ON ...
right join	right join roles on users.role_id == roles.id	RIGHT JOIN roles ON ...
full join	full join archive on a.id == archive.id	FULL JOIN archive ON ...
group	group [region] (total = sum(amount))	GROUP BY region
sort	sort [created_at desc, name asc]	ORDER BY created_at DESC, name ASC
take	take 25	LIMIT 25
skip	skip 50	OFFSET 50

Parameters

Parameters are auto-extracted from the query and converted to dialect-specific placeholders:

Syntax	Description	Postgres	SQLite / DuckDB / MySQL
$name	Named parameter	$1	?
${name}	Braced parameter	$1	?
'literal'	String literal (auto-extracted)	$1	?

from users | filter email == $email and role == 'admin'
-- Postgres: WHERE (email = $1) AND (role = $2)  → params: ["email", "admin"]
-- SQLite:   WHERE (email = ?) AND (role = ?)    → params: ["email", "admin"]

Expressions & Operators

Category	Operators	Example
Comparison	== != < <= > >=	filter price >= 10
Logical	and or not	filter a == 1 and not b
Null checks	is null is not null	filter name is not null
Set membership	in (...) not in (...)	filter id in (1, 2, 3)
Subquery	in (from ... | select ...)	filter id in (from t | select [id])
Arithmetic	+ - * /	derive [total = price * qty]
Functions	count(*) sum() avg() min() max() coalesce()	group [r] (n = count(*))
Column ref	table.column	filter users.id == orders.uid
Literals	integers, floats, strings, booleans, null	42 3.14 'text' true null

Mutations (DML)

Insert

into users | insert [name = $name, email = $email]

→ INSERT INTO users (name, email) VALUES ($1, $2) RETURNING *;

Update

from users | filter id == $id | update [name = $name, email = $email]

→ UPDATE users SET name = $1, email = $2 WHERE (id = $3);

⚠️ update requires a preceding filter stage — PipeQL enforces this to prevent accidental mass updates.

🔓 Escape hatch: to deliberately update every row, write update all [...] explicitly:

from users | update all [plan = 'free']

→ UPDATE users SET plan = $1; (no WHERE)

If a filter is present alongside all, the WHERE clause still applies.

Delete

from users | filter id == $id | delete

→ DELETE FROM users WHERE (id = $1);

⚠️ delete requires a preceding filter stage — same safety enforcement as update.

🔓 Escape hatch: to deliberately clear a table, write delete all explicitly:

from users | delete all

→ DELETE FROM users; (no WHERE)

If a filter is present alongside all, the WHERE clause still applies.

Upsert (Insert or Update on Conflict)

into users
| upsert [id = $id, name = $name, email = $email]
| conflict [id]
| do update [name = $name, email = $email]

Dialect	Output
Postgres / SQLite / DuckDB	INSERT INTO users (...) VALUES (...) ON CONFLICT (id) DO UPDATE SET name = $4, email = $5;
MySQL	INSERT INTO users (...) VALUES (...) ON DUPLICATE KEY UPDATE name = VALUES(name), email = VALUES(email);

Union

from active_users | select [id, name]
| union
from archived_users | select [id, name]

→ SELECT id, name FROM active_users UNION SELECT id, name FROM archived_users;

Use union all to include duplicates.

Subqueries

from orders
| filter customer_id in (from vip_customers | select [id])
| select [order_id, total]

→ SELECT order_id, total FROM orders WHERE customer_id IN (SELECT id FROM vip_customers);

DDL (Table Schema)

table users [
  id int primary auto,
  name string not null,
  email string not null unique,
  active bool default true,
  created_at timestamp default '2024-01-01'
]

Type	Aliases	Column Modifiers
int	integer	primary, auto, unique, not null, default <value>
float	real	primary, auto, unique, not null, default <value>
string	text	primary, auto, unique, not null, default <value>
bool	boolean	primary, auto, unique, not null, default <value>
timestamp	datetime	primary, auto, unique, not null, default <value>

Type Mapping Across Dialects

PipeQL Type	PostgreSQL	SQLite	DuckDB	MySQL
int / integer	INTEGER	INTEGER	INTEGER	INT
float / real	DOUBLE PRECISION	REAL	DOUBLE	DOUBLE
string / text	TEXT	TEXT	VARCHAR	VARCHAR(255)
bool / boolean	BOOLEAN	INTEGER	BOOLEAN	BOOLEAN
timestamp / datetime	TIMESTAMP	DATETIME	TIMESTAMP	TIMESTAMP

Comments

-- This is a line comment
from users | select [id, name]  -- inline comment

Comments are preserved in the lossless AST for IDE tooling.

---

🔌 SDK Usage

Rust

use pipeql_core::api;

let result = api::compile("from users | filter id == $id | select [name]", "postgres").unwrap();
println!("{}", result.sql);      // SELECT name FROM users WHERE (id = $1);
println!("{:?}", result.params); // ["id"]

JavaScript / TypeScript

import { compile } from '@flaxmbot/pipeql';

const { sql, params } = compile(
  "from notes | filter category == $cat | sort [updated_at desc]",
  "sqlite"
);
console.log(sql);    // SELECT * FROM notes WHERE (category = ?) ORDER BY updated_at DESC;
console.log(params); // ["cat"]

Driver adapters — zero-boilerplate DB wrappers

import { createPipeqlDriver } from '@flaxmbot/pipeql/driver';
import sqlite3 from 'sqlite3';

const db = createPipeqlDriver(new sqlite3.Database('app.db'), { dialect: 'sqlite' });

const rows = await db.query('from users | filter role == $role', { role: 'admin' });
const { lastId, changes } = await db.execute('into notes | insert [title = $title]', { title: 'Hi' });
const newNote = await db.insertAndFetch('into notes | insert $data', req.body);

Python

import pipeql_python as pipeql

res = pipeql.compile("into users | insert [name = $name, email = $email]", "postgres")
print(res["sql"])    # INSERT INTO users (name, email) VALUES ($1, $2) RETURNING *;
print(res["params"]) # ["name", "email"]

from pipeql_python.driver import create_pipeql_driver
import sqlite3

db = create_pipeql_driver(sqlite3.connect('app.db'))
rows = db.query("from users | filter role == $role", {"role": "admin"})

Go

package main

import (
    "fmt"
    "log"
    pipeql "github.com/Flaxmbot/PipeQL/go"
)

func main() {
    res, err := pipeql.Compile("from users | filter age >= $min | select [id, name]", "postgres")
    if err != nil { log.Fatal(err) }
    fmt.Println("SQL:", res.SQL)       // SELECT id, name FROM users WHERE (age >= $1);
    fmt.Println("Params:", res.Params) // ["min"]
}

C

#include <stdio.h>
#include "libpipeql.h"

int main() {
    PipeqlError err = {0};
    PipeqlResult* res = pipeql_compile("from users | filter id == $id", "postgres", &err);
    if (!res) { fprintf(stderr, "Error: %s\n", err.message); return 1; }
    printf("SQL: %s\n", res->sql);
    pipeql_result_free(res);
    return 0;
}

CLI

pipeql compile "from users | take 10" --dialect postgres
pipeql compile "from users | filter id == $id" --dialect sqlite
pipeql parse "from users | select [id, name]"
pipeql dialects
pipeql --version

Optional: Fluent builder (programmatic composition)

The string DSL above is PipeQL's primary interface. For composing queries in code — conditional or looped pipeline stages, or object-style inserts — every SDK additionally ships an optional fluent builder. A builder query and a hand-written string query are provably identical: builders assemble the exact same source string and hand it to the same compiler. Object inserts accept key → value objects and auto-generate $b0, $b1, ... bind parameters.

// Rust — pipeql_core::builder
use pipeql_core::builder::{Query, Value};

let q = Query::from("notes")
    .filter("is_archived == 0")
    .sort(["created_at desc"])
    .take(10);
let compiled = q.compile("postgres").unwrap();

let ins = Query::into_("notes").insert([("title", Value::Str("Hi".into()))]);
// source: "into notes | insert [title = $b0]"  values: [("title", "Hi")]

// JavaScript — @flaxmbot/pipeql/builder (also works through a driver: db.query(q))
import { PipeQL } from '@flaxmbot/pipeql/builder';

const q = PipeQL.from('notes')
  .filter('is_archived == 0')
  .sort(['created_at desc'])
  .take(10);
const { sql, params } = await q.compile('sqlite');

const ins = PipeQL.into('notes').insert({ title: 'Hi', flag: 1 });
// source: "into notes | insert [title = $b0, flag = $b1]"  values: { b0: 'Hi', b1: 1 }

# Python — pipeql_python.builder (also works through a driver: db.query(q))
from pipeql_python.builder import PipeQL

q = (PipeQL.from_("notes")
     .filter("is_archived == 0")
     .sort(["created_at desc"])
     .take(10))
result = q.compile("postgres")
rows = db.query(q)

ins = PipeQL.into_("notes").insert({"title": "Hi", "flag": 1})
# source: "into notes | insert [title = $b0, flag = $b1]"  values: {"b0": "Hi", "b1": 1}

// Go — pipeql (maps are sorted for deterministic SQL; PairsOf keeps exact order)
q := pipeql.From("notes").
    Filter("is_archived == 0").
    Sort([]string{"created_at desc"}).
    Take(10)
res, err := q.Compile("postgres")

ins := pipeql.Into("notes").Insert(pipeql.PairsOf("title", "Hi", "flag", 1))

// C — libpipeql (every stage returns the handle for chaining)
PipeqlQuery* q = pipeql_query_from("notes");
q = pipeql_query_filter(q, "is_archived == 0");
q = pipeql_query_sort(q, "created_at desc");
q = pipeql_query_take(q, 10);
PipeqlResult* built = pipeql_query_compile(q, "postgres", &err);
printf("SQL: %s\n", built->sql);
pipeql_result_free(built);
pipeql_query_free(q);

When to use the builder: conditional or looped pipeline stages, object-style inserts, and passing a query object straight into a driver. For one-shot queries, the string DSL is shorter and equally safe.

---

Ecosystem

Component	Description
pipeql-core	Core compiler: lexer → parser → AST → codegen
pipeql-cli	Command-line tool
pipeql-wasm	WebAssembly target for browsers
pipeql-python	Python binding (PyO3, ABI3) → pipeql on PyPI
pipeql-cffi	C ABI shared library
pipeql-lsp	Language Server Protocol
js/	JavaScript/TypeScript SDK (@flaxmbot/pipeql)
go/	Go binding (CGO) — github.com/Flaxmbot/PipeQL/go
python/	Python package + driver adapters
docs-web/	Interactive documentation + WASM playground
extensions/	VS Code extension
tree-sitter-pipeql/	Tree-sitter grammar

---

Compiler Architecture

Source Text ──→ Lexer ──→ Tokens ──→ Parser ──→ AST ──→ Codegen ──→ SQL + Params
                          │                     │                    │
                          │                     │                    ├─ PostgreSQL ($1, $2)
                          │                     │                    ├─ SQLite     (?, ?)
                          │                     │                    ├─ DuckDB     (?, ?)
                          │                     │                    └─ MySQL      (?, ?)
                          │                     │
                          │                     └─ Lossless AST (spans + comments)
                          │
                          └─ Character-level span tracking

1. Lexer — hand-written tokenizer with exact character positions for IDE support
2. Parser — Pratt parser producing a lossless abstract syntax tree
3. Codegen — walks the AST, extracts all values into bind parameters, emits dialect-specific SQL

All language bindings are thin wrappers calling the Rust core through pipeql-core's API.

Safety: #![deny(unsafe_code)] enforced across the entire compiler core.

---

Error Messages

PipeQL provides compiler-grade error messages with exact positions and actionable suggestions:

Error at line 1, col 1: Unknown keyword 'selct'
  hint: Did you mean 'select'?

Error at line 1, col 35: 'update' requires a preceding 'filter' stage
  hint: Add a filter to prevent accidental mass updates: from <table> | filter ... | update [...] (or write `update all [...]` to explicitly opt in)

Error at line 1, col 15: Unclosed string literal
  hint: Add a closing single quote (') to terminate the string.

Features: Levenshtein-based fuzzy keyword matching, contextual hints, unclosed string/subquery detection, duplicate column detection, empty pipeline errors, filter-before-mutate enforcement.

---

AI & LLM Integration

PipeQL ships with an optimized System Prompt for code generation with LLMs (GPT-4, Claude, Gemini, etc.):

# Python
import pipeql_python
system_prompt = pipeql_python.SYSTEM_PROMPT

// JavaScript — bundled in npm package
import prompt from '@flaxmbot/pipeql/ai/system_prompt.md';

Direct download: ai/system_prompt.md

---

Contributing

git clone https://github.com/Flaxmbot/PipeQL.git && cd PipeQL
cargo test --workspace              # Run all tests
cargo bench -p pipeql-core          # Benchmarks
cargo build --release -p pipeql-cli # Build CLI

---

License

MIT © Flaxmbot