inillucent

inillucent is an embedded SQL database. It speaks SQLite's dialect on its own storage, and it has keyword search and vector search built in. One .rdb file holds the tables and the search indexes.

This package gives Python two ways to use it:

• The driver: Database, Connection and Rows. It calls the inillucent C library inside your Python process through ctypes. Under CPython the library builds each result's rows as Python objects, the way Python's own sqlite3 module does. Use the driver in an application.
• The command line helpers: run() and query(). Each call starts the inillucent program and reads the JSON it prints. Use them in scripts, for commands the driver has no method for.

Install

pip install inillucent

The package needs Python 3.9 or later. The wheel for your platform holds the C library and the four programs, so the install needs no compiler, no Rust toolchain and no network access after the download.

The driver loads the C library from the wheel's _lib folder. Set INILLUCENT_DRIVER_LIB to the path of a different library file to load that one. The helpers run the programs in the wheel's _bin folder. A source install has neither folder. It needs a library you built with cargo build -p inillucent-driver-capi, named by INILLUCENT_DRIVER_LIB.

A first program

from inillucent import Database

with Database("app.rdb") as database:
    connection = database.connect()
    connection.execute("CREATE TABLE people (id INTEGER PRIMARY KEY, name TEXT)")
    connection.execute("INSERT INTO people VALUES (?1, ?2)", [1, "Ada"])
    rows = connection.execute("SELECT id, name FROM people", limit=200)
    print(rows.rows, rows.total)   # [[1, 'Ada']] 1

Database("app.rdb") creates the file when it does not exist. Leaving the with block closes every connection, writes everything to the file and closes it.

?1, ?2 and so on are bound to the values in the list, in order. Bind values this way instead of pasting them into the SQL text.

limit caps the rows returned. rows.total still counts every row the statement produced, and rows.more is True when rows were left out. With no limit, execute() returns every row.

Transactions

with connection.transaction() as transaction:
    transaction.execute("INSERT INTO people VALUES (2, 'Grace')")
    transaction.execute("UPDATE people SET name = 'Ada Lovelace' WHERE id = 1")

Leaving the block normally commits. Leaving it with an exception rolls back. A statement that fails inside Transaction.execute() rolls back the whole transaction before it raises.

Statements you run many times

with connection.prepare("INSERT INTO people VALUES (?1, ?2)") as statement:
    for key, name in [(3, "Grace"), (4, "Edsger")]:
        statement.execute([key, name])

Values

Any other type raises TypeError.

Threads

The engine is single threaded, and the interpreter lock is let go while a statement runs. So that two threads are never inside one database together, every call on a Database, and on the connections, statements and transactions opened from it, takes one lock that the Database owns. A second thread waits until the first call returns and then runs. Connection.cancel() is the one call that does not take the lock, because it exists to be made from another thread while a statement runs. The lock covers one call. A Transaction that stays open between calls does not stop another thread's statements on the same connection from running inside it, so give each thread its own connection, or keep a transaction on one thread.

Encrypted databases

run and query, the command line helpers, take a key argument.

from inillucent import query, run

key = "x'" + "5a" * 32 + "'"
run("exec", db="app.rdb", key=key, sql="CREATE TABLE note (body TEXT)")
rows = query("SELECT body FROM note", db="app.rdb", key=key)

The key travels to the program in the INILLUCENT_KEY environment variable. It is never put on the command line, because a command line is visible in the process list. When you pass no key, the environment the program inherits is left alone.

A key written x' followed by 64 hexadecimal digits and ' is a raw 32 byte key. Any other text is a passphrase, which is stretched with 600,000 rounds of PBKDF2 on every open, so a raw key opens faster.

A database created with a key is encrypted. Opening it without the key, or with a wrong key, fails with status corrupt. Opening a plaintext database with a key fails the same way.

Without the key, run returns a result whose status is corrupt, and query raises an Error with that status.

Errors

from inillucent import DriverError, Unsupported

try:
    connection.execute("SELECT * FROM absent")
except Unsupported as why:
    print("not built yet:", why.feature)
except DriverError as why:
    print(why.status_name, why.message)   # not_found no such table: absent

Every refusal raises DriverError, also exported as Error. DriverError has these attributes:

The thirteen status names are unsupported, syntax, not_found, constraint, readonly, busy, interrupted, corrupt, io, full, too_big, invalid_state and internal.

Unsupported is the subclass raised for status unsupported. It means the engine has not built that feature. The SQL is not wrong, and a different spelling fails the same way. Catch Unsupported before DriverError so the two cases stay apart. Call capabilities() before you write an unusual statement.

From the command line

pip install puts four programs on PATH:

inillucent create notes.rdb
inillucent --db notes.rdb exec "CREATE TABLE notes (id INTEGER PRIMARY KEY, body TEXT)"
inillucent --db notes.rdb query "SELECT * FROM notes"
inillucent --db notes.rdb describe notes
inillucent-shell notes.rdb
inillucent help

The inillucent program exits with code 0 on success, 1 on a failure, 2 when the command line cannot be read, and 3 when the engine has not built a feature.

The command line helpers in Python

from inillucent import run, query

print(run("describe", db="notes.rdb", table="notes")["ddl"])
print(query("SELECT * FROM notes WHERE id > ?1", db="notes.rdb", params=[3]))

Each helper call starts a process, so use the driver for anything that runs often. query() returns at most 200 rows, the default of the query command. Pass limit=0 for every row.

params reaches the program on standard input, so the number of values and the size of a value have no limit from the command line. bytes, bytearray and memoryview are bound as a BLOB. A NaN is bound as NULL, as the driver binds it. Infinity raises ValueError, because the command line has no spelling for it, and a list, a dict or any other type raises TypeError. The statement is an argument, so a statement longer than about 32,000 characters on Windows raises ValueError; open a Database and call execute for that.

For an AI agent

MCP, the Model Context Protocol, is how an AI agent calls tools. Add inillucent-mcp to an MCP client's configuration:

{
  "mcpServers": {
    "inillucent": {
      "command": "inillucent-mcp",
      "args": ["--db", "app.rdb"]
    }
  }
}

inillucent-mcp serves 30 of the command line's commands as MCP tools. The tools are generated from the same command table as the command line. --readonly refuses every statement that changes data. --root DIR refuses every path outside DIR.

The API

These are the driver's classes and functions. cargo test -p inillucent-compat --test tooling documentation:: reads this table and fails if a name in the first column is not declared in the driver source.

inillucent.__version__ is the package version.

More

• Getting started
• SQL support
• Vector and keyword search
• The C library and language bindings
• Glossary

MIT licence. Source: https://github.com/Black-Rainbow-Labs/Inillucent