neosqlite 1.14.1

NoSQL for SQLite with PyMongo-like API

NeoSQLite - NoSQL for SQLite with PyMongo-like API

NeoSQLite (new + nosqlite) is a pure Python library that provides a schemaless, PyMongo-like wrapper for interacting with SQLite databases. The API is designed to be familiar to those who have worked with PyMongo, providing a simple and intuitive way to work with document-based data in a relational database.

Keywords: NoSQL, NoSQLite, SQLite NoSQL, PyMongo alternative, SQLite document database, Python NoSQL, schemaless SQLite, MongoDB-like SQLite

Features

• PyMongo-like API: A familiar interface for developers experienced with MongoDB.
• NX-27017: MongoDB Wire Protocol Server — Use PyMongo with SQLite backend
• Schemaless Documents: Store flexible JSON-like documents.
• Lazy Cursor: find() returns a memory-efficient cursor for iterating over results.
• Raw Batch Support: find_raw_batches() returns raw JSON data in batches for efficient processing.
• Advanced Indexing: Single-key, compound-key, nested-key indexes, and FTS5 text search.
• ACID Transactions: Full ClientSession API with PyMongo 4.x parity using SQLite SAVEPOINTs.
• Change Streams: Native SQLite triggers for watch() — no replica set required.
• Advanced Aggregation: $setWindowFields, $graphLookup, $fill, streaming $facet, and more.
• Tier-1 SQL Optimization: Dozens of operators translated to native SQL for 10-100x speedup.
• Native $jsonSchema: Query filtering and write-time validation via SQLite CHECK constraints.
• Window Functions: Complete MongoDB 5.0+ suite ($rank, $top, $bottom, math operators).
• MongoDB-compatible ObjectId: Full 12-byte specification with automatic generation.
• Full GridFS Support: Modern GridFSBucket API plus legacy GridFS compatibility.
• Binary Data: PyMongo-compatible Binary class with UUID support.
• AutoVacuum & compact: Reclaim disk space with incremental or full VACUUM.
• dbStats Command: MongoDB-compatible statistics with accurate index sizes.
• SQL Translation Caching: 10-30% faster for repeated aggregation pipelines and $expr queries.
• Configurable Journal Mode: WAL (default), DELETE, MEMORY, and more.
• Security Hardening: Built-in SQL injection protection via centralized identifier quoting.

See CHANGELOG.md for the full history.

Latest Release: v1.14.1

NeoSQLite v1.14.1 is a correctness and compatibility release that eliminates major limitations in the SQL evaluation tier. It introduces Static Type Inference to perfectly match MongoDB BSON type semantics even for computed expressions and provides full SQL-tier support for the $type operator.

Critical Fix: This release resolves a functional issue where $expr queries returning strings (like $type or $concat) would return zero results in find() filters.

Key Features & Fixes

• Static Type Inference: Predicted return types for MongoDB operators allowing SQL-tier optimization for computed expressions.
• SQL-Tier $type: Full database-level support for the $type operator, including detection of NeoSQLite-encoded Binary and ObjectId.
• Corrected $expr Truthiness: Wrapped $expr SQL results in an optimized COALESCE logic to ensure strings and objects are correctly treated as truthy in filter predicates.
• Perfect Type Matching: Verified that $isNumber and $toBool now work correctly for both direct field references and computed expressions in the SQL tier.

For full details, see documents/releases/v1.14.1.md.

Installation

pip install neosqlite

Optional Extras

# Enhanced JSON/JSONB support (only needed if your SQLite lacks JSON functions)
pip install neosqlite[jsonb]

# Memory-constrained processing for large result sets
pip install neosqlite[memory-constrained]

# NX-27017 MongoDB Wire Protocol Server
pip install "neosqlite[nx27017]"          # Core
pip install "neosqlite[nx27017-speed]"    # With uvloop (Linux/macOS)

Quickstart

import neosqlite

with neosqlite.Connection(':memory:') as conn:
    users = conn.users

    # Insert
    users.insert_one({'name': 'Alice', 'age': 30})
    users.insert_many([
        {'name': 'Bob', 'age': 25},
        {'name': 'Charlie', 'age': 35}
    ])

    # Find
    alice = users.find_one({'name': 'Alice'})
    for user in users.find():
        print(user)

    # Update
    users.update_one({'name': 'Alice'}, {'$set': {'age': 31}})

    # Delete & Count
    result = users.delete_many({'age': {'$gt': 30}})
    print(f"Remaining: {users.count_documents({})}")

Drop-in Replacement for PyMongo

1. Direct API (No MongoDB)

import neosqlite
client = neosqlite.Connection('mydatabase.db')
collection = client.mycollection
collection.insert_one({"name": "test"})

2. Wire Protocol (NX-27017) — Zero Code Changes

# Start server
nx-27017 --db ./myapp.db

# Then use PyMongo normally — no code changes!
from pymongo import MongoClient
client = MongoClient('mongodb://localhost:27017/')
collection = client.mydatabase.mycollection
collection.insert_one({"name": "test"})  # Works!

PyMongo Compatibility

Metric	Result
Total Tests	386
Passed	368
Skipped	18 (architectural differences)
Failed	0
Compatibility	100%

Skipped tests are due to MongoDB requiring a replica set (change streams, transactions) or NeoSQLite extensions ($log2, $contains). All comparable APIs pass.

Run the comparison yourself: ./scripts/run-api-comparison.sh

Key APIs

Indexes

# Single-key, compound, nested
users.create_index('age')
users.create_index([('name', neosqlite.ASCENDING), ('age', neosqlite.DESCENDING)])
users.create_index('profile.followers')

# FTS5 text search
users.create_search_index('bio')

Query Operators

$eq, $gt, $gte, $lt, $lte, $ne, $in, $nin, $and, $or, $not, $nor, $exists, $type, $regex, $elemMatch, $size, $mod, $bitsAllSet, $bitsAllClear, $bitsAnySet, $bitsAnyClear, $text (FTS5), $jsonSchema, and more.

Aggregation Stages

$match, $project, $group, $sort, $skip, $limit, $unwind, $lookup, $facet, $bucket, $bucketAuto, $sample, $merge, $setWindowFields, $graphLookup, $fill, $densify, $unionWith, $replaceRoot, $replaceWith, $unset, $count, $redact, $addFields, $switch.

Transactions

with client.start_session() as session:
    with session.start_transaction():
        users.insert_one({"name": "Alice"}, session=session)
        orders.insert_one({"user": "Alice"}, session=session)
    # Commits on success, rolls back on exception

Change Streams

# Native SQLite triggers — no replica set needed
stream = collection.watch()
for change in stream:
    print(change)

Journal Mode

from neosqlite import Connection, JournalMode

db = Connection("app.db", journal_mode=JournalMode.WAL)  # Default

Mode	Use Case
WAL	Best concurrency (default)
DELETE	Single-file distribution
MEMORY	Maximum speed, no crash recovery

Documentation

Topic	Link
Release Notes	documents/releases/
Changelog	CHANGELOG.md
GridFS	documents/GRIDFS.md
Text Search	documents/TEXT_SEARCH.md
Aggregation Optimization	documents/AGGREGATION_PIPELINE_OPTIMIZATION.md
NX-27017 Server	packages/nx_27017/README.md
API Comparison	examples/api_comparison/README.md

Contribution and License

This project was originally developed as shaunduncan/nosqlite and was later forked as plutec/nosqlite before becoming NeoSQLite. It is now maintained by Chaiwat Suttipongsakul and is licensed under the MIT license.

Contributions are highly encouraged. If you find a bug, have an enhancement in mind, or want to suggest a new feature, please feel free to open an issue or submit a pull request.