bear-shelf 0.3.29

A lightweight document storage system with SQLAlchemy dialect support. Where your data or honey, hibernates.

🐻📚 Bear-Shelf

The shelf where your data hibernates. 🐻💤

A lightweight document storage system with SQLAlchemy dialect support. Store your data in JSONL, JSON, TOML, YAML, XML, MessagePack, or in-memory formats with a clean, type-safe interface — no SQLite required.

Features

• 🔌 SQLAlchemy Integration: A real SQLAlchemy dialect — use familiar ORM syntax over plain files
• 📦 Multiple Backends: JSONL (default), JSON, TOML, YAML, XML, MessagePack, or in-memory
• 🧰 Two Front Doors: Use the raw bearshelf:// dialect, or the batteries-included DatabaseManager API
• 🔒 Type-Safe: Full type hints, Pydantic models, and custom column types (Mapped[Path], Mapped[EpochTimestamp])
• 📝 Write-Ahead Logging: Optional WAL with buffered/immediate flush modes for durability and bulk-write speed
• ⚡ Compiled Core: Hot paths implemented in C++23 (pybind11) and Cython (Goal is to only have it in C++)

Installation

With uv:

uv add bear-shelf
# or
uv pip install bear-shelf

Minimum Python: 3.12.

Quick Start

Bear-Shelf gives you two equally first-class ways to work with your data. Pick whichever fits how you think.

Option A — Raw SQLAlchemy dialect

If you already know SQLAlchemy, just point an engine at a file. The backend is chosen by the file extension.

from sqlalchemy import create_engine
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, Session

engine = create_engine("bearshelf:///path/to/users.jsonl")

class Base(DeclarativeBase): ...

class User(Base):
    __tablename__ = "users"
    id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
    name: Mapped[str]
    email: Mapped[str] = mapped_column(unique=True)

Base.metadata.create_all(engine)

with Session(engine) as session:
    session.add(User(name="Bear", email="[e-mail]"))
    session.commit()

Option B — DatabaseManager API

A higher-level wrapper that handles the engine, sessions, and table registration for you, and returns convenient per-table handles.

from pathlib import Path
from sqlalchemy.orm import Mapped, mapped_column
from bear_shelf.database import BearShelfDB

# get_base() returns a declarative base wired with Bear-Shelf's custom types
Base = BearShelfDB.get_base()

class User(Base):
    __tablename__ = "users"
    id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
    name: Mapped[str]
    config_path: Mapped[Path]            # stored as text, returned as a Path

db = BearShelfDB(path="users.jsonl")

# Build only the tables you want; bare create_tables() builds them all.
db.include(User).create_tables()

with db.open_session() as session:
    session.add(User(name="Bear", config_path=Path("~/cfg.toml")))

users = db.get_all(User)                 # [User(name='Bear', ...)]
count = db.count(User)                   # 1
db.close()

include(*models) is handy when one declarative base defines many tables but a given database should only create a subset (e.g. test isolation). Creating a subset whose foreign keys reference tables you left out fails loudly with a message telling you what to add.

🎨 Custom Column Types

Annotate columns with rich Python types and Bear-Shelf converts them at the storage boundary — no manual (de)serialization:

Annotation	Stored as	Returned as
Mapped[Path]	text	pathlib.Path
Mapped[EpochTimestamp]	integer	bear_epoch_time.EpochTimestamp

These are registered on the base returned by BearShelfDB.get_base().

🎯 Storage Backends

The backend is auto-detected from the file extension (or pass storage=/schema=):

Backend	URL / extension
JSONL (default)	bearshelf:///data.jsonl
JSON	bearshelf:///data.json
TOML	bearshelf:///data.toml
YAML	bearshelf:///data.yaml
XML	bearshelf:///data.xml
MessagePack	bearshelf:///data.msgpack
Memory	bearshelf:///:memory:

📝 Write-Ahead Logging

For bulk writes and crash safety, enable WAL. In BUFFERED mode, inserts append to a log and a background thread checkpoints to the main file — turning thousands of slow full-file rewrites into fast appends.

from bear_shelf.datastore import BearBase, Columns
from bear_shelf.datastore.wal.config import WALConfig

db = BearBase(
    "events.json",
    storage="json",
    enable_wal=True,
    wal_config=WALConfig.high_throughput(),   # or .immediate() for fsync-per-op
)

db.create_table("events", columns=[
    Columns(name="id", type="int", primary_key=True),
    Columns(name="event_type", type="str"),
])

table = db.table("events")
table.insert_all([{"id": i, "event_type": "click"} for i in range(5000)])
db.close()

See examples/wal_example.py for buffered/immediate modes, custom tuning, manual checkpointing, and crash recovery, and examples/foreign_key_example.py for relationships.

🐻 About

Built with ❤️ by Bear. Part of the Bear-verse ecosystem:

• funcy_bear: functional programming & type introspection utilities
• bear-epoch-time: precision-aware timestamp handling
• codec_cub: Codec utilties for the various storages used in Bear Shelf