fdb-record-layer 0.1.0b1

Python implementation of FoundationDB Record Layer

FDB Record Layer for Python

A Python implementation of the FoundationDB Record Layer, providing a structured record-oriented store with secondary indexes, query planning, and SQL support on top of FoundationDB.

Features

• Record Store: Type-safe storage and retrieval of Protocol Buffer messages
• Secondary Indexes: Automatic index maintenance with VALUE, COUNT, RANK, and TEXT index types
• Query System: Declarative query API with cost-based optimization (Cascades planner)
• SQL Support: SQL parsing and execution with full DML/DDL support
• Async/Await: Native Python asyncio support throughout
• Production Ready: Connection pooling, circuit breakers, health checks, graceful shutdown

Installation

pip install fdb-record-layer

For SQL support:

pip install fdb-record-layer[sql]

For all optional dependencies:

pip install fdb-record-layer[all]

Requirements

• Python 3.10+
• FoundationDB 7.1+
• Protocol Buffers 3.20+

Quick Start

Define Your Schema

from fdb_record_layer import RecordMetaDataBuilder, Index, IndexTypes
from your_proto_pb2 import Person  # Your protobuf message

# Build metadata with indexes
metadata = (
    RecordMetaDataBuilder()
    .add_record_type(Person)
    .add_index(Index("person_by_name", "name"))
    .add_index(Index("person_by_age", "age", index_type=IndexTypes.VALUE))
    .build()
)

Store and Query Records

from fdb_record_layer import FDBDatabase, FDBRecordStore

async def main():
    # Connect to FoundationDB
    db = FDBDatabase.open()

    async with db.transaction() as tr:
        # Open record store
        store = await FDBRecordStore.open(tr, metadata, key_space_path=("myapp",))

        # Save a record
        person = Person(id=1, name="Alice", age=30)
        await store.save_record(person)

        # Query by index
        async for record in store.scan_index("person_by_name", equals="Alice"):
            print(f"Found: {record.name}")

Using the Query Builder

from fdb_record_layer.query import Query, Field

# Build a query
query = (
    Query.from_type("Person")
    .where(Field("age").greater_than(25))
    .where(Field("name").starts_with("A"))
    .build()
)

# Execute
async for record in store.execute_query(query):
    print(record)

SQL Queries

from fdb_record_layer.relational import RelationalDatabase

async def main():
    db = RelationalDatabase.open()

    # Execute SQL
    result = await db.execute("""
        SELECT name, age FROM Person
        WHERE age > 25
        ORDER BY name
    """)

    async for row in result:
        print(f"{row['name']}: {row['age']}")

Key Expressions

Key expressions define how to extract keys from records for indexing:

from fdb_record_layer.expressions import field, concat, record_type

# Simple field
field("name")

# Composite key
concat(field("last_name"), field("first_name"))

# Nested field
field("address").nest("city")

# Include record type in key (for union indexes)
concat(record_type(), field("id"))

Index Types

Type	Description	Use Case
VALUE	Standard B-tree index	Equality and range queries
COUNT	Aggregate count index	Fast COUNT(*) queries
SUM	Aggregate sum index	Fast SUM() queries
RANK	Skip-list based ranking	Leaderboards, percentiles
TEXT	Full-text search	Text search with tokenization

Production Features

Connection Pooling

from fdb_record_layer.utils import ConnectionPool

pool = ConnectionPool(min_size=5, max_size=20)
async with pool.acquire() as conn:
    # Use connection
    pass

Circuit Breaker

from fdb_record_layer.utils import get_circuit_breaker

breaker = get_circuit_breaker("fdb")
async with breaker:
    await store.save_record(record)

Health Checks

from fdb_record_layer.utils import get_health_checker

checker = get_health_checker()
report = await checker.check_health()
print(f"Status: {report.status}")

Graceful Shutdown

from fdb_record_layer.utils import init_lifecycle

lifecycle = init_lifecycle()
# Handles SIGTERM/SIGINT, drains connections, runs cleanup hooks

Documentation

• API Reference
• Java Record Layer Docs (conceptual reference)

Development

# Clone and install
git clone https://github.com/foundationdb/fdb-record-layer-python
cd fdb-record-layer-python
pip install -e .[dev]

# Run tests
pytest

# Type checking
mypy fdb_record_layer

# Linting
ruff check fdb_record_layer

License

Apache License 2.0. See LICENSE for details.

Contributing

Contributions are welcome! Please read our contributing guidelines and submit pull requests.

Acknowledgments

This project is a Python port of the FoundationDB Record Layer, originally developed by Apple.