Timber

Configuration-driven persistence library with automatic encryption, caching, vector search, and GDPR compliance

What is Timber?

Timber is a configuration-driven persistence library that eliminates boilerplate code by defining SQLAlchemy models in YAML instead of Python. It automatically provides encryption, caching, vector search, and GDPR compliance based on simple configuration flags.

Transform this Python boilerplate:

class StockResearchSession(Base):
    __tablename__ = 'stock_research_sessions'
    id = Column(String(36), primary_key=True, default=uuid4)
    user_id = Column(String(36), ForeignKey('users.id'), nullable=False)
    symbol = Column(String(10), nullable=False)
    analysis = Column(JSON)
    created_at = Column(DateTime, default=datetime.utcnow)
    # ... 50+ more lines of boilerplate

Into this YAML configuration:

models:
  - name: StockResearchSession
    table_name: stock_research_sessions
    
    # Enable features with one line
    encryption:
      enabled: true
      fields: [analysis]
    
    caching:
      enabled: true
      ttl_seconds: 3600
    
    vector_search:
      enabled: true
      content_field: analysis
    
    columns:
      - name: id
        type: String(36)
        primary_key: true
      - name: user_id
        type: String(36)
        foreign_key: users.id
      - name: symbol
        type: String(10)
      - name: analysis
        type: JSON

Key Features

🎯 Configuration-Driven Models

• Zero Python boilerplate - Define models in YAML
• Dynamic generation - Models created at runtime
• Full SQLAlchemy - All SQLAlchemy features supported
• Type-safe - Validated configuration with clear errors

🔐 Automatic Encryption

• Field-level encryption - Specify fields to encrypt
• Transparent - Automatic encrypt/decrypt
• Secure - Uses Fernet (symmetric encryption)
• No code changes - Enable with one config line

⚡ Multi-Level Caching

• Redis support - Distributed caching
• Local cache - In-memory fallback
• Automatic invalidation - Cache cleared on updates
• Configurable TTL - Per-model cache duration

🔍 Vector Search

• Semantic search - Find by meaning, not keywords
• Automatic embeddings - Generated on insert
• Multiple backends - Qdrant, Weaviate, Pinecone
• Hybrid search - Combine vector + keyword

✅ GDPR Compliance

• Data export - User data export in JSON
• Right to deletion - Complete data removal
• Audit trails - Track data operations
• Configurable - Specify exportable fields

🏗️ Modular Services

• Session Service - User session management
• Research Service - Store analysis and research
• Notification Service - User notifications
• Tracker Service - Event tracking and analytics
• Stock Data Service - Financial data fetching

🌐 Multi-App Support

• Shared infrastructure - One library, many apps
• Data isolation - Clear boundaries between apps
• Consistent patterns - Same API across applications

📈 Multi-Provider Stock Data

• Provider fallback chain - yfinance, Alpha Vantage, Finnhub, Polygon
• Per-call ordering - Pass provider_order=[...] to control the chain
• Skip-missing-key / fall-through - Skips keyless providers, falls through on errors and empty results
• Deterministic, normalized output - Reproducible P/E, standardized info/news across sources

🧩 Domain Plugins

• Discoverable extensions - Domains register via the timber.domains entry-point group
• No core edits - Plugins write into shared registries; core never imports a domain
• Dependency-injected - Each plugin receives a DomainContext (registries + db/llm/config)
• Registered at startup - Models/services wired in before table creation (init Step 8.5)

A domain is an installed package that advertises one entry point per layer. Timber is the substrate layer (timber.domains → models + services); grove (grove.domains → HTTP routers) and acorn (acorn.domains → agent tools) follow the same shape against their own groups, and a domain can also ship sky widgets. One package can plug into all of them with no edits to the core libraries. Two domains live on this system today:

• oak-domain-investments - the first complete business domain: goal-linked watchlist, accumulation plans, value projection, growth scheduler, and holdings linkage (in production).
• oak-domain-legal-intake - a second, unrelated access-to-justice domain, proving the pattern is general (Phase 0, instrumentation only).

See the oak-domain-plugins skill for the full plugin contract.

Quick Start

Installation

pip install timber-common

Basic Example

from timber.common import initialize_timber, get_model
from timber.common.services.persistence import session_service

# 1. Initialize Timber with your model configs
initialize_timber(
    model_config_dirs=['./data/models'],
    database_url='postgresql://localhost:5432/mydb'
)

# 2. Use services immediately
session_id = session_service.create_session(
    user_id='user-123',
    session_type='research',
    metadata={'symbol': 'AAPL'}
)

# 3. Or access models directly
Session = get_model('Session')
session = session_service.get_session(session_id)
print(f"Created session for {session.metadata['symbol']}")

Complete Workflow Example

from timber.common import initialize_timber
from timber.common.services.persistence import (
    session_service,
    research_service,
    notification_service
)

# Initialize
initialize_timber(model_config_dirs=['./data/models'])

# Create research session
session_id = session_service.create_session(
    user_id='user-123',
    session_type='research',
    metadata={'symbol': 'AAPL'}
)

# Save research (automatically encrypted if configured)
research_id = research_service.save_research(
    session_id=session_id,
    content={
        'company': 'Apple Inc.',
        'analysis': 'Strong fundamentals...',
        'recommendation': 'Buy'
    },
    research_type='fundamental'
)

# Notify user (automatically stored)
notification_service.create_notification(
    user_id='user-123',
    notification_type='research_complete',
    title='Analysis Complete',
    message='Your AAPL analysis is ready'
)

print(f"✅ Research workflow complete!")

Vector Search Example

from timber.common.services.vector import vector_service

# Semantic search (finds by meaning, not just keywords)
results = vector_service.search(
    query="companies with strong AI capabilities",
    collection_name="research_documents",
    limit=10
)

for result in results:
    print(f"{result['payload']['title']}: {result['score']:.3f}")

Multi-Provider Stock Data

Timber ships a unified stock-data service that fetches market data across multiple providers — yfinance, Alpha Vantage, Finnhub, and Polygon — with automatic fallback. yfinance is key-free and always available; the other three are used only when their API key is configured.

from common.services.data_fetcher import stock_data_service

# Historical OHLCV (returns a (DataFrame, error) tuple)
df, error = stock_data_service.fetch_historical_data("AAPL", period="1y")

# Company info / news / financials
info, error = stock_data_service.fetch_company_info("AAPL")
news, error = stock_data_service.fetch_news("AAPL", limit=10)
income, balance, cashflow, error = stock_data_service.fetch_financials("AAPL", period="yearly")

Explicit provider ordering

Every fetch method accepts an optional provider_order list. When supplied, the listed providers are tried in order; otherwise a key-derived primary/fallback ordering is used. The chain skips any provider whose API key is missing and falls through to the next provider on an error or an empty result, returning the last error only if all providers fail.

# Try Polygon first, then Alpha Vantage, then yfinance.
df, error = stock_data_service.fetch_historical_data(
    "AAPL",
    period="1y",
    provider_order=["polygon", "alphavantage", "yfinance"],
)

The available fetch methods are fetch_historical_data, fetch_company_info, fetch_news, and fetch_financials, all of which accept provider_order.

Deterministic P/E and cross-source normalization

Provider responses are normalized to a stable shape. Because a provider's live trailingPE is tied to the intraday tick (and so varies run to run), Timber derives a reproducible peRatio = previousClose / trailingEps (rounded, div-by-zero guarded) whenever both inputs are available. previousClose and trailingEps are always surfaced, and the provider's raw value is preserved under trailingPE_live. News from all providers is deduplicated and ordered deterministically.

Alpha Vantage throttling and delayed entitlement

Alpha Vantage returns HTTP 200 with a single-key payload when throttled, so Timber treats "Error Message", "Note", and the newer daily-limit "Information" responses as errors (not empty success). This lets the provider chain fall through to the next source instead of silently degrading to blank fields.

Delayed-plan support is built in: set the Alpha Vantage entitlement config to "delayed" and Timber automatically appends entitlement=delayed to the market-data functions that require it (e.g. GLOBAL_QUOTE and the TIME_SERIES_* family), while leaving fundamentals and news endpoints untouched.

Domain Plugins

Timber can be extended to new business domains without modifying the core library. A domain lives as its own installable package that declares itself and registers its models, services, and operations into Timber's shared registries at startup. The dependency direction is strictly one-way: domain plugins import Timber's core interfaces; core never imports a domain.

The plugin contract

A domain exposes an object satisfying the DomainPlugin protocol:

from common.plugins import DomainPlugin, DomainContext

class LegalIntakeDomain:
    name = "legal_intake"        # stable domain key
    version = "0.1.0"            # domain/contract version

    def register(self, ctx: DomainContext) -> None:
        # Wire models/services/operations into the shared registries.
        ctx.service_registry.register("legal_intake", "intake", IntakeService(ctx.db))
        # ctx also exposes: model_registry, operation_registry, db, llm, config
        ...

DomainContext is a dataclass injected at startup carrying the registries the domain writes to (model_registry, operation_registry, service_registry) and the shared singletons it reads (db, llm, config). register(ctx) is called exactly once per process, after core models are loaded and before table creation, so any models a domain registers get their tables created.

Discovery

Plugins are discovered two ways, in priority order:

1. Entry points (the production mechanism) — installed packages advertise themselves under the timber.domains entry-point group:

   # in a domain package's pyproject.toml
   [project.entry-points."timber.domains"]
   legal_intake = "oak_domain_legal_intake:LegalIntakeDomain"
   

2. TIMBER_DOMAIN_PLUGINS env var (local-dev / test fallback) — a comma-separated list of module:attr references:

   TIMBER_DOMAIN_PLUGINS="oak_domain_legal_intake:LegalIntakeDomain"
   

Plugins are de-duplicated by name (entry points win over env refs). With neither mechanism present, discovery is a no-op and initialization is unchanged.

The service registry

Domains register services under a domain namespace via the singleton ServiceRegistry instead of monkeypatching the common.services module. Consumers reach them through a live accessor:

from common.services.registry import service_registry

intake = service_registry.domain("legal_intake").intake   # attribute access
intake = service_registry.get("legal_intake", "intake")    # or by (domain, name)

Wiring at startup (initialize_timber Step 8.5)

initialize_timber() runs a Step 8.5 that discovers domains and registers them before table creation:

from common.plugins import discover_domains, DomainContext
from common.services.registry import service_registry

ctx = DomainContext(
    model_registry=model_registry,
    operation_registry=operation_registry,
    service_registry=service_registry,
    db=db_service,
    llm=llm_service,
    config=config,
)
for plugin in discover_domains():
    plugin.register(ctx)   # registers models/services/operations
# ...table creation (Step 9) then picks up any newly registered domain models.

If no domains are installed, Step 8.5 is a no-op and Timber behaves as core-only.

Documentation

📚 How-To Guides

• Getting Started - Setup and first model
• Creating Models - YAML model definitions
• Using Services - Persistence services

🏛️ Design Guides

• System Architecture - Overall design
• Config-Driven Models - Model factory pattern
• Persistence Layer - Database architecture
• Vector Integration - Semantic search
• Multi-App Support - Multiple applications

📖 Full Documentation Index

See DOCUMENTATION_INDEX.md for complete documentation structure.

Requirements

• Python: 3.13+
• Database: PostgreSQL 12+
• Optional: Redis (for distributed caching)
• Optional: Qdrant/Weaviate/Pinecone (for vector search)

Installation Options

Basic Installation

pip install timber-common

With Vector Search (Qdrant)

pip install timber-common[qdrant]

With All Optional Features

pip install timber-common[all]

Development Installation

git clone https://github.com/pumulo/timber-common.git
cd timber-common
poetry install

Configuration

Environment Variables

Create a .env file:

# Database
DATABASE_URL=postgresql://user:password@localhost:5432/dbname

# Redis (optional)
REDIS_URL=redis://localhost:6379/0

# Vector Database (optional)
QDRANT_URL=http://localhost:6333

# Encryption
ENCRYPTION_KEY=your-fernet-key-here

# Feature Flags
ENABLE_ENCRYPTION=true
ENABLE_VECTOR_SEARCH=true
ENABLE_GDPR=true
CACHE_ENABLED=true

Model Configuration

Create YAML files in data/models/:

# data/models/user_models.yaml
version: "1.0.0"

models:
  - name: User
    table_name: users
    
    columns:
      - name: id
        type: String(36)
        primary_key: true
        default: uuid4
      
      - name: email
        type: String(255)
        unique: true
        nullable: false
      
      - name: created_at
        type: DateTime
        default: utcnow

Use Cases

Financial Applications

• Trading platforms
• Research tools
• Portfolio management
• Market analysis

Content Platforms

• Document management
• Knowledge bases
• Content recommendation
• Semantic search

Data Analytics

• User behavior tracking
• Event analytics
• Session management
• Activity monitoring

Multi-Tenant Applications

• SaaS platforms
• Enterprise applications
• Multiple product lines
• Isolated data domains

Architecture

┌─────────────────────────────────────────┐
│          Your Application               │
└─────────────────────────────────────────┘
                  │
                  ↓
┌─────────────────────────────────────────┐
│         Timber Library                  │
│  ┌──────────────┐  ┌─────────────────┐ │
│  │ Model Factory│  │  Services Layer │ │
│  └──────────────┘  └─────────────────┘ │
│  ┌──────────────┐  ┌─────────────────┐ │
│  │   Encryption │  │  Vector Search  │ │
│  └──────────────┘  └─────────────────┘ │
└─────────────────────────────────────────┘
                  │
                  ↓
┌─────────────────────────────────────────┐
│      Infrastructure                     │
│  PostgreSQL │ Redis │ Qdrant           │
└─────────────────────────────────────────┘

Examples

E-Commerce Platform

models:
  - name: Product
    table_name: products
    
    vector_search:
      enabled: true
      content_field: description
    
    columns:
      - name: id
        type: String(36)
        primary_key: true
      - name: name
        type: String(255)
      - name: description
        type: Text
      - name: price
        type: Numeric(10, 2)

Healthcare Application

models:
  - name: PatientRecord
    table_name: patient_records
    
    encryption:
      enabled: true
      fields: [ssn, medical_history]
    
    gdpr:
      enabled: true
      user_id_field: patient_id
      export_fields: [name, date_of_birth, medical_history]
    
    columns:
      - name: id
        type: String(36)
        primary_key: true
      - name: patient_id
        type: String(36)
        foreign_key: patients.id
      - name: ssn
        type: String(11)
      - name: medical_history
        type: JSON

Testing

# Run tests
poetry run pytest

# With coverage
poetry run pytest --cov=common --cov=modules

# Run specific test
poetry run pytest tests/test_models.py::test_create_model

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Development Setup

# Clone repository
git clone https://github.com/pumulo/timber-common.git
cd timber-common

# Install dependencies
poetry install

# Run tests
poetry run pytest

# Format code
poetry run black .
poetry run isort .

# Type check
poetry run mypy common modules

Performance

Timber is designed for production use with:

• Connection pooling - Efficient database connections
• Query optimization - Built-in best practices
• Caching - Multi-level cache strategy
• Batch operations - Efficient bulk processing

Benchmarks

Operation               Time (ms)    Notes
─────────────────────────────────────────────
Simple INSERT           1-5          Single record
Batch INSERT (100)      10-20        Bulk insert
SELECT by ID            1-2          Indexed lookup
Vector search           5-15         Semantic search
Cached query            < 1          Redis/local cache

Roadmap

Version 0.2.0 (Q1 2025)

• MySQL and SQLite support
• GraphQL API generation
• CLI tools for model management
• Enhanced monitoring dashboard

Version 0.3.0 (Q2 2025)

• Real-time data streaming
• Advanced analytics
• Built-in vector store (no external DB required)
• Docker and Kubernetes templates

Future

• Multi-database transactions
• Distributed tracing
• Auto-scaling recommendations
• Visual model designer

Support

Get Help

• Documentation: Full docs
• Issues: GitHub Issues
• Email: pumulo@gmail.com

Commercial Support

For enterprise support, training, or consulting:

• Email: pumulo@gmail.com

License

Timber is released under the Apache License 2.0.

Copyright 2025 Pumulo Sikaneta

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

See the LICENSE file for the full license text.

License: Apache-2.0

Author

Pumulo Sikaneta

• Email: pumulo@gmail.com
• GitHub: @pumulo
• Website: [Your website]

Acknowledgments

Built with:

• SQLAlchemy - The Python SQL toolkit
• PostgreSQL - The world's most advanced open source database
• FastEmbed - Fast embedding generation
• Poetry - Python dependency management

Citation

If you use Timber in academic research, please cite:

@software{timber2025,
  author = {Sikaneta, Pumulo},
  title = {Timber: Configuration-Driven Persistence Library},
  year = {2025},
  url = {https://github.com/pumulo/timber-common},
  version = {0.1.0}
}

Star History

If you find Timber useful, please star the repository! ⭐

Made with ❤️ by Pumulo Sikaneta

Copyright © 2025 Pumulo Sikaneta. Licensed under Apache-2.0.