SharedData

A comprehensive ultrafast Python library for financial data.

📖 Table of Contents

• 🏗️ Core Features
• ⚡ Quick Start
• 🔧 Configuration
• 🚀 Advanced Usage
• 🔄 Development & Documentation
• 📄 License
• 🤝 Contributing

🏗️ Core Features

SharedData provides a comprehensive set of features for high-performance financial data management:

• 🗃️ Database Schema & Indexing - Optimized schemas for financial data types
• 🌐 Storage & Integration - Multi-storage support (Local, S3, MongoDB, Redis)
• 📈 Performance & Scalability - Parallel processing and advanced querying
• 📊 Data Containers - Tables, Collections, Time Series, Streams, Cache, Metadata
• ⚡ Multiprocessing & Parallel Computing - Sophisticated parallel processing library
• 🤖 Distributed Worker System - Automated task execution and job scheduling
• 📋 Comprehensive Logging System - Enterprise-grade logging with multiple destinations
• 🌐 Remote API Client - REST API for remote data access and operations

📚 Read the complete Core Features guide →

⚡ Quick Start

Get up and running with SharedData in minutes:

Installation

# Create virtual environment
python -m venv venv
source venv/bin/activate

# Install SharedData
pip install -r requirements.txt
pip install -e .

Basic Usage

import pandas as pd
from SharedData.SharedData import SharedData

# Initialize SharedData
shdata = SharedData(__file__, user='master')

# Quick example - Tables
dates = pd.date_range('2025-01-01', '2025-01-10', freq='D')
symbols = ['AAPL', 'GOOGL', 'MSFT']
idx = pd.MultiIndex.from_product([dates, symbols], names=['date', 'symbol'])
df = pd.DataFrame({'price': 100, 'volume': 1000}, index=idx)

# Write and read data
tbl = shdata.table('MarketData', 'D1', 'TEST', 'PRICES', value=df)
tbl.write()
data = tbl.loc['2025-01-05':, 'AAPL']  # Fast symbol lookup
print(f"Retrieved {data.shape[0]} rows")

📚 Read the complete Quick Start guide →

🔧 Configuration

Configure SharedData with environment variables for your specific needs:

# Required Variables
SHAREDDATA_SECRET_KEY="your-secret-key"
SHAREDDATA_TOKEN="your-auth-token"
AWS_ACCESS_KEY_ID="your-aws-access-key"
S3_BUCKET="s3://your-bucket-name"
MONGODB_HOST="your-mongodb-host"
SHAREDDATA_ENDPOINT="http://your-server:port"

📚 Read the complete Configuration guide →

🚀 Advanced Usage

For power users and complex scenarios:

• Advanced Data Operations - Complex queries and aggregations
• Performance Optimization - Memory management and parallel processing
• Distributed Computing - Worker pools and distributed data processing
• Custom Extensions - Custom data containers and worker types
• Production Deployment - High availability and monitoring
• Monitoring & Debugging - Performance profiling and health checks

📚 Read the complete Advanced Usage guide →

🧭 Data by Symbol

Fetch data by what it is instead of where it is stored. The server picks the source, the container and the partitions, and every answer says where its rows came from:

bars = shdata.dataframe('marketdata', 'AAPL', period='M15', start='2026-10-05')
bars.attrs['provenance']['dataset']        # which container answered

from SharedData.IO.ClientAPI import ClientAPI
surface = ClientAPI.get_vol('AAPL', product='tenor', tenor='1M')

Market data, signals, portfolios, risk factors and volatility are served this way over GET /api/data/.... This is a convenience, not the fast path: producers and other performance-critical code keep using table(), timeseries() and collection().

📚 Read the Data by Symbol guide →

🔄 Development & Documentation

For developers contributing to SharedData:

Building Documentation

# Generate documentation from docstrings
make gitea

# Alternative methods
python generate_docs.py
python update_docs.py
make all

Running Tests

python -m pytest tests/

📚 Read the complete Development guide →

📄 License

MIT: free to use, change and distribute, commercially too. See the LICENSE file.

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Update documentation: make gitea
5. Submit a pull request

📚 Read the complete Contributing guide →

📚 Documentation Index

• 🏗️ Core Features - Comprehensive overview of all SharedData capabilities
• ⚡ Quick Start - Get started with SharedData in minutes
• 🔧 Configuration - Complete configuration reference
• 🚀 Advanced Usage - Advanced patterns and optimization
• 🔄 Development - Contributing and development guidelines

⬆️ Back to top