shareddata 6.83.2

Memory Mapped / Shared Memory Database with S3 repository

SharedData

A comprehensive ultrafast Python library for financial data.

📖 Table of Contents

• 🏗️ Core Features
• ⚡ Quick Start
• 🔧 Configuration
• 🚀 Advanced Usage
• 🔄 Development & Documentation
• 📄 License
• 🤝 Contributing

🏗️ Core Features

SharedData provides a comprehensive set of features for high-performance financial data management:

• 🗃️ Database Schema & Indexing - Optimized schemas for financial data types
• 🌐 Storage & Integration - Multi-storage support (Local, S3, MongoDB, Redis)
• 📈 Performance & Scalability - Parallel processing and advanced querying
• 📊 Data Containers - Tables, Collections, Time Series, Streams, Cache, Metadata
• ⚡ Multiprocessing & Parallel Computing - Sophisticated parallel processing library
• 🤖 Distributed Worker System - Automated task execution and job scheduling
• 📋 Comprehensive Logging System - Enterprise-grade logging with multiple destinations
• 🌐 Remote API Client - REST API for remote data access and operations

📚 Read the complete Core Features guide →

⚡ Quick Start

Get up and running with SharedData in minutes:

Installation

# Create virtual environment
python -m venv venv
source venv/bin/activate

# Install SharedData
pip install -r requirements.txt
pip install -e .

Basic Usage

import pandas as pd
from SharedData.SharedData import SharedData

# Initialize SharedData
shdata = SharedData(__file__, user='master')

# Quick example - Tables
dates = pd.date_range('2025-01-01', '2025-01-10', freq='D')
symbols = ['AAPL', 'GOOGL', 'MSFT']
idx = pd.MultiIndex.from_product([dates, symbols], names=['date', 'symbol'])
df = pd.DataFrame({'price': 100, 'volume': 1000}, index=idx)

# Write and read data
tbl = shdata.table('MarketData', 'D1', 'TEST', 'PRICES', value=df)
tbl.write()
data = tbl.loc['2025-01-05':, 'AAPL']  # Fast symbol lookup
print(f"Retrieved {data.shape[0]} rows")

📚 Read the complete Quick Start guide →

🔧 Configuration

Configure SharedData with environment variables for your specific needs:

# Required Variables
SHAREDDATA_SECRET_KEY="your-secret-key"
SHAREDDATA_TOKEN="your-auth-token"
AWS_ACCESS_KEY_ID="your-aws-access-key"
S3_BUCKET="s3://your-bucket-name"
MONGODB_HOST="your-mongodb-host"
SHAREDDATA_ENDPOINT="http://your-server:port"

📚 Read the complete Configuration guide →

🚀 Advanced Usage

For power users and complex scenarios:

• Advanced Data Operations - Complex queries and aggregations
• Performance Optimization - Memory management and parallel processing
• Distributed Computing - Worker pools and distributed data processing
• Custom Extensions - Custom data containers and worker types
• Production Deployment - High availability and monitoring
• Monitoring & Debugging - Performance profiling and health checks

📚 Read the complete Advanced Usage guide →

🔄 Development & Documentation

For developers contributing to SharedData:

Building Documentation

# Generate documentation from docstrings
make gitea

# Alternative methods
python generate_docs.py
python update_docs.py
make all

Running Tests

python -m pytest tests/

📚 Read the complete Development guide →

📄 License

See LICENSE file for details.

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Update documentation: make gitea
5. Submit a pull request

📚 Read the complete Contributing guide →

---

📚 Documentation Index

• 🏗️ Core Features - Comprehensive overview of all SharedData capabilities
• ⚡ Quick Start - Get started with SharedData in minutes
• 🔧 Configuration - Complete configuration reference
• 🚀 Advanced Usage - Advanced patterns and optimization
• 🔄 Development - Contributing and development guidelines

---

⬆️ Back to top