sageTSDB
High-Performance Time Series Database with C++ Core
sageTSDB is a high-performance time series database designed for streaming data processing with support for out-of-order data, window-based operations, and pluggable algorithms.
Repository Owner: Debin Chen (GitHub: @pluviophile-chen)
🚀 Quick Install
pip install isage-tsdb
Requirements: Ubuntu 22.04+ (GLIBC 2.35+) or equivalent Linux distribution.
🌟 Features
- Efficient Time Series Storage: Optimized data structures for time series indexing
- Out-of-Order Data Handling: Automatic buffering and watermarking for late data
- Pluggable Algorithms: Extensible architecture for custom stream processing algorithms
- Window Operations: Support for tumbling, sliding, and session windows
- Stream Join: Window-based join for multiple time series streams
- Python Bindings: Easy-to-use Python API via pybind11
🏗️ Project Structure
sageTSDB/
├── include/sage_tsdb/ # Public header files
│ ├── core/ # Core time series database
│ ├── algorithms/ # Stream processing algorithms
│ ├── plugins/ # Plugin system (PECJ, fault detection)
│ └── utils/ # Utilities and helpers
│
├── src/ # Implementation files
│ ├── core/ # Core implementation
│ ├── algorithms/ # Algorithm implementations
│ ├── plugins/ # Plugin implementations
│ └── utils/ # Utility implementations
│
├── tests/ # 🔬 Unit tests (GoogleTest)
│ ├── test_*.cpp # All test files with detailed comments
│ └── CMakeLists.txt # Test build configuration
│
├── examples/ # 📚 Demo programs
│ ├── persistence_example.cpp # Data persistence demo
│ ├── plugin_usage_example.cpp# Plugin system demo
│ ├── integrated_demo.cpp # PECJ integration demo
│ ├── pecj_replay_demo.cpp # PECJ replay demo
│ ├── performance_benchmark.cpp # Performance testing
│ └── README.md # Examples documentation
│
├── docs/ # 📖 Documentation
│ ├── DESIGN_DOC_SAGETSDB_PECJ.md # Architecture design
│ ├── PERSISTENCE.md # Persistence guide
│ ├── LSM_TREE_IMPLEMENTATION.md # LSM Tree details
│ ├── RESOURCE_MANAGER_GUIDE.md # Resource management
│ └── README.md # Documentation index
│
├── scripts/ # 🛠️ Build and utility scripts
│ ├── build.sh # Main build script
│ ├── build_plugins.sh # Plugin build script
│ ├── build_and_test.sh # Build and test examples
│ ├── run_demo.sh # Demo launcher
│ ├── test_lsm_tree.sh # LSM Tree testing
│ └── README.md # Scripts documentation
│
├── python/ # Python bindings (pybind11)
├── cmake/ # CMake modules
└── CMakeLists.txt # Root build configuration
Directory Organization
- tests/: All test files consolidated here (removed old
test/folder) - examples/: Demo programs only (moved test programs to
tests/) - docs/: All documentation (removed duplicate/outdated docs)
- scripts/: All build scripts in one place (removed outdated scripts)
📦 Quick Start (Python)
Installation
# Install from PyPI (recommended)
pip install isage-tsdb
# Verify installation
python -c "import sage_tsdb; print(sage_tsdb.__version__)"
System Requirements:
- Ubuntu 22.04+ (GLIBC 2.35+) or equivalent
- Python 3.10+
Basic Usage
import sage_tsdb
# Create database
db = sage_tsdb.TimeSeriesDB()
# Insert data
db.add(
timestamp=1000000, # microseconds
value=23.5,
tags={"sensor": "temp_01", "location": "room_a"},
fields={"unit": "celsius"}
)
# Query data
data = db.query(start=0, end=3000000)
print(f"Found {len(data)} data points")
For more examples, see Python Examples below.
📦 Building from Source
Prerequisites
- C++17 compatible compiler (GCC 8+, Clang 7+, MSVC 2019+)
- CMake 3.15 or higher
- Python 3.8+ (for Python bindings)
- pybind11
Build Instructions
# Clone the repository
git clone https://github.com/intellistream/sageTSDB.git
cd sageTSDB
# Create build directory
mkdir build && cd build
# Configure and build
cmake ..
make -j$(nproc)
# Run tests
ctest
# Install (optional)
sudo make install
Build Python Bindings
# From build directory
cmake -DBUILD_PYTHON_BINDINGS=ON ..
make -j$(nproc)
# Install Python package
pip install .
🚀 Quick Start
C++ API
#include <sage_tsdb/core/time_series_db.h>
#include <sage_tsdb/algorithms/stream_join.h>
using namespace sage_tsdb;
int main() {
// Create database
TimeSeriesDB db;
// Add data
TimeSeriesData data;
data.timestamp = 1234567890000;
data.value = 42.5;
data.tags["sensor"] = "temp_01";
db.add(data);
// Query data
TimeRange range{1234567890000, 1234567900000};
auto results = db.query(range);
// Use algorithms
StreamJoin join(5000); // 5-second window
auto joined = join.process(left_stream, right_stream);
return 0;
}
Python API
import sage_tsdb
# Create database
db = sage_tsdb.TimeSeriesDB()
# Add data
db.add(timestamp=1234567890000, value=42.5,
tags={"sensor": "temp_01"})
# Query data
results = db.query(start_time=1234567890000,
end_time=1234567900000)
# Stream join
join = sage_tsdb.StreamJoin(window_size=5000)
joined = join.process(left_stream, right_stream)
🔌 Pluggable Algorithms
Implementing Custom Algorithms
#include <sage_tsdb/algorithms/algorithm_base.h>
class MyAlgorithm : public TimeSeriesAlgorithm {
public:
MyAlgorithm(const AlgorithmConfig& config)
: TimeSeriesAlgorithm(config) {}
std::vector<TimeSeriesData> process(
const std::vector<TimeSeriesData>& input) override {
// Your algorithm implementation
return output;
}
};
// Register algorithm
REGISTER_ALGORITHM("my_algorithm", MyAlgorithm);
🧪 Testing
# Run all tests
cd build
ctest -V
# Run specific test
./tests/test_time_series_db
./tests/test_stream_join
📊 Performance
Benchmarks on typical hardware (Intel i7, 16GB RAM):
| Operation | Throughput | Latency |
|---|---|---|
| Single insert | 1M ops/sec | < 1 μs |
| Batch insert (1000) | 5M ops/sec | < 200 ns/op |
| Query (1000 results) | 500K queries/sec | 2 μs |
| Stream join | 300K pairs/sec | 3 μs |
| Window aggregation | 800K windows/sec | 1.2 μs |
🔗 Integration with SAGE
This library is designed to be used as a submodule in the SAGE project:
# In SAGE repository
git submodule add https://github.com/intellistream/sageTSDB.git \
packages/sage-middleware/src/sage/middleware/components/sage_tsdb/sageTSDB
git submodule update --init --recursive
📚 Documentation
🤝 Contributing
Contributions are welcome! Please read our Contributing Guide for details.
📄 License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
🔗 Links
📮 Contact
For questions and support:
- GitHub Issues: https://github.com/intellistream/sageTSDB/issues
- Owner: Debin Chen (@pluviophile-chen)
- GitHub: pluviophile-chen
Metadata
Release files for isage-tsdb 0.1.6.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| isage_tsdb-0.1.6.8-cp311-cp311-manylinux_2_34_x86_64.whl | CPython 3.11 | CPython 3.11 | Linux glibc 2.34+ x86-64 | Details for isage_tsdb-0.1.6.8-cp311-cp311-manylinux_2_34_x86_64.whl |
Release files / isage_tsdb-0.1.6.8-cp311-cp311-manylinux_2_34_x86_64.whl
| Download URL | isage_tsdb-0.1.6.8-cp311-cp311-manylinux_2_34_x86_64.whl |
|---|---|
| Size | 1.4 MB |
| Tags | CPython 3.11 Linux glibc 2.34+ x86-64 |
|
SHA-256 checksum How to use checksums |
ab5ffe555c55194f5ed40a709108894d3977cb660bccb775b3d9182a7944c048
|
|
BLAKE2b-256 checksum How to use checksums |
2d1b8ae4d8277ca84b5c27df10775e06d9d3d0a67f5492dd24dfe360d5d6474c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.14
|
Security for isage-tsdb 0.1.6.8
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.