Adaptive Pipeline Cache
A high-performance adaptive caching system implemented in C++ with Python bindings. The cache dynamically adjusts its internal structure based on workload characteristics, combining FIFO, Approximate LRU (ALRU), and Cost-Aware LFU strategies.
Features
- Adaptive Architecture: Automatically adjusts cache structure based on workload patterns
- Multiple Eviction Policies: Combines FIFO, ALRU, and Cost-Aware LFU in a pipeline
- High Performance: Implemented in C++20 with efficient memory management
- Python Integration: Clean Python API via pybind11
- Cost-Aware: Takes into account both latency and token costs for eviction decisions
Installation
pip install adaptive-pipeline
Building from Source
Requirements:
- C++20 compatible compiler
- CMake >= 3.15
- Python >= 3.7
- pybind11 >= 2.6.0
git clone <repository-url>
cd pipeline-cache
pip install -e .
Usage
Notice that for this version, the size of the cache is hard-coded to be 1024, and it ignores the size parameter.
from adaptive_pipeline import AdaptivePipelineCache
# Create a cache with capacity of 1024 items
cache = AdaptivePipelineCache(1024)
# Store items with (latency, tokens) tuple
cache[key] = (latency, tokens)
# Check if key exists
if key in cache:
latency, tokens = cache[key]
# Get cache statistics
print(f"Current size: {cache.currsize}")
print(f"Max size: {cache.maxsize}")
How It Works
The Adaptive Pipeline Cache uses a novel approach that:
- Pipeline Structure: Divides the cache into three blocks (FIFO, ALRU, Cost-Aware LFU)
- Dynamic Adaptation: Periodically evaluates alternative configurations using "ghost caches"
- Quantum-Based Resizing: Moves chunks of items between blocks based on performance metrics
- Cost-Aware Eviction: Considers both access frequency and cost (latency × tokens) for eviction decisions
The cache automatically adapts its configuration every 10,240 operations (configurable) by comparing the performance of the current configuration against alternative configurations simulated by ghost caches.
Interface Compatibility
The implementation follows the interface of cacheutils for compatibility with GPT-Cache experiments.
License
MIT License - see LICENSE file for details
Author
Nadav Keren (nadavker@pm.me)
1 maintainer
Metadata
1 maintainer
Release files for adaptive-pipeline 0.1.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| adaptive_pipeline-0.1.5.tar.gz | 22.3 kB | Without provenance | Details |
Built distributions (wheels)
Total release size: 3.9 MB
Release files / adaptive_pipeline-0.1.5.tar.gz
| Download URL | adaptive_pipeline-0.1.5.tar.gz |
|---|---|
| Size | 22.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3adaeb0570c9942f82ace4db450b1329e5b6e7ccb78a12ed8105688753c0ad52
|
|
BLAKE2b-256 checksum How to use checksums |
62d204324ad8d6bb83e6b1eb8fdda0cfacd3284fb76c229b7b992373888179a1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp313-cp313-win_amd64.whl
| Download URL | adaptive_pipeline-0.1.5-cp313-cp313-win_amd64.whl |
|---|---|
| Size | 153.9 kB |
| Tags | CPython 3.13 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
5e344f04ecf4b87ba25e103527e96ea7f7d34f172269bfd18ba204e00d6982a3
|
|
BLAKE2b-256 checksum How to use checksums |
9d20ca87b01e0275c612bfeb529492552327966613e6397efff6ddc703b53ef5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 186.3 kB |
| Tags | CPython 3.13 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
a3f1dfa7e8204bcb0f6ee59897969b8a9a105231f664ada51ceb5f1779df0864
|
|
BLAKE2b-256 checksum How to use checksums |
72e17b15a3b1fdfcf11604550ca78a6f19b5974dcf7d620e2be291807882be1d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp313-cp313-macosx_11_0_arm64.whl
| Download URL | adaptive_pipeline-0.1.5-cp313-cp313-macosx_11_0_arm64.whl |
|---|---|
| Size | 149.1 kB |
| Tags | CPython 3.13 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
ba9ef582bbcca4ab474d4b24abd29a986faae0f8e7655245346fd505414b5be7
|
|
BLAKE2b-256 checksum How to use checksums |
5dc22dbdd1c8c4e0b02eddafe9f6bb122f407a3e8913fe3be053f7bb6e5b15d1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp313-cp313-macosx_10_15_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp313-cp313-macosx_10_15_x86_64.whl |
|---|---|
| Size | 169.5 kB |
| Tags | CPython 3.13 macOS 10.15+ x86-64 |
|
SHA-256 checksum How to use checksums |
03be23caeca8b9fcaa448f194df65c295abc7d2946bc8e763a3fb21cca0776e8
|
|
BLAKE2b-256 checksum How to use checksums |
a45e93c4abd291529c9ba45565f014a5bf95d10862241f771fd853bc23130b2e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp312-cp312-win_amd64.whl
| Download URL | adaptive_pipeline-0.1.5-cp312-cp312-win_amd64.whl |
|---|---|
| Size | 153.9 kB |
| Tags | CPython 3.12 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
11ade74f32d0b33503cca444856b98c68146b8dacf0876aa1c6f0c9c7ab17cb0
|
|
BLAKE2b-256 checksum How to use checksums |
7df3b1b54ee484eca4b83690283adca8e78a31e35ef54dae0f107b87db25a957
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 186.5 kB |
| Tags | CPython 3.12 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
661e59e5779da49c1a18e1507b9fb61026eb74d0ae14dd2c87ff335619e16934
|
|
BLAKE2b-256 checksum How to use checksums |
56fce191c0044873431cf91086bdc573180b932697463409e6d8623e61a1b8e1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp312-cp312-macosx_11_0_arm64.whl
| Download URL | adaptive_pipeline-0.1.5-cp312-cp312-macosx_11_0_arm64.whl |
|---|---|
| Size | 149.0 kB |
| Tags | CPython 3.12 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
88a935dfe0005103e120fed74e6e7963b9de4c752fc897f040f375561db0e2b2
|
|
BLAKE2b-256 checksum How to use checksums |
f8b29d57f5a0ab27626ca71958bb6b4bf00a273672c4c6d7d87bf41b1dee0dfd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp312-cp312-macosx_10_15_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp312-cp312-macosx_10_15_x86_64.whl |
|---|---|
| Size | 169.5 kB |
| Tags | CPython 3.12 macOS 10.15+ x86-64 |
|
SHA-256 checksum How to use checksums |
712009db6eb603ddb726c81475cba72de2421d47b410253366660d6b8580e902
|
|
BLAKE2b-256 checksum How to use checksums |
afdcce140985cbf619790960e040c0681f5fefb24f90066682e37c86523a2bba
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp311-cp311-win_amd64.whl
| Download URL | adaptive_pipeline-0.1.5-cp311-cp311-win_amd64.whl |
|---|---|
| Size | 152.7 kB |
| Tags | CPython 3.11 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
3f3bc931259300b69b8f436790fcbd4a41635da909a633a0e70bb9d84901c838
|
|
BLAKE2b-256 checksum How to use checksums |
d441b9cf9f9c81c2735094b7fa47ed1ddb77e0e1f74141698b59c91ffa0daac5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 186.5 kB |
| Tags | CPython 3.11 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
8333f425f16b7f436bf1b82ff63cfa2854399bf12a1886246c7a548d1f907e01
|
|
BLAKE2b-256 checksum How to use checksums |
bd0c5ab2994f91ac796e2fd5340d21351bc5180138b9fcfa7c1bd296e356bce3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp311-cp311-macosx_11_0_arm64.whl
| Download URL | adaptive_pipeline-0.1.5-cp311-cp311-macosx_11_0_arm64.whl |
|---|---|
| Size | 148.2 kB |
| Tags | CPython 3.11 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
4814a56e60f9bff6b9d43e9b845f268cbbbed2ad6e199271fa676e2f99e91bc1
|
|
BLAKE2b-256 checksum How to use checksums |
773867cc6d554d4cf967a6599c489e29b029694fa2c799a87deddb79cdf987ab
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp311-cp311-macosx_10_15_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp311-cp311-macosx_10_15_x86_64.whl |
|---|---|
| Size | 168.0 kB |
| Tags | CPython 3.11 macOS 10.15+ x86-64 |
|
SHA-256 checksum How to use checksums |
4b80c534d4b30c873c68d57e6129c7bca48a5cf7754d5c24235daef34cea59c0
|
|
BLAKE2b-256 checksum How to use checksums |
b72ab243fda46212bcc6859c6a76edfa98aaef0c65613134bf3a065ddaf95a9e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp310-cp310-win_amd64.whl
| Download URL | adaptive_pipeline-0.1.5-cp310-cp310-win_amd64.whl |
|---|---|
| Size | 152.1 kB |
| Tags | CPython 3.10 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
adc978b963ea0fe24f4eab220a1fadd870a5abc7a09f32c90c741b945d688ce2
|
|
BLAKE2b-256 checksum How to use checksums |
ed3f82beee720fb909ca00138fc001ed061f36634b4906d9a396f7626f183a9f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 185.0 kB |
| Tags | CPython 3.10 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
778b182e44522733442c5673e399c902cfc63f70d7281ad94477d5f3506c30e6
|
|
BLAKE2b-256 checksum How to use checksums |
689fe9b5653b83f07d5324fdebb563440d469fa7b6ea03f293338657ba0ef243
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp310-cp310-macosx_11_0_arm64.whl
| Download URL | adaptive_pipeline-0.1.5-cp310-cp310-macosx_11_0_arm64.whl |
|---|---|
| Size | 146.7 kB |
| Tags | CPython 3.10 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
b9a5b6fcaf5e342e064f67c8206f075873f97c606082bbc802916d8ed09297a0
|
|
BLAKE2b-256 checksum How to use checksums |
e49e4d3cc9002460d543fdda5cfafa018bc8f262d5f25086a8bd7680832566b7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp310-cp310-macosx_10_15_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp310-cp310-macosx_10_15_x86_64.whl |
|---|---|
| Size | 166.4 kB |
| Tags | CPython 3.10 macOS 10.15+ x86-64 |
|
SHA-256 checksum How to use checksums |
251333e7c57323b211feff26b0cf51bf572afdd3fd1def1b2c04e77ae6c904d3
|
|
BLAKE2b-256 checksum How to use checksums |
f599afa2bb979b89335eefd29575a639ce79ae049b6b78dcbadd7f4908377f29
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp39-cp39-win_amd64.whl
| Download URL | adaptive_pipeline-0.1.5-cp39-cp39-win_amd64.whl |
|---|---|
| Size | 154.1 kB |
| Tags | CPython 3.9 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
9aa2aa84c66a5c3bf17d48f3dd010cec2da55bcc2f3a1e2410a50de4445b15a5
|
|
BLAKE2b-256 checksum How to use checksums |
4c733f354e3967be2a97fba88fc4d4f496dd5544fb57cb8f853dc7b6cfa2a7b0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 185.4 kB |
| Tags | CPython 3.9 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
066194fe6a945da79f18cc3c5eea274c4be367151cef97ec09d2b6d0bd5a9059
|
|
BLAKE2b-256 checksum How to use checksums |
ef199ac6cfe6bbce8bd4783631d23205a93ed4ff77a61f23d9d6562546f78f8a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp39-cp39-macosx_11_0_arm64.whl
| Download URL | adaptive_pipeline-0.1.5-cp39-cp39-macosx_11_0_arm64.whl |
|---|---|
| Size | 146.7 kB |
| Tags | CPython 3.9 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
d33f2856078fc2fed27054ef51f90ad9a38948f7d2aa2b10bbf60b4337a0e19a
|
|
BLAKE2b-256 checksum How to use checksums |
d83683db9f18387aaaa3f169307b29a2191029285f4527a6c0b2259770cb527d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp39-cp39-macosx_10_15_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp39-cp39-macosx_10_15_x86_64.whl |
|---|---|
| Size | 166.5 kB |
| Tags | CPython 3.9 macOS 10.15+ x86-64 |
|
SHA-256 checksum How to use checksums |
a54780d820d6bae56fdcf16beed1fa9f7e2bbd1692187ed3e089bad973d095a0
|
|
BLAKE2b-256 checksum How to use checksums |
ce1cf4970006c13304ff60966008e337ff287f64cbc0acb5e7d7695ccb6df89d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp38-cp38-win_amd64.whl
| Download URL | adaptive_pipeline-0.1.5-cp38-cp38-win_amd64.whl |
|---|---|
| Size | 151.9 kB |
| Tags | CPython 3.8 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
bbd68d1b88732129afb7d6c1e468555757b45756671caa6e6ed4159e478ba64e
|
|
BLAKE2b-256 checksum How to use checksums |
041f113199cf09925f7ac970d9360cb3b2aad96af1964f32261c596b9459cbf9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 184.9 kB |
| Tags | CPython 3.8 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
210dae4a7a3fbdd4582d41ce51d7e6dd4a994640f5b23ace4e5ea3c7e8e49bae
|
|
BLAKE2b-256 checksum How to use checksums |
6e4be3a7709fe61a39bdc90c3887b7dbbda43067550558967735c9aa3dcb9b13
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp38-cp38-macosx_11_0_arm64.whl
| Download URL | adaptive_pipeline-0.1.5-cp38-cp38-macosx_11_0_arm64.whl |
|---|---|
| Size | 146.6 kB |
| Tags | CPython 3.8 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
855696183cfdd5b92f760246b6a9e92e8ab674392d97c820718898c1fe775288
|
|
BLAKE2b-256 checksum How to use checksums |
f87777f3f2250b1b172e440b8a15aad5f76370df128aebbc765903d37a8b3305
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / adaptive_pipeline-0.1.5-cp38-cp38-macosx_10_15_x86_64.whl
| Download URL | adaptive_pipeline-0.1.5-cp38-cp38-macosx_10_15_x86_64.whl |
|---|---|
| Size | 166.3 kB |
| Tags | CPython 3.8 macOS 10.15+ x86-64 |
|
SHA-256 checksum How to use checksums |
250fd62439ef48c2b37fb9bf62fff81bf0344f5c9b315e7c303b5f935a6358e9
|
|
BLAKE2b-256 checksum How to use checksums |
5d0fe7b8d40ef5d569ce8e599c193b8118639cc6939991aecb9b4f39bb71559a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Security for adaptive-pipeline 0.1.5
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
