Object Cache

Size-bounded object cache with pluggable eviction policies.

Overview

VCollab applications frequently need to hold computed or loaded objects in memory with a bounded size budget. When the budget is exceeded, old entries must be evicted according to a configurable policy.

The vcti-cache package provides ObjectCache, a generic, size-bounded cache with a pluggable eviction policy defined by the abstract CachePolicy base class. Implement CachePolicy to control how item sizes are calculated and which items are evicted first.

This package has zero external dependencies.

Installation

From PyPI

pip install vcti-cache

In requirements.txt

vcti-cache>=2.0.0

In pyproject.toml dependencies

dependencies = [
    "vcti-cache>=2.0.0",
]

Quick Start

from vcti.cache import ObjectCache, LruPolicy

# 1. Create a cache with a built-in LRU policy
cache = ObjectCache[str](max_size=100, cache_policy=LruPolicy(get_size=len))

# 2. Add, retrieve, and remove items
cache.add("greeting", "hello")
print(cache.get("greeting"))   # "hello"
print(cache["greeting"])       # "hello"  (bracket syntax)
print("greeting" in cache)     # True
print(len(cache))              # 1
print(cache.current_size)      # 5

# 3. Update (upsert) an existing item
cache.update("greeting", "hi there")
print(cache.get("greeting"))   # "hi there"

# 4. Inspect statistics
print(cache.stats.hits)        # 3  (two gets + one bracket access)
print(cache.stats.adds)        # 2  (original add + update)

cache.remove("greeting")
print(cache.get("greeting"))   # None

Five bundled policies are available — FifoPolicy, LruPolicy, LfuPolicy, MruPolicy, and RrPolicy — each accepting a get_size callable. Give each cache its own policy instance; policies keep per-key bookkeeping, and constructing a second cache around one raises ValueError. For custom eviction logic, subclass CachePolicy[T] directly (see Extending).

get_size is called once per insertion, and that number is what the cache budgets against from then on. Mutating a cached value in place does not re-measure it — call update(key, value) to record the new size. Eviction policies receive those recorded sizes and must not call get_size themselves.

Thread Safety

ObjectCache is not thread-safe. If you share a cache across threads, wrap it with an external lock (e.g. threading.Lock).

Public API

ObjectCache[T]

CachePolicy[T] (abstract)

CacheStats

Bundled Policies

All bundled policies accept get_size: Callable[[T], int] in their constructor.

Each policy also decides what a replacing update() means for its eviction order: LruPolicy and MruPolicy treat the write as a use and refresh recency, while FifoPolicy, LfuPolicy, and RrPolicy leave the entry's standing untouched — refreshing a hot LFU entry does not reset its access count.

Start with LruPolicy — it's the best general-purpose default. See Choosing a Policy for a detailed comparison of performance, trade-offs, and when to use each.

Dependencies

None. vcti-cache is pure Python with zero runtime dependencies.

Documentation