splitcache

A tiny “two-tier” caching decorator for integer-keyed functions:

• Small inputs (≤ cutoff) are cached entirely using functools.cache

• Large inputs (> cutoff) are either:

  • not cached (default when lru_size is None/0), or
  • cached with a bounded LRU using functools.lru_cache(maxsize=...)

This is useful when you have hot paths that repeatedly hit a dense “small” region, but also wander into huge integers (e.g., > $10^{20}$) where unbounded caching would explode memory.

Installation

pip install splitcache

Quick start

from splitcache import split_cache

@split_cache(cutoff=10**9, lru_size=10**6)
def nextprime(p: int) -> int:
    ...

Behavior:

• If p <= 10**9: cached completely
• If p > 10**9: cached in an LRU of size $10^6$

To disable caching above the cutoff:

@split_cache(cutoff=10**9, lru_size=None)
def nextprime(p: int) -> int:
    ...

When lru_size is None (or 0), values above the cutoff are recomputed every time.

Note about memory size

For a method with a signature (int, ) -> int with numbers below $10^{45}$ or so, each cache entry below the cutoff is measured at approximately 130 bytes, and above the cutoff each cache entry is measured at approximately 190 bytes.

If any number below $10^9$ is provided to this method, that would be a cache size of over 120GB in memory. If restricted to just primes: there are approximately 48 million primes below $10^9$, which would be a cache size of nearly 6GB. The primes can be further restricted, eg primes $\equiv 1 \bmod 4$.

The point of all this is to express the importance of being aware of the potential size of the caches you are creating.

API

split_cache

• cutoff (required): cache all calls where cutoff_value(*args, **kwargs) <= cutoff. Must be > 0.

• lru_size (optional): LRU size for calls above the cutoff.

  • None or 0 ⇒ no cache above cutoff
  • > 0 ⇒ functools.lru_cache(maxsize=lru_size) above cutoff

• cutoff_value (optional): function that chooses which numeric value to compare against cutoff. Defaults to default_cutoff_value.

Cache controls

split_cache exposes a few helpers on the wrapped function:

• fn.small_cache_clear(): clear the unbounded “small” cache
• fn.big_cache_clear(): clear the “big” LRU cache if enabled
• fn.small_cache_info / fn.big_cache_info: cache info functions when available

Example:

next_prime_1mod4.small_cache_clear()
next_prime_1mod4.big_cache_clear()

# If big cache is enabled:
info = next_prime_1mod4.big_cache_info()
print(info)