Skip to main content

Transparent Redis caching for raw SQL connections - no ORM required

Project description

ShadowCache

ShadowCache Banner

Write SQL. Get caching. Nothing else.

ShadowCache wraps your MySQL connection and transparently caches SELECT results in Redis. INSERT, UPDATE, or DELETE statements automatically evict affected cache entries so your reads never serve stale data. No ORM. No boilerplate. No config.


Table of Contents

The Problem

Every developer who writes raw SQL eventually writes this:

# 8 lines of boilerplate for every cached query
cache_key = f"user:{user_id}"
cached = redis.get(cache_key)
if cached:
    return json.loads(cached)

cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))
row = cursor.fetchone()
redis.set(cache_key, json.dumps(row), ex=300)
return row

And on every INSERT, UPDATE, or DELETE you need to remember:

cursor.execute("UPDATE users SET name = %s WHERE id = %s", (name, user_id))
redis.delete(f"user:{user_id}")  # easy to forget, easy to get wrong
- Boilerplate for every query
- Manual invalidation you will forget
+ One line. Caching and invalidation are automatic.

With ShadowCache:

cursor, rows = cache.execute("SELECT * FROM users WHERE id = %s", (42,))
cursor, _ = cache.execute("UPDATE users SET name = %s WHERE id = %s", ("Alice", 42))

Features

Zero-schema caching Works with any MySQL table, any query. No model definitions needed.
Write-triggered eviction INSERT, UPDATE, and DELETE automatically evict cached SELECTs for the same table.
TTL safety net Cached entries expire after a configurable time-to-live. Eventual consistency guaranteed.
Graceful fallback If Redis is unreachable, queries still execute against MySQL.

Installation

pip install shadowcache

Or install from source:

git clone https://github.com/pratham2402/ShadowCache.git
cd ShadowCache
pip install -r requirements.txt

Requires: Python 3.8+, Redis, MySQL.

Quick Start

import mysql.connector
from shadowcache import ShadowCache

conn = mysql.connector.connect(
    host="localhost", database="my_app",
    user="app_user", password="secret",
)

cache = ShadowCache(conn)

# Cold miss -- hits MySQL, stores in Redis
cursor, rows = cache.execute("SELECT * FROM users WHERE id = %s", (42,))

# Warm hit -- returns from Redis instantly
cursor, rows = cache.execute("SELECT * FROM users WHERE id = %s", (42,))

# Write evicts the cache
cache.execute("UPDATE users SET name = %s WHERE id = %s", ("Alice", 42))

# Cache was evicted -- fresh data from MySQL
cursor, rows = cache.execute("SELECT * FROM users WHERE id = %s", (42,))

API Reference

ShadowCache(db_connection, *, ...)

ShadowCache(
    db_connection,
    *,
    redis_client=None,
    redis_host="localhost",
    redis_port=6379,
    ttl=300,
    auto_invalidate=True,
    key_prefix="shadowcache",
)
Parameter Default Description
db_connection (required) An open DB-API2 MySQL connection
redis_client None Pre-configured redis.Redis instance; created automatically if omitted
redis_host "localhost" Redis hostname
redis_port 6379 Redis port
ttl 300 Cache TTL in seconds
auto_invalidate True Whether writes automatically evict related cache entries
key_prefix "shadowcache" Namespace prefix for all Redis keys

ShadowCache.execute(sql, params=None)

Returns (cursor, rows).

SQL Behaviour
SELECT Checks Redis first. Hit returns (None, cached_rows). Miss executes on MySQL, caches, returns (cursor, rows).
INSERT Executes on MySQL. Returns (cursor, None). See cursor.lastrowid.
UPDATE / DELETE Executes on MySQL, evicts cache for affected tables. Returns (cursor, None). See cursor.rowcount.
DDL / other Executes on MySQL. No caching, no eviction.

Other Methods

Method Description
invalidate_table(name) Evict all cached entries for a table. Returns count of keys removed.
flush_cache() Remove all ShadowCache keys from Redis. Returns count of keys removed.
stats Property. Returns a dict with keys hits, misses, total_requests, hit_ratio.
close() Close the wrapped database connection.

Configuration

Copy .env.example to .env and set your credentials:

REDIS_HOST=localhost
REDIS_PORT=6379
MYSQL_HOST=localhost
MYSQL_PORT=3306
MYSQL_USER=your_db_user
MYSQL_PASSWORD=your_db_password
MYSQL_DATABASE=your_database
LOG_LEVEL=INFO

Running Tests

# Unit tests -- no Redis or MySQL needed, all mocks
python -m pytest tests/ -v

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

shadowcache-0.1.0.tar.gz (15.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

shadowcache-0.1.0-py3-none-any.whl (11.6 kB view details)

Uploaded Python 3

File details

Details for the file shadowcache-0.1.0.tar.gz.

File metadata

  • Download URL: shadowcache-0.1.0.tar.gz
  • Upload date:
  • Size: 15.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for shadowcache-0.1.0.tar.gz
Algorithm Hash digest
SHA256 91afb6f61c3115f0236a362b06845c65d0db49e82683f6466f8464baa8a07de9
MD5 d9233b6119d0033b553b95ed15e28a3d
BLAKE2b-256 1cf8c39d96470ee24fd250187572842dd54afc2897ab894f519396ee052808b3

See more details on using hashes here.

File details

Details for the file shadowcache-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: shadowcache-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 11.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for shadowcache-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 414555027240f97b694df09dba0cc87245f3104e941f8290b264d563c7538096
MD5 0cc7d6af8ebc8773348caef52aa4cfcd
BLAKE2b-256 0e336bc20e0dca9c45e8609ffaea86a422b1e8860583e5533ee04c4a1f35c821

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page