Skip to main content

stockpyle is simple multi-layered storage and caching API

Project description


stockpyle provides a simple way to set up a series of storage containers for the purposes of creating simple write-through cache storage.


Simplest script that sets up a two-level cache with memcached and local process memory:

from stockpyle import ChainedStore, MemcacheStore, ProcessMemoryStore

# instantiate the ChainedStore as a write-through cache
pm = ProcessMemoryStore()
mc = MemcacheStore(servers=["localhost:11711"])
store = ChainedStore([pm, mc])

# declare a class that is unique for each (bar,zap) combination
class Foo(object):

    __stockpyle_keys__ = [("bar", "zap")]

    def __init__(self, bar, zap): = bar
        self.zap = zap

# create and save a Foo to the ChainedStore
obj = Foo(bar=444, zap=888)

# retrieve a Foo from the store, based on the (bar,zap) combination
# this will hit the local memory cache first, and will avoid memcache
# since the object is already cached there
retrieved_obj = store.get(Foo, {"bar": 444, "zap": 888})

This example isn’t that interesting, since we are using two caches. Let’s do one that supports SQLAlchemy objects:

from stockpyle.stores import ChainedStore, SqlAlchemyStore, MemcacheStore, ProcessMemoryStore

pm = ProcessMemoryStore()
mc = MemcacheStore(servers=["localhost:11711"])
sa = SqlAlchemyStore()
store = ChainedStore([pm, mc, sa])

# store it, this will write it through the cache and into the database
persistent_obj = MySqlAlchemyBackedClass()

Note the ordering in the ChainedStore declaration: the SqlAlchemyStore comes last, since it acts as the final persistence layer. Subsequent ‘get’ requests will attempt process memory, then attempt memcache, and finally check the database.

Want to grab a bunch of objects? Use batch_get:

obj1, obj2, obj3 = store.batch_get(Foo, [
    {"foo": 111, "bar": 777},
    {"foo": 222, "bar": 888},
    {"foo": 333, "bar": 999},

Want to store a bunch of objects? Use batch_put:

obj1 = Foo(111, 777)
obj2 = Foo(222, 888)
obj3 = Foo(333, 999)
store.batch_put([obj1, obj2, obj3])

Deleting objects is easy (batch deletes coming soon):


Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
stockpyle-0.0.5.tar.gz (60.6 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page