expiringsqlitedict -- expiring file-backed `dict`

A lightweight wrapper around Python's sqlite3 database with a MutableMapping interface:

from expiringsqlitedict import SqliteDict
with SqliteDict('./my_db.sqlite') as mydict:
    mydict['some_key'] = any_picklable_object
    print(mydict['some_key'])  # prints the new value
    for key, value in mydict.items():
        print((key, value))
    print(len(mydict)) # etc... all dict functions work

json is used internally by default to serialize the values. Keys are arbitrary strings, values arbitrary json-able objects. This must be used within a context manager, and serialization can be overridden with your own. The database is wrapped with a transaction, and any exception thrown out of the context manager rolls back all changes.

This was forked off of sqlitedict in order to add auto-expiring functionality, and initially was quite similar to it. Version 2.0 split of completely and takes the module into a complete rewrite, mostly to remove unnecessary Python 2 compatibility, simplify the API, completely enforce a context manager for typical cases, add full typing throughout, and use sqlite triggers for expiration cleanup.

Version 3 set the default encoding to json, and made many other API refinements.

This version also does not vacuum at all automatically. It did in previous versions, but this was kind of a silly behavior to put into the library itself. If you want your database file intermittently vacuumed, you should put such behavior into a crontab or use the sqlite3 module to do it yourself intermittently.

Features

Values can be any json-capable objects (this can be customized to be as flexible as you need, through custom serializers)
Support for access from multiple programs or threads, with locking fully managed by sqlite itself.
A very simple codebase that is easy to read, relying on sqlite for as much behavior as possible.
A simple autocommit wrapper (SimpleSqliteDict), if you really can't handle a context manager and need something that fully handles like a dict. You can specify a isolation_level on this to have to commit and roll back yourself.
An on-demand wrapper (OnDemand), for situations where you want to open and close the database in as narrow a window as possible.
Support for custom serialization or compression:

import orjson
  
with SqliteDict('some.db', serializer=orjson) as mydict:
    mydict['some_key'] = some_json_encodable_object
    print(mydict['some_key'])

Installation

The module has no dependencies beyond Python itself.

Install with pip as usual:

pip install expiringsqlitedict

Testing

You may test this by running test.py with PYTHONPATH set to the current working directory. There is a convenience justfile to do this for you when you run:

just test

This will run all tests against all supported versions of Python (and 3.6, for CentOS 7 support), as well as all supported versions of Alpine, CentOS, Debian, Fedora, and Ubuntu Linux.

Documentation

Documentation is available on readthedocs

Standard Python document strings are inside the module

>>> import expiringsqlitedict
>>> help(expiringsqlitedict)

Comments, bug reports

expiringsqlitedict resides on github. You can file issues or pull requests there.

Release files for expiringsqlitedict 7.0.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for expiringsqlitedict 7.0.5
File	Size	Uploaded
expiringsqlitedict-7.0.5.tar.gz	21.7 kB	May 10, 2024	Details

Built distribution (wheel)

Table of built distributions (wheels) for expiringsqlitedict 7.0.5
File	Interpreter	ABI	Platform	Reset
expiringsqlitedict-7.0.5-py3-none-any.whl 13.4 kB May 10, 2024	Python 3	none	any	Details

Total release size: 35.1 kB

Release files / expiringsqlitedict-7.0.5.tar.gz

Download URL	expiringsqlitedict-7.0.5.tar.gz
Size	21.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`5b38abd530f6e9a36e17ae6b05c72d63c6a533b836299672374b4090fbb27fc3`
BLAKE2b-256 checksum How to use checksums	`0feeeb2561b6fc2bb71a2f799603f50304d9d3245cd4400af1527efadc86f0f0`
Upload date	May 10, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`python-requests/2.31.0`

Release files / expiringsqlitedict-7.0.5-py3-none-any.whl

Download URL	expiringsqlitedict-7.0.5-py3-none-any.whl
Size	13.4 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`a2f28a5d3fd5ce966029f878c671f166fa632a8955782927587087999a4cd00d`
BLAKE2b-256 checksum How to use checksums	`fd072347a3ae5e7b3e8ceeba94eed1a1cff7a963465e5139113547152172904c`
Upload date	May 10, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`python-requests/2.31.0`