Auto-saving dicts and lists
Project description
pickledict
pickledict
/jsondict
and picklelist
/jsonlist
are auto-saving wrappers around their contained types. Their API is the same as the underlying type, dict
or list
, but writes are recorded to a file and are transparently re-loaded on subsequent runs.
from pickledict import jsondict
my_dict = jsondict(some_key="a value")
# Writes and modifications are automatically recorded and saved
my_dict["dog"] = 1
my_dict["dog"] += 1
print(my_dict)
# The first invocation creates a save file for the dict
$ python example.py
{'some_key': 'a value', 'another_key': 1}
# The data is saved to a file named: <filename>-<lineno>-<variable name>.json
$ cat .pickledict/example-3-my_dict.json
{"some_key": "a value", "dog": 2}
# A second invocation reads the saved file on construction
$ python example.py
{'some_key': 'a value', 'another_key': 2}
# As does a third, etc.
$ python example.py
{'some_key': 'a value', 'another_key': 3}
Installation
# requires Python 3.6+
pip install pickledict
Usage
dict
s can be saved with either JSON or Python's pickle format (Pickle supports the preservation of certain things JSON does not, such as non-string keys).
All of pickledict
, jsondist
, picklelist
, and jsonlist
support the same constructor arguments:
file_name
(str
) - explicitly specify the file name to save this object in (if not provided one will be generated from the assigned variable name if possible)save_on_every_write
(bool
) - save on every modification of the structure. Turning this off saves only on program exit (which may increase performance)serializer_args
(Dict[str, Any]
) - keyword arguments to pass to the relevant serializationdump
function
They all each add one method: detach()
which disconnects the in-memory object from the file on disk so it can be manipulated without worrying about affecting the underlying file.
Use Cases
Fetch data from a remote source and store it locally automatically without dealing directly with serialization:
Without pickledict
:
import json
import os
if os.path.exists("db.json"):
with open("db.json", "r") as f:
local_db = json.load(f)
else:
local_db = {}
for i in range(100):
key, data = fetch_some_data(i)
local_db[key] = data
with open("db.json", "w") as f:
json.dump(local_db, f, indent=2)
With pickledict
:
from pickledict import jsondict
local_db = jsondict(file_name="db.json", serializer_args={"indent": 2})
for i in range(100):
key, data = fetch_some_data(i)
# This will automatically persist the data out to disk
local_db[key] = data
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pickledict-1.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c7ea7b6781fbba08d42cc1daae951cb5f2cb92465a8f18cb58c742a4175bc7d3 |
|
MD5 | 2df14f56bf61579627cc3f52dfc07d2f |
|
BLAKE2b-256 | 894b4773e58fb0d01c9a1f288b5e3a894a822511797c651e90293b1380c8a197 |