File json storage backend for your collections
Project description
Slowstore
This is simple,single file, key-value store that stores your objects as JSON on file system.
It is designed to be easy to plug into your program, no servers, no connection strings, nothing, just set the directory where you would like to store your files.
Slowstore is slow, it is intended to be used for exploration.
Installation
You can install Slowstore using pip:
pip install slowstore
How is the data organized?
The data is stored in a directory, with each item stored in a separate file. The file name is the key of the item, and the content is the JSON representation of the object.
That's all there is to it.
Features
At the moment Slowstore works with Pydantic models, but I plan to add support for other types of objects that are serializable to JSON.
- Save on change: Set the store to save the data every time a property is changed
- Undo/Redo: Undo and redo changes to the object
- Dirty: Check if the object has been changed
- Filtering: Filter/Query the items in the store
- Deleting: Delete items from the store
- Commit: Commit changes to the store
- Partial load: Load only the items you need, and lazy load the rest
- Transactions: Add support for transactions
- Non-Pydantic objects: Add support for other types of objects
- Indexes: Add indexes to the store to speed up queries
Motivation
I created Slowstore because I wanted a simple way to store my objects on disk, without having to worry about databases, connection strings, or any of that. I want to be able to inspect the data, see what is going on with my program undo changes if needed, and filter the data easily.
Usage
You can create a new Slowstore instance by calling the constructor:
from slowstore import Slowstore
class SampleModel(BaseModel):
name: str
age: int = 0
# Create a new store and load the data
**One feature that I use a lot is to commit on property change, so I can inspect exactly what is going on**
```python
store = Slowstore[SampleModel](SampleModel, "mydata", save_on_change=True)).load()
You can also enable/disable the flag at any time after the store is created
store.save_on_change = True
After this code is executed, every time a property is changed, the changes will be saved to disk immediately.
This is slow, but it is useful for debugging and testing.
Adding items to the store
s1 = store.upsert(SampleModel(name="Alice", age=30))
s2 = store.upsert(SampleModel(name="Bob", age=25))
s3 = store.upsert(SampleModel(name="Charlie", age=35))
s1.age = 32
Commit the changes to the store
If the store is not set to save on change, you can commit the changes manually.
store.commit(s1)
store.commit(s2)
# or just commit all changes
store.commit_all()
Undo/Redo
Every slowstore item is a proxy object around the object you added to the store. This allows you to access the object's attributes and adds undo/redo/dirty functionality
s1.undo()
s1.redo()
Querying the store
# get all items, returns an iterator
all_items = store.all()
# yield all items that match a condition
filtered_items = store.filter(lambda x: x.age > 30)
# get the first item that matches a condition
first_item = store.first(lambda x: x.age > 30)
Check if some object or it's key is in the store
if s2 in store:
print("s2 is in store")
else:
print("s2 is not in store")
Deleting items
Any of the following will delete the item from the store, along with the file that contains the data
store.delete(s1)
del store[s2]
key = "some_key"
store.delete(key)
How it works
The slowtore instance behaves like a dictionary of objects you added to it. Instead of storing the object it creates a proxy object that stores the object's state and the changes you made to it.
When you commit the changes, the proxy object is updated with the new state, and the changes are saved to disk.
When you undo/redo the changes, the proxy object is updated with the previous state, and the changes are saved to disk.
Check the ModelProxy class for more details.
License
This project is licensed under the MIT License - see the LICENSE file for details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for slowstore-0.1.10-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c094d596ba2f4773000bcb5e2df337f23f62c14c40e362421acdc0df69259073 |
|
MD5 | b07b9c65d6f616a6963192774e7908d3 |
|
BLAKE2b-256 | 91686b496301de971cc5276a5ed90b49e6fe1dafa2329cf1649077c356d605f3 |