A simple and async json database.
Project description
simple-json-db
This is a simple json database. The package provides a simple ORM between python objects and json objects with a well type-hinted schema.
This package maps your python objects to json and then you can save, get, modify or delete them using async methods.
This package is for tiny and simple projects. with a low amount of data.
Installation
The package is available at PYPI as json-entity.
Intro
Let's see how you can get started with the package.
See also our Wiki.
You can take a look at src/examples, if you're not on reading mode.
Create Model
Step #1 is to create a model that represents something like a row in a table.
from sjd import TEntity, properties as props
class Student(TEntity):
student_id = props.IntProperty(required=True)
first_name = props.StrProperty(required=True)
last_name = props.StrProperty(required=True)
- Your model should inherit from
TEntity
. - Properties can be anything that JSON supports (
int
,str
for now).
Initialize db
We have an Engine
and some Collection
s. The first one is your database, the second is your table.
from sjd import Engine
class AppEngine(Engine):
students = Engine.set(Student)
def __init__(self):
super().__init__(Path("__test_db__"))
- The engine will create a directory named
__test_db__
. - inside
__test_db__
the collections are stored. - You SHOULD pass the type of your entity (
entity_type
) to the__Collection__
. Here isStudent
. Engine.set()
returns type__Collection__
which is a descriptor! The actual thing isCollection
.- The collection name will be class variable's name (
students
here ).
Create engine instance
After setting up engine, you can create an instance of it. And you have access to the collections.
engine = AppEngine()
collection = engine.students
Construct first data
Let's create our first data.
student = Student()
student.student_id = 123456789
student.first_name = "Arash"
student.last_name = "Enzo"
Add data
Save data.
await collection.add(student)
Good job, you've saved your first data.
Better model
Personally, i prefer classes with initializers. so let's add an __init__
to the Student
.
# ---- sniff ----
class Student(TEntity):
__json_init__ = True
student_id = props.IntProperty(required=True)
first_name = props.StrProperty(required=True)
last_name = props.StrProperty()
def __init__(self, student_id: int, first_name: str, last_name: str):
self.student_id = student_id
self.first_name = first_name
self.last_name = last_name
- For models with
__init__
that accepts some parameters, you SHOULD include__json_init__ = True
or it will fail while deserializing the data.
Even Quicker
Package provides a way to resolve model's properties from `__init__` function, without specifing them directly.
This can be used for simple models only.
```py
from src.sjd import TEntity, properties as props
@props.collect_props_from_init
class Student(TEntity):
def __init__(self, student_id: int, first_name: str, last_name: str):
self.student_id = student_id
self.first_name = first_name
self.last_name = last_name
```
Add more data
Now we can create more students quickly, so we need to add them quickly as well.
await collection.add_many(
Student(1, "John", "Doe"),
Student(2, "Jane", "Doe"),
Student(3, "Jack", "Doe"),
Student(4, "Jill", "Doe"),
)
Query data
Let's do some query stuff to get data we want.
Iterate over all.
async for student in collection:
print(student.first_name)
# John
# Jane
# Jack
# Jill
Get data with some filters.
async for student in engine.students.iterate_by(
lambda s: s.last_name, "Doe"
):
print(student.first_name)
# John
# Jane
# Jack
# Jill
Ops they were all "Doe".
async for student in engine.students.iterate_by(
lambda s: s.first_name, "John"
):
print(student.first_name)
# John
If it's going to be one, we have more options.
student = await collection.get_first(lambda s: s.first_name == "John")
print(student.first_name)
# John
And some more ...
Type hint are fully available.
Update data
I guess the name was Johnny not John 🤔, let's change the name of student John to Johnny.
async for student in collection:
if student.first_name == "John":
student.first_name = "Johnny"
collection.update(student)
Delete data
Now that i looked closer, we don't have any john or johnny at all, imma remove johnny then.
async for student in collection:
if student.first_name == "Johnny":
collection.delete(student)
Complex Properties
You can use more complex models! models that include Other models as property or a list of other models or builtin types.
Let's begin with creating another model called Grade
that includes some information about an student's grade in a course. We are going to add this to Student
later.
Since this model is an embed entity, You should inherit from EmbedEntity
.
from sjd import TEntity, EmbedEntity, properties as props
# ---- sniff ----
class Grade(EmbedEntity):
__json_init__ = True
course_id = props.IntProperty(required=True)
course_name = props.StrProperty(required=True)
score = props.IntProperty(required=True)
def __init__(self, course_id: int, course_name: str, score: int):
self.course_id = course_id
self.course_name = course_name
self.score = score
To add this as a new property to the Student
, we'll use ComplexProperty
. ( Or OptionalComplexProperty
for a complex property which is not required ).
Your Student
class should looks like this:
# ---- sniff ----
class Student(TEntity):
__json_init__ = True
student_id = props.IntProperty(required=True)
first_name = props.StrProperty(required=True)
last_name = props.StrProperty()
grade = props.OptionalComplexProperty(Grade)
def __init__(
self,
student_id: int,
first_name: str,
last_name: str,
grade: Optional[Grade] = None,
):
self.student_id = student_id
self.first_name = first_name
self.last_name = last_name
self.grade = grade
- Note that we passed
Grade
type as the first parameter to theOptionalComplexProperty
.
Now we can add this grade for all students
async for student in collection:
student.grade = Grade(1, "Math", 90)
await collection.update(student)
Let's check if it's working
jill = await collection.get_first(lambda s: s.first_name == "Jill")
if jill.grade:
print(jill.grade.course_name, jill.grade.score)
# Math
List Properties
As we all know, there may be more than one course per student! Then the grade property, can be grades.
# ---- sniff ----
class Student(TEntity):
__json_init__ = True
student_id = props.IntProperty(required=True)
first_name = props.StrProperty(required=True)
last_name = props.StrProperty()
grades = props.ListProperty(Grade, default_factory=list)
def __init__(
self,
student_id: int,
first_name: str,
last_name: str,
grades: list[Grade] = [],
):
self.student_id = student_id
self.first_name = first_name
self.last_name = last_name
self.grades = grades
Let's update all of entities again.
async for student in collection:
student.grades = [Grade(1, "Math", 90), Grade(2, "English", 80)]
await collection.update(student)
# Math 90
# English 80
Let's change Jill's english score to 50.
jill = await collection.get_first(lambda s: s.first_name == "Jill")
if jill.grades:
jill.grades[-1].score = 50
await collection.update(jill)
Oh who's score was 50 ??!
async with collection.iterate() as students:
with_50_score = await students.where(
lambda s: any(x.score == 50 for x in s.grades)).single()
print(with_50_score.first_name)
Getting data ( Fast way )
There are two methods that are probably faster than as_queryable
way.
-
get(__id: str)
Get an entity by it's id, Which is (
entity.id
). We use this forupdate
anddelete
method. -
iterate_by(__prop: str, __value: Any)
Use this to do an
async iterate
over all entities withentity[__prop] == __value
. We use this to work with virtual objects. -
get_first(__prop: str, __value: Any)
Just like 2, but returns the first.
async for student in engine.students.iterate_by("first_name", "Jill"):
print(student.last_name)
or
async for student in engine.students.iter_by_prop_value(lambda s: s.first_name, "Jill"):
print(student.last_name)
Virtual properties
It was a good idea to use another model inside our main model to store additional data, but we don't actually want to load all of data while getting our main model. ( You don't want to load students's grade every time, since it's costly. )
It's better use a separate entity for grade and make it related to the student. Here, the grade will be a virtual complex property
.
And the grade will use a reference property
to the student id.
Since the Grade
is going to be a separate entity, we should add it to our AppEngine
.
-
First, let's modify the
Student
class.We will replace
ListProperty
withVirtualListProperty
.# ---- sniff ---- class Student(TEntity): __json_init__ = True student_id = props.IntProperty(required=True) first_name = props.StrProperty(required=True) last_name = props.StrProperty() grades = props.VirtualListProperty(Grade, "student_id") def __init__( self, student_id: int, first_name: str, last_name: str, grades: list[Grade] = [], ): self.student_id = student_id self.first_name = first_name self.last_name = last_name self.grades = grades
VirtualListProperty
takes type of entity it refers to and the name ofReferenceProperty
which we'll declare later insideGrade
class (student_id
). -
Modifying
Grade
.The class should inherit from
TEntity
instead ofEmbedEntity
, since it's a separate entity now.# ---- sniff ---- class Grade(TEntity): __json_init__ = True course_id = props.IntProperty(required=True) course_name = props.StrProperty(required=True) score = props.IntProperty(required=True) student_id = props.ReferenceProperty() def __init__(self, course_id: int, course_name: str, score: int): self.course_id = course_id self.course_name = course_name self.score = score
Note that we added
student_id = ReferenceProperty()
. The attribute name should be the same we declared insideStudent
'sVirtualListProperty
's second parameter (student_id
). -
Finally, modifying
AppEngine
We only need to add
Grade
to theAppEngine
, just likeStudent
.class AppEngine(Engine): students = Engine.set(Student) grades = Engine.set(Grade) def __init__(self): super().__init__("__test_db__")
-
Add data. You can now add your data just like before.
engine = AppEngine() await engine.students.add( Student(1, "Arash", "Doe", [Grade(1, "Physics", 20)]), )
-
Getting data
If you try getting one of your students now, you'll see the
grades
property is an empty list.arash = await engine.students.get_first( lambda s: s.first_name == "Arash" ) print(arash.grades) # []
The
grades
is some kind oflazy property
.to load virtual data, you can use method
load_virtual_props
await engine.students.load_virtual_props(arash) print(arash.grades) # [<__main__.Grade object at 0x0000021B3AF7FAC0>]
or you can specify property's name ( if you have more than one ).
await engine.students.load_virtual_props(arash, "grades") print(arash.grades)
Here you go, you have your grades.
Or even better, you can iter over grades WITHOUT calling
load_virtual_props
( less costly again ).You use
iter_referenced_by
:async for grade in engine.students.iter_referenced_by(arash, lambda s: s.grades): print(grade.course_name)
Working examples are available under src/examples.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for json_entity-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 417977f9307d2fe9237c02ebc14455d9c4ec4f3550b631fa50b64e169a67fad7 |
|
MD5 | c864de66e6bbc290123772fdd88b3971 |
|
BLAKE2b-256 | bb73499eff60ba2330d85b492b60358182347b0fa11679f375a3b7da6f8acfb8 |