Skip to main content

Store big JSON-style files, but access the data quickly.

Project description

JSOP - JSON-Style Object Persistence

JSOP is a persistence engine that allows an application to store a large amount of JSON-style data on disk, but to access it in an efficient way.

It is based on the dbm module, but offers a much easier-to-use API.

JSOP is also designed to enable easy migration of data in existing applications, that already store data in JSON files, with minimal changes to the code.

Installation

pip3 install jsop

Quickstart Guide

Creating a new JSOP file

Programmatically :

# 'data' is any JSON-serializable object.

import jsop

jsop.JSOP("/path/to/jsop").init(data)

Or from the command line:

python3 -m jsop init /path/to/jsop /path/to/data.json

(If an initial JSON file is not given, the file will be initialized with an empty map.)

Read and Write

with jsop.JSOP("/path/to/jsop") as data:
    name = data["name"]
    data["age"] = 30
    for friend in data["friends"]:
        print(friend["name"])

Supported Operations

Assignments

You can store any JSON-serializable data with JSOP using simple assignment. For example:

path = "/path/to/jsop"

jsop.JSOP(path).init()      # initalize with an empty map.

with jsop.JSOP(path) as data:
    data["string"] = "Hello, World!"
    data["boolean"] = True
    data["map"] = {"a": 1, "b": 2, "c": 3}
    data["map"]["d"] = 4
    data["map"]["list"] = [5,6,7]

The file will be saved once the with block exits.

Accessing Data

When you retrieve data of primitive types, you just get the corresponding python type:

with jsop.JSOP(path) as data:
    my_string = data["string"]
    # type(my_string) is str

    my_int = data["map"]["c"]
    # type(my_int) is int

However, when you retrieve a map or a list, you get special objects, named JDict and JList, respectively.

Map Operations

With JDict, you can do most of the things you can do with a python dict. For example:

with jsop.JSOP(path) as data:
    my_map = data["map"]
    # type(my_map) is JDict
    
    a = my_map["a"]                  # item access
    my_map["b"] = 3                  # item assignment
    del my_map["c"]                  # item removal
    if "d" in my_map:
        pass                         # using the "in" operator
    length = len(my_map)             # getting map's size
    keys = my_map.keys()             # getting list of keys
    for key in my_map:
        pass                         # iteration over keys
    if my_map == my_map:
        pass                         # comparison with a JDict
    if my_map == {"a": 1, "b": 3}:
        pass                         # comparison with a Python dict
    my_map.clear()                   # removing all keys from a map

Also, you can convert the map to a regular python dict, using the export() method:

with jsop.JSOP(path) as data:
    my_map = data["map"].export()
    # type(my_map) is dict

    my_map["e"] = 5

    data["map"] = my_map

Note that like a JSON map, the keys in a JSOP map are always strings. If a different object is given as a key, it is converted to a string.

List Operations

Likewise, The JList object supports virtually all operations supported by a python list. For Example:

with jsop.JSOP(path) as data:
    my_list = data["map"]["list"]
    # type(my_list) is JList

    for item in my_list:
        pass                         # iteration over items
    my_list.append(8)                # adding an item
    eight = my_list.pop()            # removing (and returning) the last item
    six = my_list[1]    	         # item access by index
    my_list[1] = 9                   # item assignment
    my_list.remove(9)                # removing an arbitrary item
    if 8 in my_list:
        pass                         # using the "in" operator
    length = len(my_list)            # getting list's size
    if my_list == my_list:
        pass                         # comparison with a JList
    if my_list == [5,6,7]:
        pass                         # comparison with a Python list
    my_list.clear()                  # removing all items from list

Like as in JDict, JList also supports the export() method, which returns a python list:

with jsop.JSOP(path) as data:
    my_list = data["map"]["list"].export()
    # type(my_list) is list

Handling References

One should be a bit careful when keeping references to JDict and JList objects.

Unlike Python's regular dict and list objects, these are not references to objects in memory, but to certain "paths" in the JSOP root object. For example:

with jsop.JSOP(path) as data:
    data["list_of_lists"] = [[1,2,3], [4,5,6]]

    item = data["list_of_lists"][0]
    # Since `item` is a JList, it is a reference to the first item of
    # the list under the key "list_of_lists" in `data`.

    del data["list_of_lists"]
    # Now the list_of_lists does not exit, and `item` became an 
    # invalid reference.

    print(item[2])
    # This will raise an exception.

Copy and Backup

In order to create copy a JSOP file, it is recommended to export its content to JSON. The reason is that JSON files take less space, and also because of portability: this practice avoids problems resulting from the use of different dbm implementations on different systems.

This can be done from the command line:

python3 -m jsop export /path/to/jsop /path/to/copy.json

If JSON file path is not given, the result will be printed to the standard output.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for jsop, version 1.1.0
Filename, size File type Python version Upload date Hashes
Filename, size jsop-1.1.0.tar.gz (8.1 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page