file-json
In many scenarios, you may want to save a huge list, dict object and use it later.
The standard json package will load all the data into memroy.
Hence comes the file-json library. With this package, you can store the list, dict object to local file system.
How it works
all the rules should
- Each object was store in a directory, the data was store in
*.json.gz - the
.file_json.jsonfile contains some basic info of this object. - for time base list, the directory looks like this.
.file_json.json
{
"type": "time_based_list",
"key": "create_datetime",
}
2023
- 12
- 19
- 00.json.gz # this file contains data from 2023-12-19 00:00:00 to 2023-12-19 01:00:00(exclude 2023-12-19 01:00:00)
- 01.json.gz
- 02 # if there were too much data between 2023-12-19 02:00:00 and 2023-12-10 03:00:00
- 00.json.gz
- 01.json.gz
- 20
2024
- 01
- 01
- for huge size of list, the directory looks like this.
.file_json.json
{
"type": "sorted_list",
"key": "id",
}
- 00.json.gz
- 01.json.gz
- 02
- 01.json.gz
- 02.json.gz
- 03.json.gz
- for huge size of dict, the directory looks like this.
.file_json.json
{
"type": "dict",
}
- key1.json.gz
- key2
- key2_1.json.gz
- key2_2.json.gz
- key3.json.gz
Features
- support time base list
- support datetime with timezone
- support huge size of list
- support huge size of dict
- use filelock to handle concurrency
Table of Contents
Installation
pip install file-json
License
file-json is distributed under the terms of the MIT license.
Metadata
Release files for file-json 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| file_json-0.1.0.tar.gz | 7.0 kB | Without provenance | Details for file_json-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| file_json-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for file_json-0.1.0-py3-none-any.whl |
Total release size: 13.6 kB
Release files / file_json-0.1.0.tar.gz
| Download URL | file_json-0.1.0.tar.gz |
|---|---|
| Size | 7.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3e4c85e541791118921c9ceb1feabc0cf6202cabe9e9f37f688ff454c60f7435
|
|
BLAKE2b-256 checksum How to use checksums |
b3fe42173bdd9917bb5ddea9ea611dc49b4ac147ebfe64dc62b400fed5f234a6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
python-httpx/0.25.0
|
Release files / file_json-0.1.0-py3-none-any.whl
| Download URL | file_json-0.1.0-py3-none-any.whl |
|---|---|
| Size | 6.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8f47aa379a0cc8088c746c910895580b59d3dc4c2be075ce85a3f5417637be93
|
|
BLAKE2b-256 checksum How to use checksums |
60e7e534ae7fd4ad8bd82628da3ab42c3df3458689effd13786b4bcb03b69eff
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
python-httpx/0.25.0
|
Security for file-json 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.