Stream objects from json-arrays or json lines lazily..
Project description
json-streams
Read and write JSON lazy, especially json-arrays.
Handles both the JSON format:
[ { "a": 1 }, { "a": 2 } ]
As well as JSON LINES format:
{"a":1} {"a": 2}
Uses orjson
or ujson
if present, otherwise standard json
.
Usage
Installation
# Using standard json pip install json-streams # Using orjson pip install json-streams[orjson] # Using ujson pip install json-streams[ujson]
Note
This library prefers files opened in binary mode.
Therefore does all dumps
-methods return bytes
.
All loads
methods handles str
argument.
If you use the orjson
library you can also pass bytes
or bytesarray
to loads
.
The goal is to have all loads
handling str
, bytes
and bytesarray
.
Examples
Allows you to use json.load
and json.dump
with
both json and json-lines files as well as dumping generators.
import json_streams # This command tries to guess format and opens the file data = json_streams.load_from_file("data.json") # or data.jsonl # Write to file, again guessing format json_streams.dump_to_file(data, "data.jsonl")
from json_streams import json_iter, jsonl_iter # Open and read the file without guessing data = json_iter.load_from_file("data.json") # Process file # Write to file without guessing jsonl_iter.dump_to_file(data, "data.jsonl")
import json_streams def process(data): for entry in data: # process yield entry def read_process_and_write(filename_in, filename_out): json_streams.dump_to_file( process( json_streams.load_from_file(filename_in) ), filename_out )
Development
After cloning the repo, just run
$ make test
to setup a virtual environment, install dev dependencies and run the unit tests.
Note: If you run the command in a activated virtual environment, that environment is used instead.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Filename, size | File type | Python version | Upload date | Hashes |
---|---|---|---|---|
Filename, size json_streams-0.3.1-py3-none-any.whl (6.5 kB) | File type Wheel | Python version py3 | Upload date | Hashes View |
Filename, size json-streams-0.3.1.tar.gz (8.1 MB) | File type Source | Python version None | Upload date | Hashes View |
Hashes for json_streams-0.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3fc937f7679015be2c5b829fcb7e9f59e81bad511fe43f50cc5b2426617cfb64 |
|
MD5 | c3e9b35c7bd0131242d3fe1c36e682f3 |
|
BLAKE2-256 | 220b1b7d4a53abaedfeb633a3c2901453319f1e160d0814e8a9a2ed5399ee732 |