Fast Python JSON library
Project description
orjson
orjson is a fast JSON library for Python. It benchmarks as the fastest Python library for JSON. Its serialization performance is 2x to 3x the nearest other library and 4.5x to 11.5x the standard library. Its deserialization performance is 1.05x to 1.2x the nearest other library and 1.2x to 4x the standard library.
It supports CPython 3.5, 3.6, and 3.7. Its API is a subset of the API of the standard library's json module.
Usage
Install
To install a manylinux wheel from PyPI:
pip install --upgrade orjson
To build a release wheel from source, assuming a Rust nightly toolchain and Python environment:
git clone --recurse-submodules https://github.com/ijl/orjson.git && cd orjson
pip install --upgrade pyo3-pack
pyo3-pack build --release --strip --interpreter python3.7
There is no runtime dependency other than a manylinux environment (i.e., deploying this does not require Rust or non-libc type libraries.)
Serialize
def dumps(obj: Union[str, bytes, dict, list, tuple, int, float, None]) -> bytes: ...
dumps()
serializes Python objects to JSON.
It has no options, does not support hooks for custom objects, and does not support subclasses.
It raises TypeError
on an unsupported type. This exception message
describes the invalid object.
It raises TypeError
on an integer that exceeds 64 bits. This is the same
as the standard library's json
module.
It raises TypeError
if a dict
has a key of a type other than str
.
import orjson
try:
val = orjson.dumps(...)
except TypeError:
raise
Deserialize
def loads(obj: Union[bytes, str]) -> Union[dict, list, int, float, str]: ...
loads()
deserializes JSON to Python objects.
It raises orjson.JSONDecodeError
if given an invalid type or invalid
JSON. This exception is a subclass of ValueError
.
import orjson
try:
val = orjson.loads(...)
except orjson.JSONDecodeError:
raise
Comparison
There are slight differences in output between libraries. The differences are not an issue for interoperability. Note orjson returns bytes. Its output is slightly smaller as well.
>>> import orjson, ujson, rapidjson, json
>>> data = {'bool': True, '🐈':'哈哈', 'int': 9223372036854775807, 'float': 1.337e+40}
>>> orjson.dumps(data)
b'{"bool":true,"\xf0\x9f\x90\x88":"\xe5\x93\x88\xe5\x93\x88","int":9223372036854775807,"float":1.337e40}'
>>> ujson.dumps(data)
'{"bool":true,"\\ud83d\\udc08":"\\u54c8\\u54c8","int":9223372036854775807,"float":1.337000000000000e+40}'
>>> rapidjson.dumps(data)
'{"bool":true,"\\uD83D\\uDC08":"\\u54C8\\u54C8","int":9223372036854775807,"float":1.337e+40}'
>>> json.dumps(data)
'{"bool": true, "\\ud83d\\udc08": "\\u54c8\\u54c8", "int": 9223372036854775807, "float": 1.337e+40}'
Testing
The library has comprehensive tests. There are unit tests against the roundtrip, jsonchecker, and fixtures files of the nativejson-benchmark repository. It is tested to not crash against the Big List of Naughty Strings. It is tested to not leak memory. It is tested to be correct against input from the PyJFuzz JSON fuzzer. There are integration tests exercising the library's use in web servers (uwsgi and gunicorn, using multiprocess/forked workers) and when multithreaded. It also uses some tests from the ultrajson library.
Performance
Serialization and deserialization performance of orjson is better than ultrajson, rapidjson, or json. The benchmarks are done on fixtures of real data:
-
twitter.json, 631.5KiB, results of a search on Twitter for "一", containing CJK strings, dictionaries of strings and arrays of dictionaries, indented.
-
github.json, 55.8KiB, a GitHub activity feed, containing dictionaries of strings and arrays of dictionaries, not indented.
-
citm_catalog.json, 1.7MiB, concert data, containing nested dictionaries of strings and arrays of integers, indented.
-
canada.json, 2.2MiB, coordinates of the Canadian border in GeoJSON format, containing floats and arrays, indented.
twitter.json serialization
Library | Median latency (milliseconds) | Operations per second | Relative (latency) |
---|---|---|---|
orjson | 0.48 | 2077.6 | 1 |
ujson | 1.48 | 664.6 | 3.09 |
rapidjson | 1.59 | 626.5 | 3.32 |
json | 2.24 | 443.9 | 4.68 |
twitter.json deserialization
Library | Median latency (milliseconds) | Operations per second | Relative (latency) |
---|---|---|---|
orjson | 2.38 | 418.8 | 1 |
ujson | 2.67 | 373 | 1.12 |
rapidjson | 2.78 | 359.5 | 1.16 |
json | 2.77 | 359.7 | 1.16 |
github.json serialization
Library | Median latency (milliseconds) | Operations per second | Relative (latency) |
---|---|---|---|
orjson | 0.06 | 17745 | 1 |
ujson | 0.14 | 7107.1 | 2.49 |
rapidjson | 0.16 | 6253.9 | 2.86 |
json | 0.25 | 3972.5 | 4.49 |
github.json deserialization
Library | Median latency (milliseconds) | Operations per second | Relative (latency) |
---|---|---|---|
orjson | 0.2 | 4929.7 | 1 |
ujson | 0.22 | 4605.2 | 1.08 |
rapidjson | 0.24 | 4166.5 | 1.19 |
json | 0.24 | 4150.8 | 1.19 |
citm_catalog.json serialization
Library | Median latency (milliseconds) | Operations per second | Relative (latency) |
---|---|---|---|
orjson | 0.76 | 1302 | 1 |
ujson | 2.58 | 387.2 | 3.38 |
rapidjson | 2.37 | 421.1 | 3.11 |
json | 5.41 | 184.4 | 7.09 |
citm_catalog.json deserialization
Library | Median latency (milliseconds) | Operations per second | Relative (latency) |
---|---|---|---|
orjson | 4.28 | 233.1 | 1 |
ujson | 5.06 | 197.2 | 1.18 |
rapidjson | 5.82 | 171.7 | 1.36 |
json | 5.81 | 171.8 | 1.36 |
canada.json serialization
Library | Median latency (milliseconds) | Operations per second | Relative (latency) |
---|---|---|---|
orjson | 4.04 | 247.7 | 1 |
ujson | 8.43 | 118.6 | 2.09 |
rapidjson | 43.93 | 22.7 | 10.88 |
json | 47.23 | 21.1 | 11.7 |
canada.json deserialization
Library | Median latency (milliseconds) | Operations per second | Relative (latency) |
---|---|---|---|
orjson | 6.69 | 147.6 | 1 |
ujson | 7.17 | 139.4 | 1.07 |
rapidjson | 26.77 | 37.4 | 4 |
json | 26.59 | 37.6 | 3.97 |
This was measured using orjson 1.3.0 on Python 3.7.2 and Linux.
The results can be reproduced using the pybench
and graph
scripts.
License
orjson is dual licensed under the Apache 2.0 and MIT licenses. It contains tests from the hyperjson and ultrajson libraries. It is implemented using the serde_json and pyo3 libraries.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for orjson-1.3.0-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f6316d8486b337e91350bb01d445ca31ce097ace71e4ba2b017d049e933d1c8e |
|
MD5 | 71b83869b767d3b0e481cc878b7888d2 |
|
BLAKE2b-256 | cbbf6b7af8a3df27a042aa98e7c81c98a2446c421f3a749747176eb787d5c68e |
Hashes for orjson-1.3.0-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2729ee9bc83caef19012c5c7fe79edba472a2e0671ada6b02f58da34e09f2f68 |
|
MD5 | 0ae1ad7203af0cd37fa953abf22d222a |
|
BLAKE2b-256 | 29bde51fe8b3b595609d0b2bbb4f6df6fc741a5420045c147308c8c38f76a1dd |
Hashes for orjson-1.3.0-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fb9ab85eb71d0ce846e9d90b48ffd1c55c114f555c6397b2f67939888f96675d |
|
MD5 | 454c03a61f680942f1f7f5f2d39b78eb |
|
BLAKE2b-256 | 93f1cb6e35c8c9398b10e3030e05d1af60cb95dd8daa7e497042e0a4a437f14c |