Skip to main content

Fast Python JSON library

Project description

orjson

orjson is a fast JSON library for Python. It benchmarks as the fastest Python library for JSON. Its serialization performance is 2x to 3x the nearest other library and 4.5x to 11.5x the standard library. Its deserialization performance is 1.05x to 1.2x the nearest other library and 1.2x to 4x the standard library.

It supports CPython 3.5, 3.6, and 3.7. Its API is a subset of the API of the standard library's json module.

Usage

Install

To install a manylinux wheel from PyPI:

pip install --upgrade orjson

To build a release wheel from source, assuming a Rust nightly toolchain and Python environment:

git clone --recurse-submodules https://github.com/ijl/orjson.git && cd orjson
pip install --upgrade pyo3-pack
pyo3-pack build --release --strip --interpreter python3.7

There is no runtime dependency other than a manylinux environment (i.e., deploying this does not require Rust or non-libc type libraries.)

Serialize

def dumps(obj: Union[str, bytes, dict, list, tuple, int, float, None]) -> bytes: ...

dumps() serializes Python objects to JSON.

It has no options, does not support hooks for custom objects, and does not support subclasses.

It raises TypeError on an unsupported type. This exception message describes the invalid object.

It raises TypeError on an integer that exceeds 64 bits. This is the same as the standard library's json module.

It raises TypeError if a dict has a key of a type other than str.

import orjson

try:
    val = orjson.dumps(...)
except TypeError:
    raise

Deserialize

def loads(obj: Union[bytes, str]) -> Union[dict, list, int, float, str]: ...

loads() deserializes JSON to Python objects.

It raises orjson.JSONDecodeError if given an invalid type or invalid JSON. This exception is a subclass of ValueError.

import orjson

try:
    val = orjson.loads(...)
except orjson.JSONDecodeError:
    raise

Comparison

There are slight differences in output between libraries. The differences are not an issue for interoperability. Note orjson returns bytes. Its output is slightly smaller as well.

>>> import orjson, ujson, rapidjson, json
>>> data = {'bool': True, '🐈':'哈哈', 'int': 9223372036854775807, 'float': 1.337e+40}
>>> orjson.dumps(data)
b'{"bool":true,"\xf0\x9f\x90\x88":"\xe5\x93\x88\xe5\x93\x88","int":9223372036854775807,"float":1.337e40}'
>>> ujson.dumps(data)
'{"bool":true,"\\ud83d\\udc08":"\\u54c8\\u54c8","int":9223372036854775807,"float":1.337000000000000e+40}'
>>> rapidjson.dumps(data)
'{"bool":true,"\\uD83D\\uDC08":"\\u54C8\\u54C8","int":9223372036854775807,"float":1.337e+40}'
>>> json.dumps(data)
'{"bool": true, "\\ud83d\\udc08": "\\u54c8\\u54c8", "int": 9223372036854775807, "float": 1.337e+40}'

Testing

The library has comprehensive tests. There are unit tests against the roundtrip, jsonchecker, and fixtures files of the nativejson-benchmark repository. It is tested to not crash against the Big List of Naughty Strings. It is tested to not leak memory. It is tested to be correct against input from the PyJFuzz JSON fuzzer. There are integration tests exercising the library's use in web servers (uwsgi and gunicorn, using multiprocess/forked workers) and when multithreaded. It also uses some tests from the ultrajson library.

Performance

Serialization and deserialization performance of orjson is better than ultrajson, rapidjson, or json. The benchmarks are done on fixtures of real data:

  • twitter.json, 631.5KiB, results of a search on Twitter for "一", containing CJK strings, dictionaries of strings and arrays of dictionaries, indented.

  • github.json, 55.8KiB, a GitHub activity feed, containing dictionaries of strings and arrays of dictionaries, not indented.

  • citm_catalog.json, 1.7MiB, concert data, containing nested dictionaries of strings and arrays of integers, indented.

  • canada.json, 2.2MiB, coordinates of the Canadian border in GeoJSON format, containing floats and arrays, indented.

alt text alt text alt text alt text alt text alt text alt text alt text

twitter.json serialization

Library Median latency (milliseconds) Operations per second Relative (latency)
orjson 0.48 2077.6 1
ujson 1.48 664.6 3.09
rapidjson 1.59 626.5 3.32
json 2.24 443.9 4.68

twitter.json deserialization

Library Median latency (milliseconds) Operations per second Relative (latency)
orjson 2.38 418.8 1
ujson 2.67 373 1.12
rapidjson 2.78 359.5 1.16
json 2.77 359.7 1.16

github.json serialization

Library Median latency (milliseconds) Operations per second Relative (latency)
orjson 0.06 17745 1
ujson 0.14 7107.1 2.49
rapidjson 0.16 6253.9 2.86
json 0.25 3972.5 4.49

github.json deserialization

Library Median latency (milliseconds) Operations per second Relative (latency)
orjson 0.2 4929.7 1
ujson 0.22 4605.2 1.08
rapidjson 0.24 4166.5 1.19
json 0.24 4150.8 1.19

citm_catalog.json serialization

Library Median latency (milliseconds) Operations per second Relative (latency)
orjson 0.76 1302 1
ujson 2.58 387.2 3.38
rapidjson 2.37 421.1 3.11
json 5.41 184.4 7.09

citm_catalog.json deserialization

Library Median latency (milliseconds) Operations per second Relative (latency)
orjson 4.28 233.1 1
ujson 5.06 197.2 1.18
rapidjson 5.82 171.7 1.36
json 5.81 171.8 1.36

canada.json serialization

Library Median latency (milliseconds) Operations per second Relative (latency)
orjson 4.04 247.7 1
ujson 8.43 118.6 2.09
rapidjson 43.93 22.7 10.88
json 47.23 21.1 11.7

canada.json deserialization

Library Median latency (milliseconds) Operations per second Relative (latency)
orjson 6.69 147.6 1
ujson 7.17 139.4 1.07
rapidjson 26.77 37.4 4
json 26.59 37.6 3.97

This was measured using orjson 1.3.0 on Python 3.7.2 and Linux.

The results can be reproduced using the pybench and graph scripts.

License

orjson is dual licensed under the Apache 2.0 and MIT licenses. It contains tests from the hyperjson and ultrajson libraries. It is implemented using the serde_json and pyo3 libraries.

Project details


Release history Release notifications | RSS feed

This version

1.3.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

orjson-1.3.0-cp37-cp37m-manylinux1_x86_64.whl (128.7 kB view details)

Uploaded CPython 3.7m

orjson-1.3.0-cp36-cp36m-manylinux1_x86_64.whl (128.8 kB view details)

Uploaded CPython 3.6m

orjson-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (128.8 kB view details)

Uploaded CPython 3.5m

File details

Details for the file orjson-1.3.0-cp37-cp37m-manylinux1_x86_64.whl.

File metadata

  • Download URL: orjson-1.3.0-cp37-cp37m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 128.7 kB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.21.0 setuptools/40.5.0 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.7.1

File hashes

Hashes for orjson-1.3.0-cp37-cp37m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 f6316d8486b337e91350bb01d445ca31ce097ace71e4ba2b017d049e933d1c8e
MD5 71b83869b767d3b0e481cc878b7888d2
BLAKE2b-256 cbbf6b7af8a3df27a042aa98e7c81c98a2446c421f3a749747176eb787d5c68e

See more details on using hashes here.

File details

Details for the file orjson-1.3.0-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

  • Download URL: orjson-1.3.0-cp36-cp36m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 128.8 kB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.21.0 setuptools/40.5.0 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.7

File hashes

Hashes for orjson-1.3.0-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 2729ee9bc83caef19012c5c7fe79edba472a2e0671ada6b02f58da34e09f2f68
MD5 0ae1ad7203af0cd37fa953abf22d222a
BLAKE2b-256 29bde51fe8b3b595609d0b2bbb4f6df6fc741a5420045c147308c8c38f76a1dd

See more details on using hashes here.

File details

Details for the file orjson-1.3.0-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

  • Download URL: orjson-1.3.0-cp35-cp35m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 128.8 kB
  • Tags: CPython 3.5m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.21.0 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.5.6

File hashes

Hashes for orjson-1.3.0-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 fb9ab85eb71d0ce846e9d90b48ffd1c55c114f555c6397b2f67939888f96675d
MD5 454c03a61f680942f1f7f5f2d39b78eb
BLAKE2b-256 93f1cb6e35c8c9398b10e3030e05d1af60cb95dd8daa7e497042e0a4a437f14c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page