Introduction

ssrJSON is a Python JSON library that leverages modern hardware capabilities to achieve peak performance, implemented primarily in C. It offers a fully compatible interface to Python’s standard json module, making it a seamless drop-in replacement, while providing exceptional performance for JSON encoding and decoding.

If you prefer to skip the technical details below, please proceed directly to the How To Install section.

How Fast is ssrJSON?

TL;DR: ssrJSON is faster than or nearly as fast as orjson (which announces itself as the fastest Python library for JSON) on most benchmark cases.

Below is an artificial benchmark case to demonstrate the speed of encoding non-ASCII JSON (simple_object_zh.json). Upon seeing the diagram below, you might wonder: why do the performance results from other libraries appear so poor? If you are interested, please refer to the section UTF-8 Cache of str Objects.

Real-world case (twitter.json):

Real-world case II (github.json):

Floats (canada.json):

Numbers (mesh.json):

ssrjson.dumps() is about 4x-31x as fast as json.dumps() (Python3.14, x86-64, AVX2). ssrjson.loads() is about 2x-8x as fast as json.loads() for str input and is about 2x-8x as fast as json.loads() for bytes input (Python3.14, x86-64, AVX2). ssrJSON also provides ssrjson.dumps_to_bytes(), which encode Python objects directly to UTF-8 encoded bytes object using SIMD instructions.

Details of benchmarking can be found in the ssrjson-benchmark project. If you wish to run the benchmark tests yourself, you can execute the following commands:

pip install ssrjson-benchmark
python -m ssrjson_benchmark

This will generate a PDF report of the results. If you choose to, you may submit this report to the benchmark repository, allowing others to view the performance metrics of ssrJSON on your device.

SIMD Acceleration

ssrJSON is designed for modern hardware and extensively leverages SIMD instruction sets to accelerate encoding and decoding processes. This includes operations such as memory copying, integer type conversions, JSON encoding, and UTF-8 encoding. Currently, ssrJSON supports x86-64-v2 and above (requiring at least SSE4.2) as well as aarch64 devices. It does not support 32-bit systems or older x86-64 hardware with limited SIMD capabilities.

On the x86-64 platform, ssrJSON provides three distinct SIMD libraries optimized for SSE4.2, AVX2, and AVX512, respectively, automatically selecting the most appropriate library based on the device’s capabilities. For aarch64 architectures, it utilizes the NEON instruction set. Combined with Clang’s powerful vector extensions and compiler optimizations, ssrJSON can almost fully exploit CPU performance during encoding operations.

UTF-8 Cache of str Objects

The author has a detailed tech blog about this topic: Beware of Performance Pitfalls in Third-Party Python JSON Libraries.

Non-ASCII str objects may store a cached representation of their UTF-8 encoding (within the corresponding C structure PyUnicodeObject, represented as a const char * and a length with type Py_ssize_t) to minimize the overhead of subsequent UTF-8 encoding operations. When PyUnicode_AsUTF8AndSize (or other similar functions) is invoked, the CPython implementation utilizes it to store the C string along with its length. This mechanism ensures that the caller does not need to manage the lifetime of the returned C string. The str.encode("utf-8") operation does not write to the cache; however, if the cache is already present, it utilizes the cached data to create the bytes object.

Some third-party Python JSON libraries typically use certain CPython APIs to indirectly write UTF-8 results to the cache when performing dumps on non-ASCII str objects whose cache has not yet been populated. This makes benchmarks look better than real-world performance: the same object is repeatedly dumped during measurement, allowing the populated cache to be reused. In reality, UTF-8 encoding is CPU-intensive and is often the main performance bottleneck; writing the cache also increases memory usage.

ssrjson.dumps_to_bytes addresses this by leveraging SIMD instruction sets for UTF-8 encoding, achieving significantly better performance than conventional encoding algorithms implemented in CPython. Furthermore, ssrJSON lets users control whether to write this cache. Enabling it may make subsequent dumps_to_bytes calls on the same str object faster after the first call; the tradeoff is higher memory usage, as each visited non-ASCII str occupies additional memory corresponding to the length of its UTF-8 representation until the object is deallocated. Users should evaluate whether their projects repeatedly encode the same str objects before deciding whether to enable the cache.

Also, the ssrjson-benchmark project takes this aspect into account by differentiating test scenarios based on the presence or absence of this cache. The results demonstrate that ssrJSON maintains a substantial performance advantage over other third-party Python JSON libraries regardless of whether the cache exists.

Writing the cache is enabled globally by default. You can control this behavior globally with ssrjson.write_utf8_cache, or specify it per call to ssrjson.dumps_to_bytes with the is_write_cache argument.

ssrjson.dumps produces a str object and is unrelated to this section.

xjb64

ssrJSON employs xjb64 as float-to-string algorithm. Tests and comparisons reveals that the xjb64 algorithm significantly outperforms other algorithms in terms of performance and is more compatible. ssrJSON project adopts a slightly modified version to fit the standard behavior of Python's json module.

Random double on Apple M1:

Random double on AMD R7-7840H:

JSON Module compatibility

The design goal of ssrJSON is to provide a straightforward and highly compatible approach to replace the inherently slower Python standard JSON encoding and decoding implementation with a significantly more efficient and high-performance alternative. If your module exclusively utilizes dumps and loads, you can replace the current JSON implementation by importing ssrJSON as import ssrjson as json. To facilitate this, ssrJSON maintains compatibility with the argument formats of json.dumps and json.loads; however, it does not guarantee identical results to the standard JSON module, as many features are either intentionally omitted or not yet supported. For further information, please refer to the Behavior section.

Other Implementation Details

Overview of Encoding

The encoding performance of JSON libraries is not significantly limited by CPython, resulting in a very high potential maximum. As mentioned above, during string encoding, ssrJSON extensively utilizes SIMD instructions to accelerate copying and conversion operations. The implementation of dumps_to_bytes also tackles challenges related to UTF-8 encoding. ssrJSON includes a comprehensive UTF-8 encoding algorithm optimized for all supported SIMD features as well as Python’s internal string representation format (PyCompactUnicodeObject). When encoding integers, ssrJSON adapts the integer encoding approach from yyjson, a highly optimized C-language JSON parsing library.

Overview of Decoding

The main performance bottleneck in JSON decoding is the speed of creating Python objects. To address this, ssrJSON adopts the short-key caching mechanism from orjson, which greatly reduces the overhead of creating Python string objects. For string handling, when the input is of str type, ssrJSON applies SIMD optimizations similar to those used in encoding, speeding up the decoding process. For bytes inputs, string decoding is vectorized as well: a block of source bytes is validated and transcoded to the destination's UCS width with SIMD, and the destination widens in place (ASCII → UCS1 → UCS2 → UCS4) as wider code points appear. This part adapts simdutf's UTF-8 transcoding shapes and Lemire's block validation; blocks whose sequence lengths are too mixed for a vector shape fall back to yyjson's per-sequence decoding. Beyond string handling, ssrJSON extensively leverages yyjson’s codebase, including its numeric decoding algorithms and core decoding logic.

Limitations

Please note that ssrJSON is currently in its beta development stage, and some common features have yet to be implemented. We welcome your contributions to help build a highly performant Python JSON library.

To maintain stability, ssrJSON strives to minimize the addition of rarely used features for two reasons:

• ssrJSON aims to serve as a high-performance foundational library rather than one overloaded with elaborate features.
• Although C provides significant performance advantages, it also introduces considerable risks to stability. Software engineering experience shows that limiting rarely used features helps reduce critical vulnerabilities.

How To Install

Install from PyPI

Pre-built wheels are available on PyPI, you can install it using pip.

pip install ssrjson

Note: ssrJSON requires at least SSE4.2 on x86-64 (x86-64-v2), or aarch64. 32-bit platforms are not supported. ssrJSON does not work with Python implementations other than CPython. Currently supported CPython versions are 3.10, 3.11, 3.12, 3.13, 3.14, 3.15. For Python >= 3.15, you need to build it from source.

Build From Source

Since ssrJSON utilizes Clang's vector extensions, it requires compilation with Clang and cannot be compiled in GCC or pure MSVC environments. On Windows, clang-cl can be used for this purpose. Build can be easily done by the following commands (make sure CMake, Clang and Python are already installed)

# On Linux:
# export CC=clang
# export CXX=clang++
mkdir build
cmake -S . -B build  # On Windows, configure with `cmake -T ClangCL`
cmake --build build

Or you like the pip way:

mv pysrc ssrjson  # rename the python source directory to make it installable
pip install .

Usage

Basic

>>> import ssrjson
>>> ssrjson.dumps({"key": "value"})
'{"key":"value"}'
>>> ssrjson.loads('{"key":"value"}')
{'key': 'value'}
>>> ssrjson.dumps_to_bytes({"key": "value"})
b'{"key":"value"}'
>>> ssrjson.loads(b'{"key":"value"}')
{'key': 'value'}

NumPy Support

ssrJSON can directly serialize NumPy scalar types and ndarray objects without converting them to Python types first. To avoid introducing NumPy as a hard dependency, you must explicitly enable this by calling setup_numpy_types once:

>>> import numpy as np
>>> import ssrjson
>>> ssrjson.setup_numpy_types(np)

After setup, NumPy scalars and arrays are recognized in dumps and dumps_to_bytes:

>>> ssrjson.dumps(np.int64(42))
'42'
>>> ssrjson.dumps(np.array([1, 2, 3]))
'[1,2,3]'
>>> ssrjson.dumps({"data": np.array([[1, 2], [3, 4]]), "score": np.float32(0.95)})
'{"data":[[1,2],[3,4]],"score":0.95}'
>>> ssrjson.dumps_to_bytes(np.arange(5))
b'[0,1,2,3,4]'

Supported NumPy types:

np.float64 is a subclass of Python float and is always handled by the standard float path, regardless of whether setup_numpy_types has been called.

When encoding an ndarray, ssrJSON reads element data directly from the array's memory buffer. Combined with the xjb64/xjb32 floating-point encoding algorithms and yyjson-derived integer encoding algorithms, this gives ssrJSON a significant performance advantage.

Simple benchmark shows ssrJSON outperforms orjson in encoding numpy arrays:

$ python dev_tools/numpy_benchmark.py --scale 10
numpy 2.4.2  |  scale=10  number=20  repeat=7  warmup=2
Python 3.14.3

[dumps_to_bytes: ssrjson vs orjson — numpy ndarray]

  int32_1d[1000000]  shape=1000000  dtype=int32  3906.2 KiB raw
    ssrjson  : median 1.83 ms  best 1.78 ms  out=4391695B  2.24 GiB/s
    orjson   : median 4.96 ms  best 4.88 ms  out=4391695B  843.99 MiB/s
    ssrjson is 2.72x faster than orjson (median); best 2.74x

  int64_1d[1000000]  shape=1000000  dtype=int64  7812.5 KiB raw
    ssrjson  : median 4.30 ms  best 4.13 ms  out=10982023B  2.38 GiB/s
    orjson   : median 8.79 ms  best 8.66 ms  out=10982023B  1.16 GiB/s
    ssrjson is 2.05x faster than orjson (median); best 2.10x

  float32_1d[1000000]  shape=1000000  dtype=float32  3906.2 KiB raw
    ssrjson  : median 9.15 ms  best 9.04 ms  out=10627011B  1.08 GiB/s
    orjson   : median 17.25 ms  best 17.18 ms  out=10627112B  587.55 MiB/s
    ssrjson is 1.88x faster than orjson (median); best 1.90x

  float64_1d[1000000]  shape=1000000  dtype=float64  7812.5 KiB raw
    ssrjson  : median 13.27 ms  best 12.94 ms  out=19269164B  1.35 GiB/s
    orjson   : median 19.12 ms  best 18.99 ms  out=19269255B  961.00 MiB/s
    ssrjson is 1.44x faster than orjson (median); best 1.47x

  float64_2d[1000x1000]  shape=1000x1000  dtype=float64  7812.5 KiB raw
    ssrjson  : median 13.50 ms  best 13.14 ms  out=19272764B  1.33 GiB/s
    orjson   : median 19.09 ms  best 19.04 ms  out=19272837B  962.88 MiB/s
    ssrjson is 1.41x faster than orjson (median); best 1.45x

  float64_3d[100x100x100]  shape=100x100x100  dtype=float64  7812.5 KiB raw
    ssrjson  : median 13.46 ms  best 13.13 ms  out=19291108B  1.33 GiB/s
    orjson   : median 19.48 ms  best 19.42 ms  out=19291190B  944.22 MiB/s
    ssrjson is 1.45x faster than orjson (median); best 1.48x

  int32_2d[1000x1000]  shape=1000x1000  dtype=int32  3906.2 KiB raw
    ssrjson  : median 1.17 ms  best 1.17 ms  out=5391602B  4.31 GiB/s
    orjson   : median 5.04 ms  best 5.00 ms  out=5391602B  1020.42 MiB/s
    ssrjson is 4.32x faster than orjson (median); best 4.29x

  bool_1d[1000000]  shape=1000000  dtype=bool  976.6 KiB raw
    ssrjson  : median 323.7 µs  best 323.1 µs  out=5500290B  15.83 GiB/s
    orjson   : median 3.24 ms  best 3.21 ms  out=5500290B  1.58 GiB/s
    ssrjson is 10.02x faster than orjson (median); best 9.92x

Indent

ssrJSON only supports encoding with indent = 2, 4 or no indent (don't pass indent, or pass indent=None). When indent is used, a space is inserted between each key and value.

>>> import ssrjson
>>> ssrjson.dumps({"a": "b", "c": {"d": True}, "e": [1, 2]})
'{"a":"b","c":{"d":true},"e":[1,2]}'
>>> print(ssrjson.dumps({"a": "b", "c": {"d": True}, "e": [1, 2]}, indent=2))
{
  "a": "b",
  "c": {
    "d": true
  },
  "e": [
    1,
    2
  ]
}
>>> print(ssrjson.dumps({"a": "b", "c": {"d": True}, "e": [1, 2]}, indent=4))
{
    "a": "b",
    "c": {
        "d": true
    },
    "e": [
        1,
        2
    ]
}
>>> ssrjson.dumps({"a": "b", "c": {"d": True}, "e": [1, 2]}, indent=3)
Traceback (most recent call last):
  File "<python-input>", line 1, in <module>
    ssrjson.dumps({"a": "b", "c": {"d": True}, "e": [1, 2]}, indent=3)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: integer indent must be 2 or 4

Other Arguments Supported by Python's json

object_hook can be used in loads, and works the same as json.loads.

Arguments like ensure_ascii, parse_float provided by json module can be recognized but ignored by design. To treat passing these arguments as an error, call ssrjson.strict_argparse(True) once and it will take effect globally.

Inspect Features

Call get_current_features to get ssrJSON's current build configuration and settings.

>>> ssrjson.get_current_features()
{'multi_lib': True, 'write_utf8_cache': True, 'strict_arg_parse': False, 'free_threading': False, 'lockfree': False, 'simd': 'AVX2'}

Behavior

Generally, ssrjson.dumps behaves like json.dumps with ensure_ascii=False, and ssrjson.loads behaves like json.loads. Below we explain some behavior details of ssrJSON, which might differ from the standard json module or other third-party JSON libraries.

Strings

Code points within the range [0xd800, 0xdfff] cannot be represented in UTF-8 encoding, and the standard JSON specification typically prohibits the presence of such characters. However, since Python's str type is not stored as UTF-8, ssrjson.dumps allows these characters to remain compatible with the Python json module, while other third-party Python JSON libraries may reject them. In contrast, ssrjson.dumps_to_bytes produces UTF-8 output and treats these characters in the input as invalid.

>>> s = chr(0xd800)
>>> (json.dumps(s, ensure_ascii=False) == '"' + s + '"', json.dumps(s, ensure_ascii=False))
(True, '"\ud800"')
>>> (ssrjson.dumps(s) == '"' + s + '"', ssrjson.dumps(s))
(True, '"\ud800"')
>>> ssrjson.dumps_to_bytes(s)
Traceback (most recent call last):
  File "<python-input>", line 1, in <module>
    ssrjson.dumps_to_bytes(s)
    ~~~~~~~~~~~~~~~~~~~~~~^^^
ssrjson.JSONEncodeError: Cannot encode unicode character in range [0xd800, 0xdfff] to UTF-8
>>> json.loads(json.dumps(s, ensure_ascii=False)) == s
True
>>> ssrjson.loads(ssrjson.dumps(s)) == s
True

Integers

ssrjson.dumps can only handle integers that can be expressed by either uint64_t or int64_t in C.

>>> ssrjson.dumps(-(1<<63)-1)
Traceback (most recent call last):
  File "<python-input>", line 1, in <module>
    ssrjson.dumps(-(1<<63)-1)
    ~~~~~~~~~~~~~^^^^^^^^^^^^
ssrjson.JSONEncodeError: convert value to long long failed
>>> ssrjson.dumps(-(1<<63))
'-9223372036854775808'
>>> ssrjson.dumps((1<<64)-1)
'18446744073709551615'
>>> ssrjson.dumps(1<<64)
Traceback (most recent call last):
  File "<python-input>", line 1, in <module>
    ssrjson.dumps(1<<64)
    ~~~~~~~~~~~~~^^^^^^^
ssrjson.JSONEncodeError: convert value to unsigned long long failed

ssrjson.loads parses integers outside the ranges of int64_t and uint64_t as float objects.

>>> ssrjson.loads('-9223372036854775809')  # -(1<<63)-1
-9.223372036854776e+18
>>> ssrjson.loads('-9223372036854775808')  # -(1<<63)
-9223372036854775808
>>> ssrjson.loads('18446744073709551615')  # (1<<64)-1
18446744073709551615
>>> ssrjson.loads('18446744073709551616')  # 1<<64
1.8446744073709552e+19

Floats

For floating-point encoding, ssrJSON employs the xjb64 algorithm. xjb64 is a highly efficient algorithm for converting floating-point to strings.

Encoding and decoding math.inf are supported. ssrjson.dumps outputs the same result as json.dumps. The input of ssrjson.loads should be "infinity" with lower or upper cases (for each character), and cannot be "inf".

>>> json.dumps(math.inf)
'Infinity'
>>> ssrjson.dumps(math.inf)
'Infinity'
>>> json.dumps(-math.inf)
'-Infinity'
>>> ssrjson.dumps(-math.inf)
'-Infinity'
>>> ssrjson.loads("[infinity, Infinity, InFiNiTy, INFINITY]")  # allowed but not recommended to write `InFiNiTy` in JSON
[inf, inf, inf, inf]

The case of math.nan is similar, but NaN has no sign: both positive and negative NaN are encoded as NaN.

>>> json.dumps(math.nan)
'NaN'
>>> ssrjson.dumps(math.nan)
'NaN'
>>> json.dumps(-math.nan)
'NaN'
>>> ssrjson.dumps(-math.nan)
'NaN'
>>> ssrjson.loads("[nan, Nan, NaN, NAN]")  # allowed but not recommended to write `Nan` in JSON
[nan, nan, nan, nan]

Free Threading

ssrJSON experimentally supports free-threading (Python >= 3.14). You can find stable wheel releases on PyPI. When building from source, enable this feature by specifying -DBUILD_FREE_THREADING=ON. In that build, during encoding ssrJSON acquires locks on dict and list objects from outer to inner; if another thread attempts to lock those objects in a different order, a deadlock may occur — this is expected behavior. If you encounter unexpected crashes, please file an issue. Decoding is lock-free.

If you require a lock-free encoding variant, build from source with -DFREE_THREADING_LOCKFREE=ON. Compared with the lock-based version, the lock-free version achieves approximately a 13% improvement in single-threaded encoding performance. In that configuration, multi-threaded modifications of the same dict/list can cause the program to crash; users are responsible for ensuring there are no race conditions. Lock-free builds are not distributed on PyPI.

License

This project is licensed under the MIT License. Licenses of other repositories are under licenses directory.

Acknowledgments

We would like to express our gratitude to the outstanding libraries and their authors:

• CPython
• yyjson: ssrJSON draws extensively from yyjson’s highly optimized implementations, including the core decoding logic, the decoding of bytes objects, the integer encoding and number decoding routines.
• orjson: ssrJSON references parts of orjson’s SIMD-based ASCII string encoding and decoding algorithms, as well as its short-key caching mechanism. Additionally, ssrJSON used orjson's pytest code as its initial test code.
• xjb64: ssrJSON employs the xjb32/64 algorithms for high-performance floating-point encoding.
• xxHash: ssrJSON leverages xxHash to efficiently compute hash values for key caching.
• klib: ssrJSON uses khash to implement circular detection in free-threading build.
• simdutf: the vectorized UTF-8 decoder used for bytes input adapts simdutf's UTF-8 to UTF-16 transcoding shapes and its lookup tables, together with Lemire's utf8_lookup4 block validation algorithm.