Ultra fast JSON encoder and decoder for Python with NumPy support
Project description
nujson: The UltraJSON Fork That Support Numpy Serialization
Inspired by Pandas' ujson
How to install
Python version: Python2.7, Python3.5+
Using pip:
pip install nujson
Clone and install:
# git clone
git clone https://github.com/caiyunapp/ultrajson
# Don't use `python setup.py install`
# NumPy will install automatically
pip install -e .
If get error like this:
ERROR: Could not find a version that satisfies the requirement numpy>=1.16.4 (from nujson) (from versions: 1.9.3)
ERROR: No matching distribution found for numpy>=1.16.4 (from nujson)
Try this:
pip uninstall numpy
pip install numpy==1.16.4
pip install nujson
Example
>>> import numpy as np
>>> import nujson as ujson
>>> a = {"a": np.int64(100)}
>>> ujson.dumps(a)
'{"a":100}'
>>> a["b"] = np.float64(10.9)
>>> ujson.dumps(a)
'{"a":100,"b":10.9}'
>>> a["c"] = np.str_("12")
>>> ujson.dumps(a)
'{"a":100,"b":10.9,"c":"12"}'
>>> a["d"] = np.array(list(range(10)))
>>> ujson.dumps(a)
'{"a":100,"b":10.9,"c":"12","d":[0,1,2,3,4,5,6,7,8,9]}'
>>> a["e"] = np.repeat(3.9, 4)
>>> ujson.dumps(a)
'{"a":100,"b":10.9,"c":"12","d":[0,1,2,3,4,5,6,7,8,9],"e":[3.9,3.9,3.9,3.9]}'
Why make such a package and what is modified
On Python3, some data types of NumPy is not serializable. Here is some references we searched:
- python - Why are some numpy datatypes JSON serializable and others not? - Stack Overflow
- Maximum recursion level reached in Python 3 - Issue #221 - esnme/ultrajson
- Issue 24313: json fails to serialise numpy.int64 - Python tracker
One solution is type conversion like: int(numpy.int64)
and numpy.array.tolist()
.
But it's not good for performance. After searching Internet, find a no longer maintained project Komnomnomnom/ultrajson recommond to use Pandas' ujson.
We tried but found Pandas is to heavy for our projects. So we decide to build our own light weight fork. Currentltly, the esn's ujson master branch has some problems we need to solve, and the master
branch is based on the the v1.35 ujson.
The main point is convert NumPy data type in C, with calling NumPy's header. Commit 187bd15 has the most changes we made to support NumPy, and Commit afedc42 fix a build issue on macOS caused by Clang.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.