Iterative JSON parser with a standard Python iterator interface
Project description
ijson
Ijson is an iterative JSON parser with a standard Python iterator interface.
Usage
All usage example will be using a JSON document describing geographical objects:
{ "earth": { "europe": [ {"name": "Paris", "type": "city", "info": { ... }}, {"name": "Thames", "type": "river", "info": { ... }}, // ... ], "america": [ {"name": "Texas", "type": "state", "info": { ... }}, // ... ] } }
Most common usage is having ijson yield native Python objects out of a JSON stream located under a prefix. Here’s how to process all European cities:
import ijson f = urlopen('http://.../') objects = ijson.items(f, 'earth.europe.item') cities = (o for o in objects if o['type'] == 'city') for city in cities: do_something_with(city)
Sometimes when dealing with a particularly large JSON payload it may worth to not even construct individual Python objects and react on individual events immediately producing some result:
import ijson parser = ijson.parse(urlopen('http://.../')) stream.write('<geo>') for prefix, event, value in parser: if (prefix, event) == ('earth', 'map_key'): stream.write('<%s>' % value) continent = value elif prefix.endswith('.name'): stream.write('<object name="%s"/>' % value) elif (prefix, event) == ('earth.%s' % continent, 'end_map'): stream.write('</%s>' % continent) stream.write('</geo>')
Backends
Ijson provides several implementations of the actual parsing in the form of backends located in ijson/backends:
yajl2_c: a C extension using YAJL 2.x. This is the fastest, but might require a compiler and the YAJL development files to be present when installing this package. Binary wheel distributions exist for major platforms/architectures to spare users from having to compile the package.
yajl2_cffi: wrapper around YAJL 2.x using CFFI.
yajl2: wrapper around YAJL 2.x using ctypes, for when you can’t use CFFI for some reason.
yajl: deprecated YAJL 1.x + ctypes wrapper, for even older systems.
python: pure Python parser, good to use with PyPy
You can import a specific backend and use it in the same way as the top level library:
import ijson.backends.yajl2_cffi as ijson for item in ijson.items(...): # ...
Importing the top level library as import ijson uses the pure Python backend.
Acknowledgements
ijson was originally developed and actively maintained until 2016 by Ivan Sagalaev. In 2019 he handed over the maintenance of the project and the PyPI ownership.
Python parser in ijson is relatively simple thanks to Douglas Crockford who invented a strict, easy to parse syntax.
The YAJL library by Lloyd Hilaiel is the most popular and efficient way to parse JSON in an iterative fashion.
Ijson was inspired by yajl-py wrapper by Hatem Nassrat. Though ijson borrows almost nothing from the actual yajl-py code it was used as an example of integration with yajl using ctypes.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for ijson-2.4-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 165f51858dbde00a714f06ecec067adb81016188699a78f2afc2300abfdf7ad6 |
|
MD5 | 5dd5fa32b673ffe5559c71df76abb544 |
|
BLAKE2b-256 | c4ec6e835735bdd0552bf4ad8ee021b573e371541b9c38f06e6d41d78dd8542d |
Hashes for ijson-2.4-cp37-cp37m-macosx_10_6_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 558dd253510314087f960a81e5bef5fdce8abb3576626bdc38736f95c2cf51c0 |
|
MD5 | 852fc3f22fba5ac664ace0a74e0bdeeb |
|
BLAKE2b-256 | 671088c317a7d57a3405ab7c18970a0380de8796d90c5a4f3fe72dbac9cf8e0a |
Hashes for ijson-2.4-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b405b4411ea5668832edc7aa4ea1a34536e7917a9d24edee2031a7caa6597b06 |
|
MD5 | ee4dd7f1d6a00265c968699c192b262e |
|
BLAKE2b-256 | 4fdc9fa0f204490229f9d4e40169ede79e1d6a29c432ff9061055aab3b517873 |
Hashes for ijson-2.4-cp36-cp36m-macosx_10_6_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d8a927a72c69df4a786943a514c9cd63621b76eac8143b11d58e7c625c81ba74 |
|
MD5 | cf64264c7331cc21d1ba87addaa55565 |
|
BLAKE2b-256 | 80146c67715c09f176405e9e1e3c094c2f7ef39b4f0f574a0577bb3d253323d6 |
Hashes for ijson-2.4-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3fccd8e0808b603afe8594f6fbf1b71cc3e69bc7eedec5b7a88d828448b8d21c |
|
MD5 | 9e64b012effdb0c7372b6e47a352d1b8 |
|
BLAKE2b-256 | b4497525d5a38e1662d7efa1ec6e7e656bab416b80d790afe7602e32689d3209 |
Hashes for ijson-2.4-cp35-cp35m-macosx_10_6_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 659ea9f2ab515b39cc72e604e64c4fcce7d66dbbda281b4cf76066be1a37f2fe |
|
MD5 | 642ace5e6364e25cfb8e4b9be18d0f2a |
|
BLAKE2b-256 | a0b64ccf845118385c356b957c82858a8f7d0d0411a7c960e4ce737c55e04654 |
Hashes for ijson-2.4-cp34-cp34m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e6307625812a663dd0ee9c398087340a79e77b1e26ff7532bf537980df167ddd |
|
MD5 | 6344a19f3fbf9e2318c87177484508bc |
|
BLAKE2b-256 | 918aca4b5cb1c0707142716ded80b2d11555a4b21ade68e05930161694660448 |
Hashes for ijson-2.4-cp34-cp34m-macosx_10_6_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8145053b83c24a42313d616a92cc46a06fcf1365bc3dd4f97b4be69bf8f4b92c |
|
MD5 | ac4413886f864020b885884767e3cc93 |
|
BLAKE2b-256 | 73d15e076512f3186a76011a36f0aefcd807e1647f22a12694a3569276f672f4 |
Hashes for ijson-2.4-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 88911b226d718af9a217746bae0859ae7e9e4c211ba97d1e1744a29726d57672 |
|
MD5 | be66f4cd5b1b74b894054f6f7df77c2d |
|
BLAKE2b-256 | 75de38e0e45283638a0fe6354d684a6718da5237421494c6f960f1878af70fd6 |
Hashes for ijson-2.4-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f7f88203b41e1ceb5ebc9b2de804005923b78ff86af431ecbad65c3078e65f0d |
|
MD5 | 50ec031dc8dbb2632b03cf31a78bff1b |
|
BLAKE2b-256 | 9a55eb973069f46f0535402ab1231a29819747cb3b0cebdeb69fb9ee52259505 |
Hashes for ijson-2.4-cp27-cp27m-macosx_10_6_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ca97e844a23729e06608465a0c24f57d4ce89299fa1640fddd165847d319c99a |
|
MD5 | 8445d440b28eecf41a060117a71d7dec |
|
BLAKE2b-256 | 24623983259477849d50e134f1bde14a8aa85b756dc435a4a2250ac186bac124 |