Skip to main content
https://travis-ci.com/ICRAR/ijson.svg?branch=master https://coveralls.io/repos/github/ICRAR/ijson/badge.svg?branch=master https://badge.fury.io/py/ijson.svg

ijson

Ijson is an iterative JSON parser with a standard Python iterator interface.

Usage

All usage example will be using a JSON document describing geographical objects:

{
  "earth": {
    "europe": [
      {"name": "Paris", "type": "city", "info": { ... }},
      {"name": "Thames", "type": "river", "info": { ... }},
      // ...
    ],
    "america": [
      {"name": "Texas", "type": "state", "info": { ... }},
      // ...
    ]
  }
}

Most common usage is having ijson yield native Python objects out of a JSON stream located under a prefix. Here’s how to process all European cities:

import ijson

f = urlopen('http://.../')
objects = ijson.items(f, 'earth.europe.item')
cities = (o for o in objects if o['type'] == 'city')
for city in cities:
    do_something_with(city)

For how to build a prefix see the Prefix section below.

Other times it might be useful to iterate over object members rather than objects themselves (e.g., when objects are too big). In that case one can use the kvitems functions instead:

import ijson

f = urlopen('http://.../')
european_places = ijson.kvitems(f, 'earth.europe.item')
names = (v for k, v in european_places if k == 'name')
for name in names:
    do_something_with(name)

Sometimes when dealing with a particularly large JSON payload it may worth to not even construct individual Python objects and react on individual events immediately producing some result:

import ijson

parser = ijson.parse(urlopen('http://.../'))
stream.write('<geo>')
for prefix, event, value in parser:
    if (prefix, event) == ('earth', 'map_key'):
        stream.write('<%s>' % value)
        continent = value
    elif prefix.endswith('.name'):
        stream.write('<object name="%s"/>' % value)
    elif (prefix, event) == ('earth.%s' % continent, 'end_map'):
        stream.write('</%s>' % continent)
stream.write('</geo>')

Events

When using the lower-level ijson.parse function, three-element tuples are generated containing a prefix, an event name, and a value. Events will be one of the following:

  • start_map and end_map indicate the beginning and end of a JSON object, respectively. They carry a None as their value.

  • start_array and end_array indicate the beginning and end of a JSON array, respectively. They also carry a None as their value.

  • map_key indicates the name of a field in a JSON object. Its associated value is the name itself.

  • null, boolean, integer, double, number and string all indicate actual content, which is stored in the associated value.

Prefix

A prefix represents the context within a JSON document where an event originates at. It works as follows:

  • It starts as an empty string.

  • A <name> part is appended when the parser starts parsing the contents of a JSON object member called name, and removed once the content finishes.

  • A literal item part is appended when the parser is parsing elements of a JSON array, and removed when the array ends.

  • Parts are separated by ..

When using the ijson.items function, the prefix works as the selection for which objects should be automatically built and returned by ijson.

Backends

Ijson provides several implementations of the actual parsing in the form of backends located in ijson/backends:

  • yajl2_c: a C extension using YAJL 2.x. This is the fastest, but might require a compiler and the YAJL development files to be present when installing this package. Binary wheel distributions exist for major platforms/architectures to spare users from having to compile the package.

  • yajl2_cffi: wrapper around YAJL 2.x using CFFI.

  • yajl2: wrapper around YAJL 2.x using ctypes, for when you can’t use CFFI for some reason.

  • yajl: deprecated YAJL 1.x + ctypes wrapper, for even older systems.

  • python: pure Python parser, good to use with PyPy

You can import a specific backend and use it in the same way as the top level library:

import ijson.backends.yajl2_cffi as ijson

for item in ijson.items(...):
    # ...

Importing the top level library as import ijson uses the first available backend in the same order of the list above.

Acknowledgements

ijson was originally developed and actively maintained until 2016 by Ivan Sagalaev. In 2019 he handed over the maintenance of the project and the PyPI ownership.

Python parser in ijson is relatively simple thanks to Douglas Crockford who invented a strict, easy to parse syntax.

The YAJL library by Lloyd Hilaiel is the most popular and efficient way to parse JSON in an iterative fashion.

Ijson was inspired by yajl-py wrapper by Hatem Nassrat. Though ijson borrows almost nothing from the actual yajl-py code it was used as an example of integration with yajl using ctypes.

Metadata

Release files for ijson 2.6.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ijson 2.6.1
File Size Uploaded Provenance Actions
ijson-2.6.1.tar.gz 29.4 kB Without provenance Details for ijson-2.6.1.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for ijson 2.6.1
File Actions
ijson-2.6.1-cp37-cp37m-manylinux1_x86_64.whl CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.5+ x86-64 Details for ijson-2.6.1-cp37-cp37m-manylinux1_x86_64.whl
ijson-2.6.1-cp37-cp37m-macosx_10_6_x86_64.whl CPython 3.7 CPython 3.7 pymalloc macOS 10.6+ x86-64 Details for ijson-2.6.1-cp37-cp37m-macosx_10_6_x86_64.whl
ijson-2.6.1-cp36-cp36m-manylinux1_x86_64.whl CPython 3.6 CPython 3.6 pymalloc Linux glibc 2.5+ x86-64 Details for ijson-2.6.1-cp36-cp36m-manylinux1_x86_64.whl
ijson-2.6.1-cp36-cp36m-macosx_10_6_x86_64.whl CPython 3.6 CPython 3.6 pymalloc macOS 10.6+ x86-64 Details for ijson-2.6.1-cp36-cp36m-macosx_10_6_x86_64.whl
ijson-2.6.1-cp35-cp35m-manylinux1_x86_64.whl CPython 3.5 CPython 3.5 pymalloc Linux glibc 2.5+ x86-64 Details for ijson-2.6.1-cp35-cp35m-manylinux1_x86_64.whl
ijson-2.6.1-cp35-cp35m-macosx_10_6_x86_64.whl CPython 3.5 CPython 3.5 pymalloc macOS 10.6+ x86-64 Details for ijson-2.6.1-cp35-cp35m-macosx_10_6_x86_64.whl
ijson-2.6.1-cp34-cp34m-manylinux1_x86_64.whl CPython 3.4 CPython 3.4 pymalloc Linux glibc 2.5+ x86-64 Details for ijson-2.6.1-cp34-cp34m-manylinux1_x86_64.whl
ijson-2.6.1-cp34-cp34m-macosx_10_6_x86_64.whl CPython 3.4 CPython 3.4 pymalloc macOS 10.6+ x86-64 Details for ijson-2.6.1-cp34-cp34m-macosx_10_6_x86_64.whl
ijson-2.6.1-cp27-cp27mu-manylinux1_x86_64.whl CPython 2.7 CPython 2.7 pymalloc wide-unicode Linux glibc 2.5+ x86-64 Details for ijson-2.6.1-cp27-cp27mu-manylinux1_x86_64.whl
ijson-2.6.1-cp27-cp27m-manylinux1_x86_64.whl CPython 2.7 CPython 2.7 pymalloc Linux glibc 2.5+ x86-64 Details for ijson-2.6.1-cp27-cp27m-manylinux1_x86_64.whl
ijson-2.6.1-cp27-cp27m-macosx_10_6_x86_64.whl CPython 2.7 CPython 2.7 pymalloc macOS 10.6+ x86-64 Details for ijson-2.6.1-cp27-cp27m-macosx_10_6_x86_64.whl

Total release size: 546.7 kB

Release files / ijson-2.6.1.tar.gz

Download URL ijson-2.6.1.tar.gz
Size 29.4 kB
Tags Source
SHA-256 checksum
How to use checksums
75ebc60b23abfb1c97f475ab5d07a5ed725ad4bd1f58513d8b258c21f02703d0
BLAKE2b-256 checksum
How to use checksums
d0123116e1d5752aa9d480eb58ae4b348d38c1aeaf792c5fbca22e44c27d4bf1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp37-cp37m-manylinux1_x86_64.whl

Download URL ijson-2.6.1-cp37-cp37m-manylinux1_x86_64.whl
Size 65.7 kB
Tags CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.5+ x86-64
SHA-256 checksum
How to use checksums
25d4d159405f75a7443c1fe83b6d7be5a7da017b4aa9cc1bb5cda3feb74aaf32
BLAKE2b-256 checksum
How to use checksums
191d7c97e1dd574667bbaefc9540feafbcdb8c0641ec2c88410c174b7b95e2cb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp37-cp37m-macosx_10_6_x86_64.whl

Download URL ijson-2.6.1-cp37-cp37m-macosx_10_6_x86_64.whl
Size 25.5 kB
Tags CPython 3.7 CPython 3.7 pymalloc macOS 10.6+ x86-64
SHA-256 checksum
How to use checksums
26978c02314233c87bddad8800b7b9a56a052334f495e2bce93b282397c6931d
BLAKE2b-256 checksum
How to use checksums
fcd88966b7f32166e04a336a35a49867b39e45f92e3d34849b5f4a3aa3a3b4ce
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp36-cp36m-manylinux1_x86_64.whl

Download URL ijson-2.6.1-cp36-cp36m-manylinux1_x86_64.whl
Size 65.7 kB
Tags CPython 3.6 CPython 3.6 pymalloc Linux glibc 2.5+ x86-64
SHA-256 checksum
How to use checksums
c0042bb768fb890c177af923c0ead157cdc70c6dfa64827765c1a3676a879190
BLAKE2b-256 checksum
How to use checksums
23422066f77a714ab7221542ea23710b35a96c5dd398f1933429088afd888293
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp36-cp36m-macosx_10_6_x86_64.whl

Download URL ijson-2.6.1-cp36-cp36m-macosx_10_6_x86_64.whl
Size 25.5 kB
Tags CPython 3.6 CPython 3.6 pymalloc macOS 10.6+ x86-64
SHA-256 checksum
How to use checksums
a8b486bdf24e389947e588f4021498f6cc56cafdfaec1c78e9952e0f338aef23
BLAKE2b-256 checksum
How to use checksums
3178f13a7c5a7b14974348bdcd7879890bcd847e6cfb8252087297f0a9d31eba
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp35-cp35m-manylinux1_x86_64.whl

Download URL ijson-2.6.1-cp35-cp35m-manylinux1_x86_64.whl
Size 65.8 kB
Tags CPython 3.5 CPython 3.5 pymalloc Linux glibc 2.5+ x86-64
SHA-256 checksum
How to use checksums
8ce67b7d3435c3fc831d5c06f60b2d20a853b599cdf885478e575a3416fbf655
BLAKE2b-256 checksum
How to use checksums
20064c28511c77b658d58917b7057e42d447353e9d29cd6d8ff8f0f00c37ac2f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp35-cp35m-macosx_10_6_x86_64.whl

Download URL ijson-2.6.1-cp35-cp35m-macosx_10_6_x86_64.whl
Size 25.5 kB
Tags CPython 3.5 CPython 3.5 pymalloc macOS 10.6+ x86-64
SHA-256 checksum
How to use checksums
ae9cc3ebbe8fa030b923b5dff912a61980edd03dc00b92f5c0223e44cbc51d9f
BLAKE2b-256 checksum
How to use checksums
2834ea5383be46d1e6231d4958d3b84a3c94935b256ffaa7cb01da132a055448
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp34-cp34m-manylinux1_x86_64.whl

Download URL ijson-2.6.1-cp34-cp34m-manylinux1_x86_64.whl
Size 65.6 kB
Tags CPython 3.4 CPython 3.4 pymalloc Linux glibc 2.5+ x86-64
SHA-256 checksum
How to use checksums
7bac04b23691e6ab122d8f9ff06b26dbbb6df01babbf6bf8856ccad1c505278b
BLAKE2b-256 checksum
How to use checksums
f6d3494a24842f4595766faf07b4530539b01d9fc12a2e4ba9e8d048e5ba59e8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp34-cp34m-macosx_10_6_x86_64.whl

Download URL ijson-2.6.1-cp34-cp34m-macosx_10_6_x86_64.whl
Size 25.5 kB
Tags CPython 3.4 CPython 3.4 pymalloc macOS 10.6+ x86-64
SHA-256 checksum
How to use checksums
9904bf55bc1f170353c32144861d8295f0bdc41034e5e6ae58cbf30610023ca6
BLAKE2b-256 checksum
How to use checksums
7beed07ebb0f02a57162979ce40f207680cf68d55d11b3a75df8116f1197b8a8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp27-cp27mu-manylinux1_x86_64.whl

Download URL ijson-2.6.1-cp27-cp27mu-manylinux1_x86_64.whl
Size 63.5 kB
Tags CPython 2.7 CPython 2.7 pymalloc wide-unicode Linux glibc 2.5+ x86-64
SHA-256 checksum
How to use checksums
a4cd7f8ecf035d0e23db1cc6d6036e6c563f31abacbceae88904bb8b7f88b1f6
BLAKE2b-256 checksum
How to use checksums
53fc207cd7a039975e6073185fe2012f2ecdfaee5b65727edf92bd801235630c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp27-cp27m-manylinux1_x86_64.whl

Download URL ijson-2.6.1-cp27-cp27m-manylinux1_x86_64.whl
Size 63.5 kB
Tags CPython 2.7 CPython 2.7 pymalloc Linux glibc 2.5+ x86-64
SHA-256 checksum
How to use checksums
d320dc1c1c9adbe404668b0fed6bfa0ac8693911159564f4655a5f2059746993
BLAKE2b-256 checksum
How to use checksums
1d7c1ea72b0777b70246a54e6e7904c17cb1b2d73bf00b72b4c06cd55648eda5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Release files / ijson-2.6.1-cp27-cp27m-macosx_10_6_x86_64.whl

Download URL ijson-2.6.1-cp27-cp27m-macosx_10_6_x86_64.whl
Size 25.4 kB
Tags CPython 2.7 CPython 2.7 pymalloc macOS 10.6+ x86-64
SHA-256 checksum
How to use checksums
60393946d73792d5adeeaa25e82ff2f5bf19b17f6617a468743a4db4a07298a0
BLAKE2b-256 checksum
How to use checksums
f3f63110ff5af7e0658720aabf8b9f24c8beb7e71b5f4b89143cd0e43c73eed5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.8.0

Security for ijson 2.6.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

2.6.1 This release

12 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page