This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Iterative JSON parser with a standard Python iterator interface

Project Description

ijson

Ijson is an iterative JSON parser with a standard Python iterator interface.

Usage

All usage example will be using a JSON document describing geographical objects:

{
  "earth": {
    "europe": [
      {"name": "Paris", "type": "city", "info": { ... }},
      {"name": "Thames", "type": "river", "info": { ... }},
      // ...
    ],
    "america": [
      {"name": "Texas", "type": "state", "info": { ... }},
      // ...
    ]
  }
}

Most common usage is having ijson yield native Python objects out of a JSON stream located under a prefix. Here’s how to process all European cities:

import ijson

f = urlopen('http://.../')
objects = ijson.items(f, 'earth.europe.item')
cities = (o for o in objects if o['type'] == 'city')
for city in cities:
    do_something_with(city)

Sometimes when dealing with a particularly large JSON payload it may worth to not even construct individual Python objects and react on individual events immediately producing some result:

import ijson

parser = ijson.parse(urlopen('http://.../'))
stream.write('<geo>')
for prefix, event, value in parser:
    if (prefix, event) == ('earth', 'map_key'):
        stream.write('<%s>' % value)
        continent = value
    elif prefix.endswith('.name'):
        stream.write('<object name="%s"/>' % value)
    elif (prefix, event) == ('earth.%s' % continent, 'end_map'):
        stream.write('</%s>' % continent)
stream.write('</geo>')

Backends

Ijson provides several implementations of the actual parsing in the form of backends located in ijson/backends:

  • yajl2_cffi: wrapper around YAJL 2.x using CFFI, this is the fastest.
  • yajl2: wrapper around YAJL 2.x using ctypes, for when you can’t use CFFI for some reason.
  • yajl: deprecated YAJL 1.x + ctypes wrapper, for even older systems.
  • python: pure Python parser, good to use with PyPy

You can import a specific backend and use it in the same way as the top level library:

import ijson.backends.yajl2_cffi as ijson

for item in ijson.items(...):
    # ...

Importing the top level library as import ijson uses the pure Python backend.

Acknowledgements

Python parser in ijson is relatively simple thanks to Douglas Crockford who invented a strict, easy to parse syntax.

The YAJL library by Lloyd Hilaiel is the most popular and efficient way to parse JSON in an iterative fashion.

Ijson was inspired by yajl-py wrapper by Hatem Nassrat. Though ijson borrows almost nothing from the actual yajl-py code it was used as an example of integration with yajl using ctypes.

Release History

Release History

This version
History Node

2.3

History Node

2.2

History Node

2.1

History Node

2.0

History Node

1.1

History Node

1.0

History Node

0.8.0

History Node

0.7.0

History Node

0.6.1

History Node

0.6.0

History Node

0.5.0

History Node

0.4.0

History Node

0.3.0

History Node

0.2.0

History Node

0.1.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
ijson-2.3-py2.py3-none-any.whl (15.5 kB) Copy SHA256 Checksum SHA256 3.4 Wheel Feb 6, 2016
ijson-2.3.tar.gz (10.3 kB) Copy SHA256 Checksum SHA256 Source Feb 6, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting