Map flat data to structured JSON via a mapping.
Project description
# jsonmapping [![Build Status](https://travis-ci.org/pudo/jsonmapping.svg?branch=master)](https://travis-ci.org/pudo/jsonmapping)
To transform flat data structures into nested object graphs matching JSON
schema definitions, this package defines a mapping language. It defines how
the columns of a source data set (e.g. a CSV file, database table) are to be
converted to the fields of a JSON schema.
The format allows mapping nested structures, including arrays. It also supports
the application of very basic data transformation steps, such as generating a
URL slug or hashing a column value.
## Example mapping
The mapping format is independent of any particular JSON schema, such that
multiple mappings could be defined for any one particular schema.
```json
{
"schema": {"$ref": "http://www.popoloproject.com/schemas/person.json"},
"mapping": {
"id": {"column": "person_id"},
"name": {"column": "person_name"},
"memberships": [{
"mapping": {
"role": {"default": "Member of Organization"},
"organization": {
"mapping": {
"id": {
"columns": ["org_id"],
"constant": "default-org"
},
"name": {
"column": "org_name",
"constant": "Default Organization",
"transforms": ["strip"]
}
}
}
}
}]
}
}
```
This mapping would apply to a four-column CSV file and map it to a set of
nested JSON objects (a [Popolo](http://www.popoloproject.com/) person, with a
membership in an organization).
## Data Transforms
While ``jsonmapper`` is not a data cleaning tool, it supports some very basic
data transformation operations that can be applied on a particular column or
set of columns. These include:
* ``coalesce``: Select the first non-null value from the list of items.
* ``slugify``: Transform each string into a URL slug form.
* ``join``: Merge together the string values of all selected columns.
* ``upper``: Transform the text to upper case.
* ``lower``: Transform the text to lower case.
* ``strip``: Remove leading and trailing whitespace.
* ``hash``: Generate a SHA1 hash of the given value.
## Tests
The test suite will usually be executed in it's own ``virtualenv`` and perform a
coverage check as well as the tests. To execute on a system with ``virtualenv``
and ``make`` installed, type:
```bash
$ make test
```
To transform flat data structures into nested object graphs matching JSON
schema definitions, this package defines a mapping language. It defines how
the columns of a source data set (e.g. a CSV file, database table) are to be
converted to the fields of a JSON schema.
The format allows mapping nested structures, including arrays. It also supports
the application of very basic data transformation steps, such as generating a
URL slug or hashing a column value.
## Example mapping
The mapping format is independent of any particular JSON schema, such that
multiple mappings could be defined for any one particular schema.
```json
{
"schema": {"$ref": "http://www.popoloproject.com/schemas/person.json"},
"mapping": {
"id": {"column": "person_id"},
"name": {"column": "person_name"},
"memberships": [{
"mapping": {
"role": {"default": "Member of Organization"},
"organization": {
"mapping": {
"id": {
"columns": ["org_id"],
"constant": "default-org"
},
"name": {
"column": "org_name",
"constant": "Default Organization",
"transforms": ["strip"]
}
}
}
}
}]
}
}
```
This mapping would apply to a four-column CSV file and map it to a set of
nested JSON objects (a [Popolo](http://www.popoloproject.com/) person, with a
membership in an organization).
## Data Transforms
While ``jsonmapper`` is not a data cleaning tool, it supports some very basic
data transformation operations that can be applied on a particular column or
set of columns. These include:
* ``coalesce``: Select the first non-null value from the list of items.
* ``slugify``: Transform each string into a URL slug form.
* ``join``: Merge together the string values of all selected columns.
* ``upper``: Transform the text to upper case.
* ``lower``: Transform the text to lower case.
* ``strip``: Remove leading and trailing whitespace.
* ``hash``: Generate a SHA1 hash of the given value.
## Tests
The test suite will usually be executed in it's own ``virtualenv`` and perform a
coverage check as well as the tests. To execute on a system with ``virtualenv``
and ``make`` installed, type:
```bash
$ make test
```
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
jsonmapping-0.2.tar.gz
(7.2 kB
view details)
Built Distribution
File details
Details for the file jsonmapping-0.2.tar.gz
.
File metadata
- Download URL: jsonmapping-0.2.tar.gz
- Upload date:
- Size: 7.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 54c5afd7fbb0f169ada1934eca2d9a5c691ac47be36a284385c7303ef8712ff1 |
|
MD5 | 104c43d2d5304dcbbbe2840c7ce44fda |
|
BLAKE2b-256 | 2e491f160712dedab8264248ca00a4d0a48726fd5f736012f1d5c474740aec2d |
File details
Details for the file jsonmapping-0.2-py2-none-any.whl
.
File metadata
- Download URL: jsonmapping-0.2-py2-none-any.whl
- Upload date:
- Size: 9.4 kB
- Tags: Python 2
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 208208e98a8640e445dadb7e809cea17ba37a65a606e42a05de17d4a7d8fdcb4 |
|
MD5 | d577f9c8884970718833bbe4e2f60823 |
|
BLAKE2b-256 | 26a967a756b862d4b523d871f79561db624b3e404c722f9398227bea50ad311a |