Skip to main content

Configurable and documentable Json transformation and mapping

Project description

Piri

Configurable Data Mapping for mortals


test codecov wemake-python-styleguide


Documentation | Source Code | Task Tracker

Goal

The goal of this library is to make JSON to JSON transformation/mapping configurable. We achieve this by using a simple but feature-rich JSON configuration which then also acts as a contract.

Features

  • Mapping with configuration File.
  • Transforms JSON
  • Combine multiple values to one.
  • Default values
  • If statements
    • is, contains, not
  • casting
    • integer, decimal, iso date

Contributing

Please see contribute

Installation

!!! info Package is on pypi. Use pip or poetry to install

pip install piri
poetry add piri

Quickstart

import simplejson

from piri.mapper import map_data

my_config = {
    'name': 'schema',
    'array': False,
    'objects': [
        {
            'name': 'invoices',
            'array': True,
            'path_to_iterable': ['root', 'invoices'],
            'attributes': [
                {
                    'name': 'amount',
                    'mappings': [
                        {
                            'path': ['invoices', 'amount'],
                        },
                    ],
                    'casting': {
                        'to': 'decimal',
                        'original_format': 'integer_containing_decimals',
                    },
                    'default': 0,
                },
                {
                    'name': 'debtor',
                    'mappings': [
                        {
                            'path': ['root', 'customer', 'first_name'],
                        },
                        {
                            'path': ['root', 'customer', 'last_name'],
                        },
                    ],
                    'separator': ' ',
                },
            ],
            'objects': [],
        },
    ],
}

example_data = {
    'root': {
        'customer': {
            'first_name': 'John',
            'last_name': 'Smith',
        },
        'invoices': [
            {
                'amount': 10050,
            },
            {
                'amount': 20050,
            },
            {
                'amount': -15005,
            },
        ],
    },
}

mapped_data = map_data(example_data, my_config)

with open('resultfile.json', 'w') as output_file:
    output_file.write(simplejson.dumps(mapped_data.unwrap()))

contents of resultfile.json

{
    "invoices": [
        {
            "amount": 100.5,
            "debtor": "John Smith"
        },
        {
            "amount": 200.5,
            "debtor": "John Smith"
        },
        {
            "amount": -150.05,
            "debtor": "John Smith"
        }
    ]
}

Process

The Process function tries to make it easy to run all steps in order. Since its a callable object then dependencies(functions) can be changed before calling. This is the execution order:

  • validate configuration data
    • configuration must be valid and also we apply some output formatting to the configuration data.
  • run pre processing
    • a pre_process function can be supplied that must change raw data to python dictionary if it isn't already
  • map data
    • run mapping function with the validated configuration and pre_processed data.
    • this outputs a dictionary
  • validate and marshall
    • this loads the mapping result into a marshmallow Schema that will validate the data for us. after that it will return the dump(marshall) of the schema that can apply any output formatting rules needed.
    • this is the only function that is required to provide when process is initiated.
  • create output
    • run provided output function.
    • if none is provided it simply returns the dictionary.
    • to provide an output function simply add it while initiating Process. with the argument _output=function. The function must receive dictionary and can return anything

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for piri, version 1.0.0
Filename, size File type Python version Upload date Hashes
Filename, size piri-1.0.0-py3-none-any.whl (16.5 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size piri-1.0.0.tar.gz (15.0 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page