pystyx
Python implementation of Styx ETL declarative TOML.
For more information about Styx, please see the definition here.
Quick Start
-
In your Python project,
mkdirmaps -
touchfunctions.styx (TODD: Add init for this) and functions.py (if you want to write custom functions) -
In
functions.styx, add an array calledfunctions, with the names of any functions you plan on using in your mappings:# functions.styx functions = ["to_upper_case"]
-
In
functions.pyimplement your custom function. Ensure that your function takes the same number of arguments as your expectedinput_paths, or use*args. The# functions.py from pystyx.functions import styx_function @styx_function def to_upper_case(value): return value.upper()
-
Add any of your Styx Definitions to the
maps/directory. For example,from_type = "erp_address" to_type = "Address" [fields] [fields.address1] input_paths = ["addr1"] [fields.address2] input_paths = ["addr2"] [fields.city] input_paths= ["city"] [fields.state] input_paths = ["province"] function = "to_upper_case" [fields.zip] input_paths = ["postalCode"]
-
Initialize maps in your Python code:
// blob.json { "addr1": "123 Street", "addr2": "Ste. 800", "city": "Dallas", "province": "tx", "postalCode": "75080", }
# main.py import json import pathlib import pystyx # Ensure that your decorated functions are evaluated from functions import * with pathlib.Path("blob.json").open() as json_file: blob = json.load(json_file) maps = pystyx.create_maps() # Defaults to `/maps` and `functions.styx` mapper = maps.get("erp_address") # 'from_type` from the Definition above mapped_obj = mapper(blob) print(mapped_obj)
Prints:
{ "address1": "123 Street", "address2": "Ste. 800", "city": "Dallas", "state": "TX", // Upper case now "zip": "75080" }
Styx Validation
pystyx.create_maps() parses (and thereby validates) the Styx files before loading them. I hope to extract this validation as a CLI tool (along with generating Styx structures).
Styx Implementation
I wanted this to be a good first example of a Styx implementation, so I believe it currently implements all features of the Styx standard. Parsing and validation I would like to extract into its own library eventually.
| Feature | Implemented |
|---|---|
| Structures | |
| - JSON objects | ✔️ |
| Validation/Parsing | |
| - Full parsing | ✔️ |
| Preprocessing | |
| - input_paths | ✔️ |
| - output_paths | ✔️ |
| - function | ✔️ |
| - or_else | ✔️ |
| - on_throw | ✔️ |
| Fields | |
| - field header | ✔️ |
| - input_paths | ✔️ |
| - possible_paths | ✔️ |
| - const | ✔️ |
| - from_type | ✔️ |
| - or_else | ✔️ |
| - on_throw | ✔️ |
| - many | ✔️ |
| Postprocessing | |
| - input_paths | ✔️ |
| - output_paths | ✔️ |
| - function | ✔️ |
| - or_else | ✔️ |
| - on_throw | ✔️ |
Metadata
Release files for pystyx 0.8.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pystyx-0.8.5-py3-none-any.whl | Python 3 | none | any | Details for pystyx-0.8.5-py3-none-any.whl |
Release files / pystyx-0.8.5-py3-none-any.whl
| Download URL | pystyx-0.8.5-py3-none-any.whl |
|---|---|
| Size | 10.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b190e4bebc1edb8bcf0311f0d2c00c2074046644adc86e76bef0d5c4fd18e969
|
|
BLAKE2b-256 checksum How to use checksums |
a23e9dd322b623a57c12afed614bc48e9a1b25d9cc6f0b534cc340fa7c8b8b24
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.48.0 CPython/3.8.2
|
Security for pystyx 0.8.5
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.