detakon 0.2.0

No project description provided

detakon

---

Description

A data converter than uses configuration maps (referred to as detamaps) to map fields in the source data to output data, and perform operations on the data before outputting. Detamaps may also contain configuration data for the data source and output, and defaults to use if fields are missing, or blank/empty.

[!NOTE] Detakon is in early development and may have breaking changes between versions. Detamaps, methods, and usage may change between minor versions until version 1.0 release. After version 1.0 release, any breaking changes will result in a major version bump. Recommended to use PyPI version. Version in source code is next planned release on PyPI, and is not official until pushed to PyPI.

Table of Contents

• Installation
• Usage
• Detamap Files
• License

Installation

pip install detakon

Usage

[!WARNING] Status: Currently in pre-release, as minimal viable product for CSV to CSV conversions only. Not for usage in production systems, breaking changes are coming in future releases. Some planned changes can be viewed in the ROADMAP.md file.

Detakon(detamap, source, destination)

Sample:

from detakon.detakon import Detakon

converter = Detakon("invoice_detamap.json", "data_dump_2026-01-01.csv", "invoice_2026-01-01.csv")
converter.convert()

Detamap Files

A detamap configuration file is used to provide all details necessary for the data conversion.

A datamap must be a Python dictionary, or convertable to a Python dictionary. Currently this means either a dictionary or JSON file (plans to add TOML support).

A detamap MUST include the following key:value pairs:

• "Mappings" with a sub-dictionary of source data field names as keys, mapped to output field names as values.
• "Defaults" with a sub-dictionary of source field names that will supply a default value if either: (not currently implemented)
  • Matching field was not found in source.
  • Matching source data was empty, such as an empty string.
• "Operations" with a list of operations to perform on source data: (not currently implemented)
  • Example operations may be "upper", "convertValue", "strip", etc.
  • Operations entries are provided as dictionaries with sub-dictionaries for the fields the operations are to be performed on (single string value, list of strings, or "*" to indicate all fields) and an arguments key that contains a list of arguments to be passed. Arguments may be an empty list if no arguments. A dictionary may be passed as a single argument, if the operation expects a dictionary as the argument, but must be passed withing a list.
  • Operations are performed in order they appear in the list.
  • Operations may be listed multiple times with different argument and field values.
• "Source" with a sub-dictionary that defines the type of source being passed, and the arguments to pass to Path.open() if source is a file, or csv.DictReader if source is CSV data.
  • Required keys:
    • "argument" as type of argument being passed as the source argument to the Detakon initializer. Accepted values: "filepath".
      • "type" key may be required depending on the argument being passed.
• "Output" with a sub-dictionary that defines the type of output expected, and the arguments to pass to Path.open() if destination is a file, or csv.DictReader if output is CSV data.
  • "fields" sub-key as a list of strings corresponding to the output field names in the desired order.

A detamap JSON file may look similar to:

{
    "Mappings": {
        "Customer Id": "External ID",
        "First Name": "First Name",
        "Last Name": "Last Name",
        "Company": "Company",
        "Country": "Country",
        "Phone 1": "Phone",
        "Phone 2": "Cell",
        "Email": "Email",
        "Website": "URL",
        "Postal Code": "Zip"
    },
    "Defaults": {
        "Postal Code": "00000"
    },
    "Operations": [
        {"hashmap": {"fields": "Country",
                        "arguments": [{
                            "United States of America": "USA",
                            "Antarctica (the territory South of 60 deg S)": "Antarctica"}]
                        }
                    },
        {"strip": {"fields": "*"}},
        {"strip": {"arguments": ["+1-"], "fields": ["Phone 1", "Phone 2"]}},
        {"upper": {"fields": ["Email", "Last Name", "First Name"]}}
    ],
    "Source": {
        "argument": "filepath",
        "type": "str",
        "encoding": "utf-8",
        "format": "csv",
        "delimiter": ","
    },
    "Output": {
        "fields": ["External ID", "Company", "Last Name", "First Name", "Country", "Zip", "Phone", "Cell", "Email", "URL"],
        "argument": "filepath",
        "type": "str",
        "append": false,
        "omit_heading": false,
        "encoding": "utf-8",
        "format": "csv",
        "delimiter": ","
    }
}

Above sample is not all-inclusive. Detamaps, source data, and output expect full chain of custody. No security guarantees are made. Users are advised not to use untrusted, or malformed data.

License

detakon is distributed under the terms of the MIT license.