mappings_utility 0.2.5

mapping utility tool for sdmx based partial mappings

An SDMX mapping utility to generate partial key maps for referential metadata. Although not designed to do that, data mappings are also possible. The package is structured in such a way that it can be used as a package without the FASTAPI overhead.

Three methods are available:

• map_withFile GET method (receiving the mapping source and mapping rules both as file references)
• map_withURN GET method (receiving the mapping source as file and the mapping rules as SDMX registry endpoint + mapping ID)
• map_json_withURN POST method (receiving the mapping source as the body of the request in json - pandas dataframe dictionary style - and mapping rules as SDMX registry endpoint + mapping ID) (added in version 0.1.1)

A typical usage example in package mode:

    from mappings_utility.sdmx_mapping_utility import SDMXMappingUtility 
    
    mf = Path('to-mapping-artefacts.xml')
    sf = Path('to-file-to-be-mapped.csv')
    mu = SDMXMappingUtility.initialise_with_file(mf, sf, sid_based=False)
    df = mu.generate_mappings_partial_keys(
        includeSourceColumns=False,
        includeAttributesMeasures=False,
        nulledFixedTargets=True,
        writeTargetStructInfo=False,
        dropInvalid=False,
        sid_based=False
        )
    df.to_csv(Path('mapped-file.csv'), index=False)

The parameters fine-tune the output file:

• includeSourceColumns: True/False; when set to True the resulting file would include the fields of the source csv file
• includeAttributesMeasures: True/False; when True it will include the attribute columns and obs_value, otherwise it will suppress those from the output (this is needed for referential metadata partial key mappings)
• nulledFixedTargets: True/False, when True fixed values in the target mapping will be set to Null, also a partial-key mapping motivated feature
• writeTargetStructInfo: True/False, when True it adds or maps columns necessary for the output to be SDMX-CSV compliant
• dropInvalid: True/False, when set to True the output only contains valid mappings (although validity checks are lazy), invalid rows are ommitted; with False the returned dataframe will contain cells with a special string highlighting the positions where the target value could not be determined
• sid_based: True/False, when set to True the resulting file will contain a Series ID (SID) field with the dimensions of the target data model dot-concatenated

Changelog:

• Version 0.2.3 Introduced validation for implicit maps (target codes are checked whether they are in the referenced codelist - constraints are not yet taken into account)
• Version 0.2.4 Bug-fix for regex substitution (previously substitutions with more than one substitution failed)
• Version 0.2.5
  • Attributes scaffolding for future features (validates coded attributes)
  • Bug-fix for optional attributes missing when all values are null