xmldataset: simple XML parsing 🗃️

Extract Python datasets from XML with a simple profile that mirrors the XML.

A profile lists the elements and attributes you want, indented the same way as the XML, and names the dataset each value belongs to. xmldataset returns every dataset as a list of records (plain dicts), ready for JSON, CSV, pandas or a database.

Profiles can be written as indented text, YAML or JSON. All three are equivalent, and you can convert between them at any time.

• Documentation: https://xmldataset.readthedocs.io

Installation

pip install xmldataset

xmldataset requires Python 3.11 or newer. Its only dependency is PyYAML, for YAML profiles.

Example

Given this XML:

<colleagues>
    <colleague>
        <title>The Boss</title>
        <email>boss@example.com</email>
    </colleague>
    <colleague>
        <title>Minion</title>
    </colleague>
</colleagues>

this profile collects every colleague into a dataset called colleagues:

colleagues
    colleague
        title = dataset:colleagues
        email = dataset:colleagues,default:none

colleagues:
  colleague:
    title: colleagues
    email:
      $dataset: colleagues
      $default: none

{
  "colleagues": {
    "colleague": {
      "title": "colleagues",
      "email": {
        "$dataset": "colleagues",
        "$default": "none"
      }
    }
  }
}

import xmldataset

result = xmldataset.parse_using_profile(xml, profile)

{
  "colleagues": [
    {"title": "The Boss", "email": "boss@example.com"},
    {"title": "Minion", "email": "none"}
  ]
}

The Minion has no email address, so the default fills it in.

Load a profile file in any format with xmldataset.load_profile("profile.yaml"), or stream records from a large file in flat memory:

from pathlib import Path

for dataset, record in xmldataset.iter_records(Path("huge.xml"), profile):
    ...

Or skip Python entirely:

xmldataset run profile.yaml data.xml --format csv --dataset colleagues
xmldataset convert profile.txt --to yaml

Features

• Three profile formats: indented text, YAML or JSON, with a JSON Schema for editor completion and validation.
• Elements and attributes, with renaming, prefixes, defaults, lists of repeated values and per-value process functions.
• Correct records by construction: a record never mixes values from two elements, and values from an enclosing element (a shop's number, say) are copied into every record inside it.
• Streams documents of any size in flat memory from a string, bytes, a path or a file, and hands over each record as soon as it is complete.
• Namespaces, matched by local name or by prefix.
• A command-line tool to run, convert and validate profiles.
• Typed, with PyYAML as the only dependency.

Contributing

See CONTRIBUTING.md. Changes are recorded in CHANGELOG.md.

License

BSD-3-Clause. See LICENSE.