ASN2RDF

A tool to convert ASN.1 modules into SHACL shapes, an OWL ontology and SKOS concept schemes, with the mapping model of xsd2rdf.

Overview

asn2rdf reads ASN.1 modules with asn1tools and maps each construct onto the construct xsd2rdf produces for its XML Schema counterpart: a SEQUENCE is handled like a complexType, a CHOICE like a strict xs:choice, an ENUMERATED like an enumeration, and so on. It then uses xsd2rdf 2.1 itself for everything after the translation: the shape model, the OWL derivation from the shapes, the -k naming policy, the SKOS writer, the output file names and the SHACL-SHACL check. The output of both tools can therefore be merged and treated alike.

asn2rdf knows nothing about a particular domain. Namespaces and naming are options; domain-specific fixes belong in the pipeline that uses the output.

Installation

pip install asn2rdf

From source:

python -m pip install poetry
poetry install

Basic usage

asn2rdf convert path/to/module.asn -o out

This writes, as xsd2rdf does for an XSD:

• module.asn.shape.ttl (SHACL shapes)
• module.asn.owl.ttl (OWL ontology)
• module.asn.skos.<Name>.ttl (SKOS concept schemes, one file per ENUMERATED)
• module.asn.provenance.ttl with --source-provenance

Several files or folders (their *.asn and *.asn1 files) can be given; each file is converted on its own, and the other files of the same call are used to resolve its IMPORTS. python -m asn2rdf works as well.

From Python

from asn2rdf import convert

result = convert("module.asn", property_kind_qualified_iris=True, library=["other.asn"])
result.shacl, result.owl  # the graphs, nothing is written yet
result.skos               # one graph per concept scheme
result.write("out")       # writes the same files as the command line

convert returns xsd2rdf's Result. The settings are the fields of asn2rdf.Options; pass them as keywords or as convert(path, Options(...)). library lists other ASN.1 files whose modules IMPORTS may refer to. Progress and warnings go to the asn2rdf and xsd2rdf loggers.

Command line parameters

The flags are those of xsd2rdf convert where the concept exists.

• -x, --asn-file: ASN.1 file to be converted
• -f, --folder: folder whose ASN.1 files are converted one by one
• -o, --output-dir: output directory (default: the folder of the ASN.1 file)
• -a, --abbreviations-file: file with abbreviations that labels keep in capitals, one per line
• -d, --debug: debug output
• -q, --quiet: only print warnings and errors
• -n, --namespaced-concepts: concept IRIs <ns>concepts/<scheme>/<item> instead of <ns>concepts/<scheme>_<item>
• -u, --namespace: namespace of the generated IRIs. An ASN.1 module has no namespace, so the default is http://example.com/; shapes go in <namespace>shapes/
• -p, --has-prefix-for-properties: property IRIs start with has
• -s, --skip-shacl-validation: skip the check of the shapes against SHACL-SHACL
• --date YYYY-MM-DD: date written as dct:created, dct:issued and dct:modified (default: the day of SOURCE_DATE_EPOCH if set, else today)
• --strict: stop with an error when the shapes are not well-formed SHACL
• -k, --property-kind-qualified-iris: append DP or OP to every property IRI according to its OWL property kind (xsd2rdf's naming policy), so that no IRI is both an owl:DatatypeProperty and an owl:ObjectProperty
• --functional-properties / --no-functional-properties: declare owl:FunctionalProperty for properties whose shapes all have sh:maxCount 1. Off by default (xsd2rdf has it on): it is a global axiom derived from a local cardinality
• --source-provenance: also write <file>.provenance.ttl (see below)
• --language: language tag of labels and of the comments read from the module (default en)
• --license IRI: the licence of the output, the object of cc:license on each SKOS concept scheme. The default is the EUPL 1.2 that xsd2rdf writes (http://data.europa.eu/eli/dec_impl/2017/863/oj); xsd2rdf puts no licence on the ontology header, so neither does asn2rdf. xsd2rdf has no such option

xsd2rdf options without an ASN.1 counterpart are not offered: --no-pruning (every type assignment of the given file is converted; imported types only when used), --source-namespaces and --namespace-map (ASN.1 modules have no namespace IRI) and --simple-type-subproperties.

Mapping rules

Every rule has a fixture in tests/fixtures with a minimal module, the golden output and instance data. ex: is the namespace, nodeshapes: and propertyshapes: are <ns>shapes/NodeShape/ and <ns>shapes/PropertyShape/.

Types and components

Built-in types and constraints

A list of lists (SEQUENCE OF SEQUENCE OF X) cannot be kept in RDF without an intermediate node; it is flattened to one list of values, with a warning. A component whose type is neither built in, defined nor imported gets a property shape without value constraints, with a warning.

Enumerations

Items without a number get the numbers ASN.1 assigns. Items after the extension marker are included.

Documentation

ASN.1 has no annotation syntax, so comments are the documentation of a module. asn1tools drops them; asn2rdf scans the module once more and attaches:

• the comment lines directly above a type assignment, component or enumeration item (a blank line or a line with code ends the block), and
• a comment after the item on the same line, continued by the comment lines right below it that are indented at least as far,

as sh:description of the shape (and so rdfs:comment of the class or property), or skos:definition of a concept scheme or concept. Banner lines made only of #, =, *, - or . are dropped. Labels (sh:name, rdfs:label, skos:prefLabel) are made from the names as in xsd2rdf (issuingDay becomes "Issuing day"; -a keeps abbreviations in capitals).

Same value in two forms

Some modules carry one value in two forms of which only one is to be present, for example productOwnerNum INTEGER and productOwnerIA5 IA5String. asn2rdf maps them as written: two properties, each with its own datatype and constraints. Merging them into one property is a modelling decision for the ontology that uses the output; the provenance file gives the ASN.1 identifier and type of each.

Ontology

The OWL file declares <namespace> a owl:Ontology with rdfs:label, dct:title and dct:description Ontology generated from the ASN.1 module <name> and the date as dct:created, dct:issued and dct:modified. The classes, properties, domains and ranges are derived from the shapes by xsd2rdf; see its README.

Source provenance

With --source-provenance, <file>.provenance.ttl links each shape, class, property and concept scheme to the ASN.1 component it was made from with dct:source (xsd:anyURI): the file, relative to the converted file, and the ASN.1 absolute reference of X.680 clause 14, for example

uicRailTicketData_v3.0.3.asn#@ASN-Module-RailTicketData.DocumentData.ticket.reservation

where * stands for the element of a SEQUENCE OF, as in ...Document.legs.*.km. Each property shape also gets the identifier and the type of its component as written (asn2rdf:asnIdentifier "productOwnerNum", asn2rdf:asnType "INTEGER" or "SEQUENCE OF TicketLinkType"), so that data can be mapped back to the ASN.1 structure. The other output files do not change.

The asn2rdf vocabulary

asn2rdf:asnIdentifier and asn2rdf:asnType are annotation properties in the namespace https://w3id.org/asn2rdf#, defined in vocabulary/asn2rdf.ttl. They are informative only and have no logical meaning. The provenance file declares them too.

Tests

poetry install
poetry run pytest

The mapping rules are covered by the fixture suite in tests/fixtures: one folder per rule with a minimal module, the golden output and instance data that must (not) conform to the generated shapes. After an intended change of the output, regenerate the golden files with poetry run pytest --update-golden and review the diff. tests/golden/b12 is a golden test on real modules: the B12 modules (UIC rail ticket data) of the ERA telematics ontology. ruff check src tests and mypy must pass.

The fixtures are also published as a documentation site on GitLab Pages, built by the pages job on main: one page per case with the ASN.1 source, the golden output, the instance data and UML-like class diagrams of the source, the SHACL shapes and the OWL ontology, plus the mapping rules above and a summary of the B12 golden test. The site is at https://era-europa-eu.gitlab.io/private/OPDU/SOT/era-ontology/telematics/asn2rdf/ (Deploy → Pages in GitLab shows the address if the project uses a unique Pages domain; the site is visible to whoever can see the project). To build it locally (Mermaid then loads from a CDN):

poetry run python docs/build_site.py public

Releasing

Releases are published to PyPI from GitLab CI.

1. Update CHANGELOG.md: move the changes to a new [X.Y.Z] - YYYY-MM-DD section and merge it to main.
2. In GitLab, create a release (Deploy → Releases → New release) with a new tag vX.Y.Z created from main. The release notes can go in its description.
3. The tag pipeline runs lint and the tests, sets the package version from the tag (v1.2.3 gives 1.2.3), builds the wheel and the sdist, and uploads them to PyPI with trusted publishing: no PyPI password or token is stored in GitLab. Only tags of the form vX.Y.Z (or X.Y.Z) build and publish.

The PyPI project has a GitLab trusted publisher with these settings (before the first release, when the project does not exist on PyPI yet, add it as a pending publisher under Your account → Publishing):

License

EUPL 1.2