Mash

Mash makes working with standardized data using RDF easy. It provides ORM-like functionality for mapping your RDF data to classes, and to serialize these classes into RDF. Furthermore it provides concepts like profiles and repositories which allow you to describe, organize and store your data, metadata and information models.

(the docs are undergoing a rewrite, so some things might not make sense right now)

Mash ORM

Mash features a simple RDF ORM which allows you to extract data from RDF graphs into Pydantic BaseModel classes, and to serialize BaseModel classes into RDF graphs. Using the SemanticModel class and Predicate annotation you can define your own rich metadata classes - either based on existing standards or your own models.

A basic example

from typing import Annotated, ClassVar
from mash import models, serialize, Predicate as P


class Person(models.SemanticModel):
    _type: ClassVar = "foaf:Person"
    name: Annotated[str, P("foaf:name")]
    mail: Annotated[str | None, P("foaf:mbox")] = None
    friends: Annotated[list["Person"], P("mash:friends")] = []


christopher_robin = Person(identifier="cr", name="Christopher Robin")
winnie = Person(identifier="wp", name="Winnie The Pooh", mail="Winnie@hundredacres.com", friends=[christopher_robin])
turtle_text = serialize(winnie)
print(turtle_text)

<urn:uuid:cr>    a foaf:Person ; 
    foaf:name "Christopher Robin" .

<urn:uuid:wp>    a foaf:Person ; 
    mash:friends <urn:uuid:cr> ;
    foaf:mbox "Winnie@hundredacres.com" ;
    foaf:name "Winnie The Pooh" .

Making data easier to parse with ReverseRelations

Normally in RDF some relations might go in directions that are not pracitical. If you want everything attached to one class rather than to point in all directions from different classes, you can achieve this with ReverseRelations.

Notice how in the previous example Substation has no references to VoltageLevel or SubstationPart since those actually contain the reference to Substation.

from typing import Annotated, ClassVar

import mash

from mash import Predicate as P, ReverseRelation
from mash.grid import get_latest_nemo
from tests.grid.profiles.nemo import IdentifiedObject, VoltageLevel, SubstationPart, NemoProfile


class SubstationEmbed(IdentifiedObject):
    _type: ClassVar = "cim:Substation"
    substation_kind: Annotated[list[str], P("elb:Substation.substationKind")]
    substation_parts: Annotated[list[SubstationPart], ReverseRelation("elb:SubstationPart.Substation")]
    voltage_levels: Annotated[list[VoltageLevel], ReverseRelation("cim:VoltageLevel.Substation")]


latest_nemo = get_latest_nemo()
m = mash.model_from_dataset(latest_nemo)
with NemoProfile:
    substations = mash.instantiate(m, model=SubstationEmbed)
print(substations[0:2])

Mash profiles

Profiles group a set of models, namespaces and static resources alongside relevant metadata using prof - The Profiles Ontology. This makes it very convenient to store and retrieve example data, validation shapes, vocabularies and more in a structured fashion. Mash then allows you to activate everything associated with a profile whenever it is relevant, which avoids collisions between models and namespaces and allows you to think about only what you need.

Using a profile

latest_nemo = get_latest_nemo()
m = mash.model_from_dataset(latest_nemo)
with NemoProfile:
    substations = mash.instantiate_all(m, model=SubstationEmbed)

Making a profile

Mash repositories

Mash provides the ability to store data alongside rich metadata inside of repositories. The rich metadata models are based on widely-implemented standard vocabularies like dcat, prof and prov-o, and mash can both read and write data following these vocabularies.

This brings with it organization, searchability, traceability and many other useful tings. Using functions like mash.query you can search the datasets based on pre-configured filters or ones you make yourself. By using the ORM you can make your own metadata models to use alongsiede Mash's preconfigured ones.

Define new data using datasets and distributions

import mash

from mash import models
from mash.core.profiles import MetadataProfile
from mash.core.utilities.organizations import make_organization


dist_id = "http://example.com/example#metadata-plaintext"
dataset = models.Dataset(
    identifier="http://example.com/example#metadata",
    title="Example metadata",
    description="Example dataset containing metadata with little",
    conforms_to=[MetadataProfile],  # This is a lie
    information_owner=make_organization("ElBits AS", "931264079"),
    distributions=[models.Distribution(identifier=dist_id, mime_type="text/plain")],
)

mash.save(dataset)
data = b"this is some text"
mash.write_data(data, dist_id)


print(mash.read_data(dist_id).decode())

Write a SPARQL query towards a graph using maplib

import mash

# This dataset would already have been created like above, and would contain a RDF distribution
rdf_dataset = mash.fetch("urn:uuid:5393e12b-96c3-4fab-8b98-d4ba2e940b94")

# model_from_dataset is a utility method that finds RDF-comliant distributions in a dataset
# loading it from a connected repository into a maplib Model.
m = mash.model_from_dataset(rdf_dataset)
res = m.query("""
    PREFIX cim: <http://iec.ch/TC57/CIM100#>
    SELECT (COUNT(?s) as ?substation_count) WHERE {
        ?s a cim:Substation
    }
""")
print(res)

Visualization and exploration

Data catalog

Mash features an integrated data catalog that allows you to browse all datasets in all connected repositories, to open it simply run: mash.open_catalog() - this should open a browser window with the location http://localhost:8234/catalog/.

Profile documentation

It can sometimes be nice with a visual overview of a profile - especially if you are not familiar with it. Mash can automatically generate documentation for any MashProfile and open it in your browser using mash.explore_profile(). Test it with the below code:

import mash
from mash.core.profiles import MetadataProfile

mash.explain_profile(MetadataProfile)