CIM query utilities
Project description
CIMSPARQL Query CIM data using sparql
This Python package provides functionality for reading/parsing cim data from either xml files or GraphDB into Python memory as pandas dataframes.
The package provides a set of predefined functions/queries to load CIM data such generator or branch data, though the user can easiliy extend or define their own queries.
Usage
Load data using predefined functions/queries
>>> from cimsparql.graphdb import GraphDBClient
>>> from cimsparql.url import service
>>> gdbc = GraphDBClient(service(repo='<repo>', server=127.0.0.1:7200))
>>> ac_lines = gdbc.ac_lines(limit=3)
>>> print(ac_lines[['name', 'x', 'r', 'bch']])
name x r bch
0 <branch 1> 1.9900 0.8800 0.000010
1 <branch 2> 1.9900 0.8800 0.000010
2 <branch 3> 0.3514 0.1733 0.000198
In the example above the client will query repo "" in the default server GraphDB for AC line values.
Inspect/view predefined queries
To see the actual sparql use the dry_run
option:
>>> from cimsparql.queries import ac_line_query
>>> print(ac_line_query(limit=3, dry_run=True))
The resulting string contains all the prefix's available in the Graphdb repo making it easier to copy and past to graphdb. Note that the prefixes are not required in the user specified quires described below.
The dry_run
option is available for all the predefined queries.
Load data using user specified queries
>>> query = 'SELECT ?mrid where { ?mrid rdf:type cim:ACLineSegment } limit 2'
>>> query_result = gdbc.get_table(query)
>>> print(query_result)
List of available repos at the server
>>> from cimsparql.url import GraphDbConfig
>>> print(GraphDbConfig().repos)
Prefix and namespace
Available namespace for current graphdb client (gdbc
in the examples above),
which can be used in queries (such as rdf
and cim
) can by found by
>>> print(gdbc.ns)
{'wgs': 'http://www.w3.org/2003/01/geo/wgs84_pos#',
'rdf': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#',
'owl': 'http://www.w3.org/2002/07/owl#',
'cim': 'http://iec.ch/TC57/2010/CIM-schema-cim15#',
'gn': 'http://www.geonames.org/ontology#',
'xsd': 'http://www.w3.org/2001/XMLSchema#',
'rdfs': 'http://www.w3.org/2000/01/rdf-schema#',
'SN': 'http://www.statnett.no/CIM-schema-cim15-extension#',
'ALG': 'http://www.alstom.com/grid/CIM-schema-cim15-extension#'}
Running Tests Against Docker Databases
Tests can be run agains RDF4J and/or BlazeGraph databases if a container with the correct images are available.
docker pull eclipse/rdf4j-workbench
docker pull openkbs/blazegraph
Launch one or both containers and specify the following environment variables
RDF4J_URL = "localhost:8080/rdf4j-server"
BLAZEGRAPH_URL = "localhost:9999/blazegraph/namespace
Note 1: The port numbers may differ depending on your local Docker configurations. Note 2: You don't have to install RDF4J or BlazeGraph. Tests requiring these will be skipped in case they are not available. They will in any case be run in the CI pipeline on GitHub (where both always are available).
Data Assumptions
CimSPARQL makes certain assumptions about the data which is required to be present for the queries to work. The script modify_xml
should be able to modify
the XML files such that they are compliant with CimSPARQL.
- There is a valid
xml:base
attribute in the top-levelrdf:RDF
element. This is required for uploading files (at least for RDF4J which is used in the CI pipeline) - All items
cimsparql.constants.CIM_TYPES_WITH_MRID
hascim:IdentifiedObject:mRID
cim:Terminal.endNumber
is of typexsd:integer
poetry run python scripts/modify.xml -h
usage: Program that modifies XML files to be compatible with cimsparql [-h] [--baseURI BASEURI] [--suffix SUFFIX] file
positional arguments:
file File or glob pattern for files to modify
optional arguments:
-h, --help show this help message and exit
--baseURI BASEURI Base URI to insert in all XML files. For example: http://iec.ch/TC57/2013/CIM-schema-cim16
--suffix SUFFIX Suffix to the filename after modifying them. If given as an empty string the original files will be overwritten. Default 'mod'
In order to use the script to convert XML files into a format that can be used with cimsparql
poetry run scripts/modify_xml.py "path/to/model/*.xml"
Ontology (for developers)
Ontologies for the CIM model can be found at (ENTSOE's webpages)[https://www.entsoe.eu/digital/common-information-model/cim-for-grid-models-exchange/].
For convenience and testing purposes the ontology are located under tests/data/ontology
. CIM models used for testing purposes in Cimsparql should
be stored in N-quads format. In case you have a model in XML format it can be converted to N-quads by launching a DB (for example RDF4J) and upload
all the XML files and the ontology.
Execute
PREFIX cims: <http://iec.ch/TC57/1999/rdf-schema-extensions-19990926#>
DELETE {?s ?p ?o}
INSERT {?s ?p ?o_cast} WHERE {
?s ?p ?o .
?p cims:dataType ?_dtype .
?_dtype cims:stereotype ?stereotype .
BIND(IF(?stereotype = "Primitive",
URI(concat("http://www.w3.org/2001/XMLSchema#", lcase(strafter(str(?_dtype), "#")))),
?_dtype) as ?dtype)
BIND(STRDT(?o, ?dtype) as ?o_cast)
}
and export as N-quads.
Note: Make sure the base URI is either specified in the XML-files or when you upload. It should be set to
<rdf:RDF xml:base="http://iec.ch/TC57/2013/CIM-schema-cim16">
Test models
- micro_t1_nl:
MicroGrid/Type1_T1/CGMES_v2.4.15_MicroGridTestConfiguration_T1_NL_Complete_v2
Rest APIs
CimSparql mainly uses SparqlWrapper
to communicate with the databases. However, there are certain operations which are performed
directly via REST calls. Since there are small differences between different APIs you may have to specify which API you are using.
This can be done when initializing the ServiceCfg
class or by specifying the SPARQL_REST_API
environment variable. Currently,
RDF4J
and blazegraph
is supported (if not given RDF4J
is default).
export SPARQL_REST_API=RDF4J # To use RDF4J
export SPARQL_REST_API=BLAZEGRAPH # To use BlazeGraph
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for cimsparql-1.12.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fddafd92c9f11f40398aa805f423289edbb6bc7afedee2db7f80914012e10a9e |
|
MD5 | de722da4ff08211e235b328606ad0a20 |
|
BLAKE2b-256 | 9ae0cdfd2448fff652a9e7159578663793d81bdb092e4ed95cb01dbc51171d21 |