Jupyter-RDFify

Note: With Version 1.2, the package name is moved to rwth-jupyter-rdfify!
IPython Extension for semantic web technology support (Turtle, SPARQL, SHACL, etc.)

This extension is meant to be used together with Jupyter Notebooks for educational purposes. Its focus is neither performance nor scalability but instead ease-of-use.

Teaching Semantic Web Technologies with Jupyter-RDFify

Jupyter-RDFify was developed by the Chair of Information Systems at RWTH Aachen University to support interactive teaching of Semantic Web Technologies. You can find an examplary set of tutorial-like Jupyter Notebooks using Jupyter-RDFify at https://github.com/SemWebNotebooks/Notebooks/tree/main/Notebooks and further information on how to teach Semantic Web Technologies with Jupyter Notebooks, Jupyter-RDFify, the Moodle elearning system and automatic grading at https://github.com/SemWebNotebooks/Notebooks.

Installation

Install using Pip

python -m pip install rwth-jupyter-rdfify

Install from Source using Pip

python -m pip install git+https://github.com/SemWebNotebooks/Jupyter-RDFify

Dependencies

You will need to have Graphviz installed and added to your path.

If you're using Anaconda, you can install the Graphviz binaries using conda:

conda install -c conda-forge graphviz

Usage

Basic Usage

You first need to use the predefined %load_ext or %reload_ext magic to load Jupyter-RDFify. You need to do this each time your kernel is restarted. Note that the module name uses underscores, while the package name on PyPI uses hyphens:

%load_ext rwth_jupyter_rdfify

If you've installed the extension correctly, this should register the %rdf magic. This magic is special in that it is interpreted like a command line interface. If at any point you're wondering what arguments there are and what they do, do not hesitate to use the --help or -h flag.

To list all submodules:

%rdf --help

To list all arguments of the Turtle submodule:

%rdf turtle --help

Submodules

Jupyter-RDFify is split into several submodules. Select the submodule you want using %rdf <submodule>

Graph Serialization Submodules

These submodules allow you to parse and then visualize or convert graphs. The following graph serialization modules are provided by default: turtle, n3, json-ld, xml

Visualization

To parse and visualize a graph, just use the %%rdf cell magic with the right submodule. For example to visualize a graph in Turtle notation:

%%rdf turtle
@prefix : <http://example.org/> .
:JupyterRDF :is :Awesome .

If this throws an error you probably do not have Graphviz installed. If you do not want to use the visualization, just use --display none or --display table.

Conversion

To parse and convert a graph into a different format, use a combination of --display raw and --serialize <format>. Possible formats are: turtle, n3, json-ld, xml

To convert a graph in Turtle notation to JSON-LD notation:

%%rdf turtle --display raw --serialize json-ld
@prefix : <http://example.org/> .
:JupyterRDF :is :Awesome .

Labelling

After parsing a graph, you may want to give it a label. You can later use this label to reference your graph in other submodules. With this you can for instance query, validate, draw or entail your graph later on. To give your graph a label just use the --label <label> or -l <label> argument.

%%rdf turtle --label awesome_graph
@prefix : <http://example.org/> .
:JupyterRDF :is :Awesome .

The special label last will always hold the last object, even if no --label argument was supplied.

SPARQL Submodule

You can use the SPARQL submodule to query existing endpoints or to query local graphs.

Query SPARQL Endpoints

Use the --endpoint argument to query an endpoint. An example using the Wikidata endpoint:

%%rdf sparql --endpoint https://query.wikidata.org/sparql
SELECT ?item ?itemLabel 
WHERE 
{
  ?item wdt:P31 wd:Q146.
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
} LIMIT 10

Query Local Graphs

You can query labelled graphs using the --local <label> argument. Note that this overrides the endpoint argument. With --store <label>, the graph a local CONSTRUCT or DESCRIBE query returns is stored under that label, so it can be queried, drawn or validated like a parsed graph.

Example querying the above labelled graph:

%%rdf sparql --local awesome_graph
PREFIX : <http://example.org/>
SELECT ?x WHERE {
    ?x :is :Awesome
}

SHACL Validation Submodule

The SHACL validation submodule allows you to validate RDF graphs against SHACL shapes with pySHACL. After parsing both a data graph and a shapes graph with labels, you can validate them:

%rdf shaclvalidate --data-graph my_data --shapes-graph my_shapes

The data graph is validated together with its RDFS closure (pySHACL's inference="rdfs"). If the data does not conform, the report is stored as a graph with the label validation_report_<data label>.

(Earlier versions of Jupyter-RDFify had a ShEx submodule. It was removed in version 1.2; use SHACL for validation.)

Persistence Submodule

The persistence submodule allows you to load graph from both local and remote source. To download the FOAF ontology and store it with the label "test" you can use the following command:

%rdf persistence --download http://xmlns.com/foaf/spec/ --format xml --label test

It also allows to persistently store graphs to the disk. For example, with this magic line command you can store the graph "test" in Turtle format to the file "test.ttl":

%rdf persistence --save --output test.ttl --format turtle --label test

And with this command you could load it again from disk:

%rdf persistence --load test.ttl --format turtle --label test

Other Features

Prefixes

As many of the languages/formats use prefix declarations and these usually just distract from the actual task, most submodules let you outsource them. Using the --prefix flag, you can define a string which gets prepended to every cell magic of that submodule. A simple example for Turtle which defines some very frequent prefixes:

%%rdf turtle --prefix
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

Graph Manager

The graph manager submodule lets you list, draw, entail and delete labelled graphs. You just need to specify the action and usually a graph label. To draw awesome_graph:

%rdf graph draw --label awesome_graph

Entailment

Using OWL-RL, you can generate the finite closure of a graph under either RDFS semantics, OWL-RL semantics or both. This uses a brute-force approach, so it may easily take a long time or fail for large graphs. You can either entail a parsed graph directly using the --entail <regime> argument or entail a labelled graph later using the graph manager:

%rdf graph entail-<regime> --label awesome_graph

For now you can only entail graphs in-place. Possible values for <regime>: rdfs, owl, rdfs+owl

Dependencies

Note that these dependencies will be installed automatically if you use Pip.

RDFLib (6 or 7): The heart of this extension, including JSON-LD
SPARQLWrapper: Extension of RDFLib for SPARQL
OWL-RL: Library for RDFS and OWL-RL entailment
pySHACL: SHACL validation
prov (with its RDF and dot extras): PROV documents and their drawing
Graphviz python wrapper
Requests: Downloads for the persistence submodule
IPython