DeTrusty - Decentralized and Trustable Query Engine
Project description
DeTrusty
DeTrusty is a federated query engine. At this stage, only SPARQL endpoints are supported. DeTrusty differs from other query engines through its focus on the explainability and trustworthiness of the query result.
Notice: DeTrusty is under active development! The current version is a federated query engine following the SPARQL 1.1 protocol, i.e., you can use the SERVICE clause to specify the endpoint. However, the parts about the explainability and trustworthiness have not been implemented yet.
DeTrusty only supports SELECT
queries.
The following SPARQL 1.1 operations are not supported (yet):
- Aggregation functions, e.g.,
COUNT
,MAX
- Conditional functions, e.g.,
IF
,BOUND
,COALESCE
GROUP BY
clause (includingHAVING
)BIND
clause- Specifying RDF datasets using
FROM
,FROM NAMED
, orGRAPH
This list might not be exhaustive. Please, let us know if we missed something.
Running DeTrusty
DeTrusty can be used in two different ways: (i) as a Service by running it with Docker, and (ii) as a Python library within your own project. Both methods will be explained in the following section.
DeTrusty as a Service
In order to use DeTrusty as a service, you need to execute the following steps:
- Pull the Docker image
docker pull sdmtib/detrusty:latest
- Start the container
docker run --name DeTrusty -p 5000:5000 -v $(pwd)/Config:/DeTrusty/Config -d sdmtib/detrusty:latest
- Create the file
./Config/endpoints.txt
- Add the URLs to the SPARQL endpoints you want to include in the federation to that file; one URL per line and accessible from within the container.
- Now it is time to collect the metadata necessary for DeTrusty to operate as expected.
docker exec -it DeTrusty bash -c 'create_rdfmts.py -s /DeTrusty/Config/endpoints.txt'
- Once the metadata is collected, restart the workers in order to reload the configuration.
docker exec -it DeTrusty restart_workers.sh
- After restarting the workers, you can use DeTrusty to query your federation. When running DeTrusty as a service, you can access it through (i) an API, (ii) a command line interface, and (iii) a graphical user interface.
DeTrusty as a Service: API
You can use DeTrusty by making POST requests to its API.
If you started the container as described above, the base URL of the API is localhost:5000
.
In the following, the different API calls are described.
/version
Returns the version number of DeTrusty.
Example call:
curl -X POST localhost:5000/version
Example output:
DeTrusty v0.5.0
/sparql
This API call is used to send a query to the federation and retrieve the result. The result will be returned as a JSON (see example below).
Example call:
curl -X POST -d "query=SELECT ?s WHERE { ?s a <http://dbpedia.org/ontology/Scientist> } LIMIT 10" localhost:5000/sparql
Example output for the above query (shortened to two results):
{
"cardinality": 10,
"execution_time": 0.1437232494354248,
"output_version": "2.0",
"head": { "vars": ["s"] },
"results": {
"bindings": [
{
"__meta__": { "is_verified": True },
"s": {
"type": "uri",
"value": "http://dbpedia.org/resource/A.E._Dick_Howard"
}
},
{
"__meta__": { "is_verified": True },
"s": {
"type": "uri",
"value": "http://dbpedia.org/resource/A.F.P._Hulsewé"
}
},
]
}
}
- 'cardinality' is the number (integer) of results retrieved
- 'execution_time' (float) gives the time in seconds the query engine has spent collecting the results
- 'output_version' (string) indicates the version number of the output format, i.e., to differentiate the current output from possibly changed output in the future
- 'variables' (list) returns a list of the variables found in the query
- 'result' is a list of dictionaries containing the results of the query, using the variables as keys; metadata about the result verification is included in the key '__meta__'. The current version returns all results as verified as can be seen in the key 'is_verified' of the metadata.
When sending a SPARQL 1.1 query with the SERVICE clause, you need to set the sparql1_1
flag to True
:
curl -X POST -d "query=SELECT ?s WHERE { SERVICE <https://dbpedia.org/sparql> { ?s a <http://dbpedia.org/ontology/Scientist> }} LIMIT 10" -d "sparql1_1=True" localhost:5000/sparql
DeTrusty as a Service: CLI
You can also run DeTrusty from the command line.
The following example call assumes a query stored in a file ./query.sparql
.
docker exec -it DeTrusty bash -c 'python runDeTrusty.py -q ./query.sparql'
If you want to execute a SPARQL 1.1 query with the SERVICE clause, add -o True
.
DeTrusty as a Service: GUI
Starting with version 0.4.0, DeTrusty also comes with a Web interface.
Hence, you can run queries using your favorite browser.
The Web interface uses the JavaScript library YASGUI.
If you started the container as described above, the interface is accessible at localhost:5000/sparql
.
DeTrusty as a Library
Starting with version 0.5.0, DeTrusty can also be used as a library. You first need to install it via python3 -m pip install DeTrusty
.
The following example script shows how you can use DeTrusty within your project.
from DeTrusty.Molecule.MTCreation import create_rdfmts
from DeTrusty import run_query
endpoints = ['http://url_to_endpoint_1', 'https://url_to_endpoint_2:port/sparql']
create_rdfmts(endpoints, './Config/rdfmts.json')
query = "SELECT ?s WHERE { ?s a <http://dbpedia.org/ontology/Scientist> } LIMIT 10"
query_result = run_query(query)
print(query_result)
Of course, you can also further process the results according to your needs.
If you want to execute a SPARQL 1.1 query with the SERVICE clause, add the parameter sparql_one_dot_one=True
to your call of run_query
.
License
DeTrusty is licensed under GPL-3.0.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file DeTrusty-0.5.1.tar.gz
.
File metadata
- Download URL: DeTrusty-0.5.1.tar.gz
- Upload date:
- Size: 74.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.9.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 036a839e7afe50378b30d8aa0b7503fe584291ab3b5991828587444039dc844c |
|
MD5 | 9139bdb6fcd70f0ebabd17ff56431e64 |
|
BLAKE2b-256 | e25ad6af9b1d4329d1bfcf9a8a315892aa68f3b935ebbf20d7f445c55a6243b0 |
File details
Details for the file DeTrusty-0.5.1-py3-none-any.whl
.
File metadata
- Download URL: DeTrusty-0.5.1-py3-none-any.whl
- Upload date:
- Size: 101.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.9.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6e11e23d2130e7dbc65fd0aabbe45450b05674b39bb2898041d60985ebc764d4 |
|
MD5 | ce94a341af6b5f63a268d6831f0442cb |
|
BLAKE2b-256 | 544fc83508352b1fd98b3b4b240ee506202c68aad808e4cacf4746e5b4aeb5a6 |