bikiDATA
Developer-friendly Queries over RDF triples
Using this tool you can hit the ground running, and make some useful queries over RDF data using simple JSON calls.
Getting started TL;DR
You have a n-triple file called myfile.nt.
Install bikidata by saying pip install bikidata
Import it into bikidata with: python -m bikidata myfile.nt
And now, in a python prompt, you can query things, for example:
import bikidata
r = bikidata.query({
"filters": [
{"p":"fts",
"o":"something"
}
]
})
or
r = bikidata.query({
"filters": [
{ "p":"id",
"o":"<http://example.com/id/123>"
}
]
})
or
r = bikidata.query({
"filters": [
{"p":"fts",
"o":"something"
},
{ "op":"not",
"p":"<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>",
"o":"<https://swapi.co/vocabulary/Species>"
}
]
})
For more examples, see the file: examples.ipynb
Full-text search
The fts filters are answered by a Tantivy index, that is built
together with the database, in a directory next to it (bikidata.duckdb.fts, or set BIKIDATA_FTS_PATH).
The index has one document per entity, with:
- its labels (
rdfs:label,skos:prefLabel,schema:name, ...), weighted highest - all its other literals
- the labels of the IRIs it links to, weighted lowest
- the literals of the blank nodes it references, unrolled up to 3 levels deep. So a painting can be found by the name of its artist, even when the production event in between is a blank node.
All the terms of a query have to match, and the Tantivy query syntax
can be used, for example "night watch" for a phrase or rembrandt OR vermeer. The filter "p": "fts 1" returns
the entities that link to the matches, "p": "fts <iri>" only searches the literals of property <iri>.
The results are sorted by score.
To (re)build the index of an existing database, for example one that was built with an earlier version of bikidata that used the DuckDB FTS extension:
import bikidata
bikidata.build_fts(stemmer="porter") # or "german", "french", ..., or "none"
bikidata.fts_search("rembrandt", size=10) # [(entity, score), ...]
Note that the index is not updated by insert_async and delete_async, rebuild it with build_fts after changes.
SPARQL queries
Next to the JSON queries, the data can also be queried with SPARQL 1.1. The SPARQL query is translated into a single SQL query that runs on the same DuckDB tables, no extra index is needed.
import bikidata
r = bikidata.sparql("""
PREFIX schema: <http://schema.org/>
SELECT ?person ?name WHERE {
?person a schema:Person ;
schema:name ?name .
FILTER(LANG(?name) = "en")
}
ORDER BY ?name
LIMIT 10
""")
for row in r["results"]["bindings"]:
print(row["person"]["value"], row["name"]["value"])
- SELECT queries return a dict in the SPARQL 1.1 Query Results JSON Format.
- ASK queries return
{"head": {}, "boolean": True}. - CONSTRUCT and DESCRIBE queries return a list of
(s, p, o)tuples, each term encoded as in N-Triples.
Optional parameters are db_path (defaults to the BIKIDATA_DB database), base for resolving relative IRIs
and union_default_graph. Triples loaded from a .trig file are in named graphs, that you can query with GRAPH.
By default the default graph only contains the triples that are not in a named graph, pass union_default_graph=True
to use the union of all the graphs as the default graph.
To see the SQL a SPARQL query is translated into, use bikidata.sparql_engine.to_sql(query).
The implementation is tested with the W3C SPARQL 1.0 and 1.1 query test suites, see tests/test_w3c.py, where
the few tests that do not pass are listed in tests/w3c_known_failures.txt. Not supported are SERVICE (federated queries),
SPARQL Update, and entailment regimes.
Redis support
When querying non-trivial datasets of a few million triples, or handling many concurrent users, we do not want to open a new database connection for each query, and cache the results in memory.
By using Redis we can scale the number of bikidata workers horizontally, and share the cache between them.
The query_async function can now be awaited in an async context. This function places the query in a Redis queue, and awaits the result.
To use this, you need to have a Redis server running, and install the redis python package:
pip install redis
And then you need to run the bikidata worker in a separate terminal:
python -m bikidata worker
It is possible to run multiple workers, and they will share the load.
Then, in your code, you can await the query_async function in stead of the regular query function:
Metadata
Release files for bikidata 0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| bikidata-0.5.tar.gz | 52.6 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| bikidata-0.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 98.7 kB
Release files / bikidata-0.5.tar.gz
| Download URL | bikidata-0.5.tar.gz |
|---|---|
| Size | 52.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
df82f630e073872b415c73d2ceef174439f1960379c9c8a29763b8fa79805a81
|
|
BLAKE2b-256 checksum How to use checksums |
7b31c5801d2acb652ea8e52e7694516490190ebf55359de943c177601feb7838
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / bikidata-0.5-py3-none-any.whl
| Download URL | bikidata-0.5-py3-none-any.whl |
|---|---|
| Size | 46.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
7f2994507b5404b0905ca3e1d47d8b09ac55368591b4f3eada7b3400c4329a3e
|
|
BLAKE2b-256 checksum How to use checksums |
b17133d88ac7f128ba9edc252881d531fa061d7026b842dab05a69cfc503cbb1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for bikidata 0.5
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.