Search service for the Software Heritage archive.
It is similar to swh-storage in what it contains, but provides different ways to query it: while swh-storage is mostly a key-value store that returns an object from a primary key, swh-search is focused on reverse indices, to allow finding objects that match some criteria; for example full-text search.
Currently uses ElasticSearch, and provides only origin search (by URL and metadata).
Dependencies
Python tests for this module include tests that cannot be run without a local ElasticSearch instance, so you need the ElasticSearch server executable on your machine (no need to have a running ElasticSearch server).
Debian-like host
The elasticsearch package is required. As it’s not part of debian-stable, another debian repository is required to be configured
Non Debian-like host
The tests expect:
/usr/share/elasticsearch/jdk/bin/java to exist.
org.elasticsearch.bootstrap.Elasticsearch to be in java’s classpath.
Emscripten is required for generating tree-sitter WASM module. The following commands need to be executed for the setup:
cd /opt && git clone https://github.com/emscripten-core/emsdk.git && cd emsdk && \ ./emsdk install latest && ./emsdk activate latest PATH="${PATH}:/opt/emsdk/upstream/emscripten"Note: If emsdk isn’t found in the PATH, the tree-sitter cli automatically pulls emscripten/emsdk image from docker hub when make ts-build-wasm or make ts-build is used.
Make targets
Below is the list of available make targets that can be executed from the root directory of swh-search in order to build and/or execute the swh-search under various configurations:
ts-install: Install node_modules and emscripten SDK required for TreeSitter
ts-generate: Generate parser files(C and JSON) from the grammar
ts-repl: Starts a web based playground for the TreeSitter grammar. It’s the recommended way for developing TreeSitter grammar.
ts-dev: Parse the query_language/sample_query and print the corresponding syntax expression along with the start and end positions of all the nodes.
ts-dev sanitize=1: Same as ts-dev but without start and end position of the nodes. This format is expected by TreeSitter’s native test command. sanitize=1 cleans the output of ts-dev using sed to achieve the desired format.
ts-test: executes TreeSitter’s native tests
ts-build-so: Generates swh_ql.so file from the previously generated parser using py-tree-sitter
ts-build-so: Generates swh_ql.wasm file from the previously generated parser using emscripten
ts-build: Executes both ts-build-so and ts-build-so
Metadata
Release files for swh.search 0.26.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| swh_search-0.26.0.tar.gz | 78.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| swh_search-0.26.0-cp311-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl | CPython 3.11 | abi3 | Linux glibc 2.28+ x86-64, Linux glibc 2.5+ x86-64 | Details |
Total release size: 176.8 kB
Release files / swh_search-0.26.0.tar.gz
| Download URL | swh_search-0.26.0.tar.gz |
|---|---|
| Size | 78.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a318d995be54a90023fe676c2fc8a9153dee68b5726672aa31a87c15be17ab7a
|
|
BLAKE2b-256 checksum How to use checksums |
a5da247a018744a814a7c8a173e1319d504677dcf8dce29938ab7ad04107cf23
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.15
|
Release files / swh_search-0.26.0-cp311-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
| Download URL | swh_search-0.26.0-cp311-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl |
|---|---|
| Size | 97.9 kB |
| Tags | CPython 3.11 Linux glibc 2.28+ x86-64 Linux glibc 2.5+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
afe16719f12be7ef131b98ab25ba232a80126a78ab992e6a11d6dc254bebe7f6
|
|
BLAKE2b-256 checksum How to use checksums |
389b09fd9b703a49b71ea40713babb1f20c8001d320c1af4fbdbc96435b932ab
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.15
|
