Search service for the Software Heritage archive.
It is similar to swh-storage in what it contains, but provides different ways to query it: while swh-storage is mostly a key-value store that returns an object from a primary key, swh-search is focused on reverse indices, to allow finding objects that match some criteria; for example full-text search.
Currently uses ElasticSearch, and provides only origin search (by URL and metadata).
Dependencies
Python tests for this module include tests that cannot be run without a local ElasticSearch instance, so you need the ElasticSearch server executable on your machine (no need to have a running ElasticSearch server).
Debian-like host
The elasticsearch package is required. As it’s not part of debian-stable, another debian repository is required to be configured
Non Debian-like host
The tests expect:
/usr/share/elasticsearch/jdk/bin/java to exist.
org.elasticsearch.bootstrap.Elasticsearch to be in java’s classpath.
Emscripten is required for generating tree-sitter WASM module. The following commands need to be executed for the setup:
cd /opt && git clone https://github.com/emscripten-core/emsdk.git && cd emsdk && \ ./emsdk install latest && ./emsdk activate latest PATH="${PATH}:/opt/emsdk/upstream/emscripten"Note: If emsdk isn’t found in the PATH, the tree-sitter cli automatically pulls emscripten/emsdk image from docker hub when make ts-build-wasm or make ts-build is used.
Make targets
Below is the list of available make targets that can be executed from the root directory of swh-search in order to build and/or execute the swh-search under various configurations:
ts-install: Install node_modules and emscripten SDK required for TreeSitter
ts-generate: Generate parser files(C and JSON) from the grammar
ts-repl: Starts a web based playground for the TreeSitter grammar. It’s the recommended way for developing TreeSitter grammar.
ts-dev: Parse the query_language/sample_query and print the corresponding syntax expression along with the start and end positions of all the nodes.
ts-dev sanitize=1: Same as ts-dev but without start and end position of the nodes. This format is expected by TreeSitter’s native test command. sanitize=1 cleans the output of ts-dev using sed to achieve the desired format.
ts-test: executes TreeSitter’s native tests
ts-build-so: Generates swh_ql.so file from the previously generated parser using py-tree-sitter
ts-build-so: Generates swh_ql.wasm file from the previously generated parser using emscripten
ts-build: Executes both ts-build-so and ts-build-so
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file swh_search-0.25.1.tar.gz.
File metadata
- Download URL: swh_search-0.25.1.tar.gz
- Upload date:
- Size: 78.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.11.15
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d00f868d89e9430bb7e225d11a86d956eb2c5f6767575939f5748412aeeaf38a
|
|
| MD5 |
f90392729e66abe75626930ebae6623e
|
|
| BLAKE2b-256 |
c18292de85d513ecdbf4944a942c46c9f552d29e03d85e074d4cddb81aa31945
|
File details
Details for the file swh_search-0.25.1-cp311-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.
File metadata
- Download URL: swh_search-0.25.1-cp311-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
- Upload date:
- Size: 97.6 kB
- Tags: CPython 3.11+, manylinux: glibc 2.28+ x86-64, manylinux: glibc 2.5+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.11.15
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e862a4928c4d1ed55f067bcfcf9d104b1133ca1dae7a263532c013554b6a770a
|
|
| MD5 |
49b16dd117481b487bea1606617afc88
|
|
| BLAKE2b-256 |
f41d25cf267d2a92505be46d56acef68c46d4148cbed8b99955d5a8e9151ebd4
|
