ftmq-search
Search stores logic for FollowTheMoney data.
The aim is to experiment around with different full-text search backends for efficient shallow search of entities.
Currently supported backends:
- Sqlite FTS5
- Elasticsearch
- Tantivy (persistent or in-memory)
Install
Python 3.11 or later.
pip install ftmq-search
Generate search documents
ftmqs transform -i entities.ftm.json > entities.transformed.json
Speed it up via GNU Parallel
cat entities.ftm.json | parallel -j8 --pipe --roundrobin ftmqs transform > entities.transformed.json
Index transformed documents
Sqlite FTS
ftmqs --uri sqlite:///ftmqs.store index -i entities.transformed.json
Elasticsearch
ftmqs --uri http://localhost:9200 index -i entities.transformed.json
ES can be parallelized:
cat entities.transformed.json | parallel -j8 --pipe --roundrobin ftmqs --uri http://localhost:9200 index
Tantivy
ftmqs --uri tantivy://tantivy.db index -i entities.transformed.json
Search
ftmqs search <query>
Autocomplete
ftmqs autocomplete <query>
Python
from ftmq import Query, smart_stream_proxies
from ftmqs import get_store
from ftmqs.logic import index_proxies
# elasticsearch
store = get_store("http://localhost:9200")
# sqlite
store = get_store("sqlite:///ftmqs.db")
# tantivy
store = get_store("tantivy://tantivy.db")
# tantivy in-memory
store = get_store("memory://")
# index entity data
proxies = smart_stream_proxies("./entities.ftm.json")
index_proxies(proxies, store)
# search
store.search("jane doe")
# filter for country and schema
q = Query().where(country="de", schema="Person")
store.search("jane doe", q)
Metadata
Release files for ftmq-search 0.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ftmq_search-0.0.4.tar.gz | 22.2 kB | Without provenance | Details for ftmq_search-0.0.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ftmq_search-0.0.4-py3-none-any.whl | Python 3 | none | any | Details for ftmq_search-0.0.4-py3-none-any.whl |
Total release size: 48.7 kB
Release files / ftmq_search-0.0.4.tar.gz
| Download URL | ftmq_search-0.0.4.tar.gz |
|---|---|
| Size | 22.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f6e94d9bf3925d7869226a1e5418154ca33b5eafcb49aabc73bc8ffb9d640a55
|
|
BLAKE2b-256 checksum How to use checksums |
85d2cfbc51c51bda061c2fb6e4f4d346f8f7b44f40b2768ce631386934083a75
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.1.2 CPython/3.13.3 Linux/6.12.22-amd64
|
Release files / ftmq_search-0.0.4-py3-none-any.whl
| Download URL | ftmq_search-0.0.4-py3-none-any.whl |
|---|---|
| Size | 26.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
911c40edbd11e0203d143e2ae337bd20f54a105f1acff19b309ad94be05bf525
|
|
BLAKE2b-256 checksum How to use checksums |
38f8a698e86a2690f6b365c6b0739a0e63484b5e7b562bb33879ccd1c720251a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.1.2 CPython/3.13.3 Linux/6.12.22-amd64
|
Security for ftmq-search 0.0.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.