topic-benchmark
Command Line Interface for benchmarking topic models.
The package contains catalogue registries for all models, datasets and metrics for model evaluation,
along with scripts for producing tables and figures for the S3 paper.
Usage
Installation
You can install the package from PyPI.
pip install topic-benchmark
Commands
run
Run the benchmark. Defaults to running all models with the benchmark used in Kardos et al. (2024).
python3 -m topic_benchmark run
| Argument | Short Flag | Description | Type | Default |
|---|---|---|---|---|
--out_dir OUT_DIR |
-o |
Output directory for the results. | str |
results/ |
--encoders ENCODERS |
-e |
Which encoders should be used for conducting runs? | str |
None |
--models MODELS |
-m |
What subsection of models should the benchmark be run on. | Optional[list[str], NoneType] |
None |
--datasets DATASETS |
-d |
What datasets should the models be evaluated on. | Optional[list[str], NoneType] |
None |
--metrics METRICS |
-t |
What metrics should the models be evaluated on. | Optional[list[str], NoneType] |
None |
--seeds SEEDS |
-s |
What seeds should the models be evaluated on. | Optional[list[int], NoneType] |
None |
Push to hub
Push results to a HuggingFace repository.
python3 -m topic_benchmark push_to_hub "your_user/your_repo"
| Argument | Description | Type | Default |
|---|---|---|---|
hf_repo |
HuggingFace repository to push results to. | str |
N/A |
results_folder |
Folder containing results for all embedding models. | str |
results/ |
Reproducing $S^3$ paper results
Result files to all runs in the $S^3$ publication can be found in the results/ folder in the repository.
To reproduce the results reported in our paper, please do the following.
First, install this package by running the following command:
pip install topic-benchmark
python3 -m topic-benchmark run -o results/
The results for each embedding model will be found in the results folder (unless a value for --out_file is explicitly passed).
To produce figures and tables in the paper, you can use the scripts in the scripts/s3_paper/ folder.
1 maintainer
Metadata
1 maintainer
Release files for topic-benchmark 0.6.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| topic_benchmark-0.6.0.tar.gz | 16.0 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| topic_benchmark-0.6.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 39.0 kB
Release files / topic_benchmark-0.6.0.tar.gz
| Download URL | topic_benchmark-0.6.0.tar.gz |
|---|---|
| Size | 16.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
73e3b58f4b8925cfb279a92f1fd3b5cd7a3e150be558a434ec4c510f3de6adde
|
|
BLAKE2b-256 checksum How to use checksums |
1aaac306464c4660319e0004e8679b11b53d4a0f62dbcbfda41f070cc954b55c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.7.1 CPython/3.11.5 Linux/5.15.0-124-generic
|
Release files / topic_benchmark-0.6.0-py3-none-any.whl
| Download URL | topic_benchmark-0.6.0-py3-none-any.whl |
|---|---|
| Size | 23.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
dfc7884c3148b58c91e125ba021ca13abc83116bf07042c4dc22634fd1d99bc6
|
|
BLAKE2b-256 checksum How to use checksums |
fdf0194187c269f17dc34552e76ca1c7621ded94bfa948c20c43ea9118f8a6f6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.7.1 CPython/3.11.5 Linux/5.15.0-124-generic
|
Security for topic-benchmark 0.6.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
