The TIRA Client

This is a python client for TIRA.io. Please find the documentation online.

Setup REST Client to Access Non-Public Endpoints

To get your API key, please navigate to the task for which you want to submit, click on "Submit" -> "Code Submission" or "Docker Submission" and the UI will show you your access token. With this, you can login:

tira-cli login --token YOUR-TOKEN-HERE

Admin-Only: Setup REST Client to Access Non-Public Endpoints

For admin-access to non-public endpoints, you will need an authentication via an API key to ensure that you have the correct access credentials. Please generate your admin API key online at tira.io/admin/api/keys and login your tira client:

tira-cli login --token YOUR-TOKEN-HERE

Admin-Only: Batch-Execution of jobs

To execute all approaches that have a positive review on one dataset on all datasets of a task, please run:

tira-cli admin batch-execution --task TASK

Admin-Only: Dataset submission

When preparing a dataset for tira-cli dataset-submission, the dataset card in README.md must define tira_configs.default_upload_name. This is the default file name participants should upload for the dataset and the dataset submission fails if it is missing.

tira_configs:
  default_upload_name: predictions.jsonl

Download The results of some Submission

You can download runs of published and unblinded submissions via:

from tira.rest_api_client import Client

tira = Client()
output = tira.get_run_output('<task>/<team>/<approach>', '<dataset>')

As an example, you can download all baseline BM25 runs submitted to TIREx via:

from tira.rest_api_client import Client
from tira.tirex import TIREX_DATASETS

tira = Client()

for dataset in TIREX_DATASETS:
    output = tira.get_run_output('ir-benchmarks/tira-ir-starter/BM25 Re-Rank (tira-ir-starter-pyterrier)', dataset)

Overview of public submissions

As an example, you can see all public software submissions submitted to TIREx via:

from tira.rest_api_client import Client

tira = Client()
submissions = tira.all_softwares("ir-benchmarks")

Export datasets

You can export datasets if you are the owner or if the dataset is public. Export a dataset via the cli:

tira-run --export-dataset '<task>/<tira-dataset>' --output-directory tira-dataset

Export a dataset via the python API:

from tira.rest_api_client import Client

tira = Client()
tira.download_dataset('<task>', '<tira-dataset>')

Running Jupyter Notebooks with TIRA

docker build -t tira/submission-base-image:1.0.0 -f Dockerfile .

Testing the model locally can be done using the following command:

tira-run \
  --input-directory ${PWD}/input \
  --output-directory ${PWD}/output \
  --image tira/submission-base-image:1.0.0 \
  --command 'tira-run-notebook --input $inputDataset --output $outputDir /workspace/template-notebook.ipynb'

Afterwards you can push the image to TIRA

docker push tira/submission-base-image:1.0.0

and set the command:

tira-run-notebook --input $inputDataset --output $outputDir /workspace/template-notebook.ipynb

Finally, if the actual processing in notebook is toggled via is_running_as_inference_server() (as seen in the template notebook) and your notebook defines a function named predict in the format

def predict(input_list: List) -> List:

you can start an inference server for your model with:

PORT=8001

docker run --rm -it --init \
  -v "$PWD/logs:/workspace/logs" \
  -p $PORT:$PORT \
  tira/submission-base-image:1.0.0 \
  tira-run-inference-server --notebook /workspace/template-notebook.ipynb --port $PORT

Exemplary request for a server running on localhost:8001 are

# POST (JSON list as payload)
curl -X POST -H "application/json" \
  -d "[\"element 1\", \"element 2\", \"element 3\"]" \
  localhost:8001

and

# GET (JSON object string(s) passed to the 'payload' parameter)
curl "localhost:8001?payload=\"element+1\"&payload=\"element+2\"&payload=\"element+3\""

Metadata

Release files for tira 0.0.205

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tira 0.0.205
File	Size	Uploaded
tira-0.0.205.tar.gz	1.4 MB	Sep 28, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for tira 0.0.205
File	Interpreter	ABI	Platform	Reset
tira-0.0.205-py3-none-any.whl 3.7 MB Sep 28, 2026	Python 3	none	any	Details

Total release size: 5.1 MB

Release files / tira-0.0.205.tar.gz

Download URL	tira-0.0.205.tar.gz
Size	1.4 MB
Tags	Source
SHA-256 checksum How to use checksums	`f1c4295b807f0fc5d0be3f4dc7cc288122736b7163bb206c948639d98b56d5ac`
BLAKE2b-256 checksum How to use checksums	`da39727f8062af2d2eee6c9b68d98ae518508d39faf7f6455cab188f118dcfb4`
Upload date	Sep 28, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.14.7`

Release files / tira-0.0.205-py3-none-any.whl

Download URL	tira-0.0.205-py3-none-any.whl
Size	3.7 MB
Tags	Python 3
SHA-256 checksum How to use checksums	`463c6f6e9c7a2fdf36ae4f2997d6c76f99aea12301bed681ba912a6a141497d6`
BLAKE2b-256 checksum How to use checksums	`f3ae5bdd799e75051675bc05d844447b2b4d0953e4e38fafe01c9c64f873eadf`
Upload date	Sep 28, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.14.7`