Skip to main content

Simple access to the TIRA API.

Project description

The TIRA Client

This is a python client for TIRA.io.

Setup REST Client to Access Non-Public Endpoints

To access non-public endpoints, you will need an authentication via an API key to ensure that you have the correct access credentials. Please generate your API key online at tira.io/admin/api/keys and create a credentials file at ~/.tira/.tira-settings.json with the following content:

{"api_key": "<YOUR-API-KEY>"}

Download The results of some Submission

You can download runs of published and unblinded submissions via:

from tira.rest_api_client import Client

tira = Client()
output = tira.get_run_output('<task>/<team>/<approach>', '<dataset>')

As an example, you can download all baseline BM25 runs submitted to TIREx via:

from tira.rest_api_client import Client
from tira.tirex import TIREX_DATASETS

tira = Client()

for dataset in TIREX_DATASETS:
    output = tira.get_run_output('ir-benchmarks/tira-ir-starter/BM25 Re-Rank (tira-ir-starter-pyterrier)', dataset)

Overview of public submissions

As an example, you can see all public software submissions submitted to TIREx via:

from tira.rest_api_client import Client

tira = Client()
submissions = tira.all_softwares("ir-benchmarks")

Export datasets

You can export datasets if you are the owner or if the dataset is public. Export a dataset via the cli:

tira-run --export-dataset '<task>/<tira-dataset>' --output-directory tira-dataset

Export a dataset via the python API:

from tira.rest_api_client import Client

tira = Client()
tira.download_dataset('<task>', '<tira-dataset>')

Running Jupyter Notebooks with TIRA

docker build -t tira/submission-base-image:1.0.0 -f Dockerfile .

Testing the model locally can be done using the following command:

tira-run \
  --input-directory ${PWD}/input \
  --output-directory ${PWD}/output \
  --image tira/submission-base-image:1.0.0 \
  --command 'tira-run-notebook --input $inputDataset --output $outputDir /workspace/template-notebook.ipynb'

Afterwards you can push the image to TIRA

docker push tira/submission-base-image:1.0.0

and set the command:

tira-run-notebook --input $inputDataset --output $outputDir /workspace/template-notebook.ipynb

Finally, if the actual processing in notebook is toggled via is_running_as_inference_server() (as seen in the template notebook) and your notebook defines a function named predict in the format

def predict(input_list: List) -> List:

you can start an inference server for your model with:

PORT=8001

docker run --rm -it --init \
  -v "$PWD/logs:/workspace/logs" \
  -p $PORT:$PORT \
  tira/submission-base-image:1.0.0 \
  tira-run-inference-server --notebook /workspace/template-notebook.ipynb --port $PORT

Exemplary request for a server running on localhost:8001 are

# POST (JSON list as payload)
curl -X POST -H "application/json" \
  -d "[\"element 1\", \"element 2\", \"element 3\"]" \
  localhost:8001

and

# GET (JSON object string(s) passed to the 'payload' parameter)
curl "localhost:8001?payload=\"element+1\"&payload=\"element+2\"&payload=\"element+3\""

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tira-0.0.101.tar.gz (37.1 kB view details)

Uploaded Source

Built Distribution

tira-0.0.101-py3-none-any.whl (46.3 kB view details)

Uploaded Python 3

File details

Details for the file tira-0.0.101.tar.gz.

File metadata

  • Download URL: tira-0.0.101.tar.gz
  • Upload date:
  • Size: 37.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for tira-0.0.101.tar.gz
Algorithm Hash digest
SHA256 5bdd8e690e4f08f4376ce6a6ca6706fbb78a05ab626fc2d84846dec711e41b82
MD5 256efcfc5acc560bf97967180e20b210
BLAKE2b-256 e4057a791d0fb5030071ce64012ee3f6b5b0971245959e117a6da85f34a004fb

See more details on using hashes here.

File details

Details for the file tira-0.0.101-py3-none-any.whl.

File metadata

  • Download URL: tira-0.0.101-py3-none-any.whl
  • Upload date:
  • Size: 46.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for tira-0.0.101-py3-none-any.whl
Algorithm Hash digest
SHA256 42cc4f65b5a25b74b78fc6f8f5cf537101913d0113912340e060d9cb7a6481d0
MD5 cc0167ddf825e11ec88b50204d9a0cf5
BLAKE2b-256 1978a41469c158f96d884a5c3bdde35eef068dc06b8e32dfa76ce6577be530c0

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page