gerbil-api-wrapper 1.0.3

A package that provides wrapper functionality for the Gerbil Benchmark Service

GerbilExperimentApiWrapper

This API wrapper is meant to automate the testing of QA results with a standardized process using Gerbil for QA (see also Paper and GitHub repository).

initialization

The Gerbil object is initialized with a Gold standard dataset, a test dataset or alternatively a live annotator, as well as the used language.

As part of the initialization, the files are uploaded to the Gerbil service and an experiment is started. If upload of the experiment configuration fails or the received results page URL is not valid, then an exception is thrown and the api wrapper is not initialized.

Note: File names for the upload to the Gerbil API must not exceed 100 characters.

Examples

Running an expeiment with a local test results file:

from gerbil_api_wrapper.gerbil import Gerbil

wrapper = Gerbil(
    gold_standard_file="original_data/qald_9_plus_test_dbpedia.json",
    test_results_file="original_data/qald_9_plus_test_dbpedia-test.json",
    test_results_name="MyTestResults",
    gold_standard_name="MyGoldStandard",
    language="de"
)

Running an experiment with a live annotator:

from gerbil_api_wrapper.gerbil import Gerbil

wrapper = Gerbil(
    gold_standard_file="results_gold.json",
    live_annotator_name="QAnswer",
    live_annotator_url="http://qanswer-core1.univ-st-etienne.fr/api/gerbil",
    language="en")

After successful initialization the results can be accessed with the following functions:

• get_results_url() returns the URL of the Gerilb website for the started experiment.
• get_results() returns a dict containing the JSON-LD data for the started experiment.