neuralqa 0.0.16a0

NeuralQA: Question Answering on Large Datasets

NeuralQA: Question Answering on Large Datasets with BERT

NeuralQA (still in alpha) provides a visual interface for end-to-end question answering (passage retrieval, query expansion, document reading, model explanation), on large datasets. Passage retrieval is implemented using ElasticSearch and Document Reading is implemented using pretrained BERT models via the Huggingface transformers api.

How Does it Work?

• Passage Retrieval
  • For each search query, scan an index (elasticsearch), retrieve matched passages
• Query Enrichment
  • Optionally apply contextual query enrichment before retrieving passages
  • Optionally construct new passages from retrieved highlights (smaller passages for BERT to read)
• Explanation
  • Provide explanations for answer queries using gradients
• Launch a user interface that allows you to perform search queries.

Usage

Create a folder you would like to use for NeuralQA. Run the following command line interface from within that folder.

pip3 install neuralqa
neuralqa ui --host localhost --port 4000

navigate to http://127.0.0.1:4000/#/.

Note: You can specify configuration for a retriever (host, port). To use NeuralQA with a retriever such as ElasticSearch, follow the instructions here to download, install, and launch a local elasticsearch instance.

Configuration [In Progress]

Neuralqa provides an interface to specify properties of each module (ui, retriever, reader, expander) via a yaml configuration file. When you launch the ui, you can specify the path to your config file --config-path. If this is not provided, NeuralQA will search for a config.yaml in the current folder or create a default copy) in the current folder. Sample configuration for the UI is shown below:

ui:
  queryview:
    intro:
      title: "NeuralQA: Question Answering on Large Datasets"
      subtitle: "Subtitle of your choice"
    views:    # select sections of the ui to hide or show
      intro: True
      advanced: True
      samples: False
      passages: True
      explanations: True
      allanswers: True
    options:  # values for advanced options
      model:  # list of models the user can select from
        title: QA models
        selected: distilbertsquad2
        options:
          - name: DistilBERT SQUAD2
            value: distilbertsquad2
          - name: BERT SQUAD2
            value: bertsquad2
      index: # search indices the user can select from
        title: Search Index
        selected: manual
        options:
          - name: Manual
            value: manual
          - name: Case Law
            value: cases 
      stride: ..
      maxpassages: ..
      highlightspan: ..

  header: # header tile for ui
    appname: NeuralQA
    appdescription: Question Answering on Large Datasets