NeuralQA: A Usable Library for Question Answering (Contextual Query Expansion + BERT) on Large Datasets
release_ibb7d576l5bb5br4ev37ovbln4
by
Victor Dibia
2020
Abstract
Existing tools for Question Answering (QA) have challenges that limit their
use in practice. They can be complex to set up or integrate with existing
infrastructure, do not offer configurable interactive interfaces, and do not
cover the full set of subtasks that frequently comprise the QA pipeline (query
expansion, retrieval, reading, and explanation/sensemaking). To help address
these issues, we introduce NeuralQA - a usable library for QA on large
datasets. NeuralQA integrates well with existing infrastructure (e.g.,
ElasticSearch instances and reader models trained with the HuggingFace
Transformers API) and offers helpful defaults for QA subtasks. It introduces
and implements contextual query expansion (CQE) using a masked language model
(MLM) as well as relevant snippets (RelSnip) - a method for condensing large
documents into smaller passages that can be speedily processed by a document
reader model. Finally, it offers a flexible user interface to support workflows
for research explorations (e.g., visualization of gradient-based explanations
to support qualitative inspection of model behaviour) and large scale search
deployment. Code and documentation for NeuralQA is available as open source on
Github.
In text/plain
format
Archived Files and Locations
application/pdf 755.9 kB
file_r2ojvpsn5fgujapolh6qyqg3ve
|
arxiv.org (repository) web.archive.org (webarchive) |
2007.15211v1
access all versions, variants, and formats of this works (eg, pre-prints)