5

I'm working on a project related to machine Q&A, using the SQuAD dataset. I've implemented a neural-net solution for finding answers in the provided context paragraph, but the system (obviously) struggles when given questions that are unanswerable from the context. It usually produces answers that are nonsensical and of the wrong entity type.

Is there any existing research in telling whether or not a question is answerable using the info in a context paragraph? Or whether a generated answer is valid? I considered textual entailment but it doesn't seem to be exactly what I'm looking for (though maybe I'm wrong about that?)

BriJia
  • 51
  • 2

1 Answers1

1

This problem is very challenging since you need to evaluate the quality of the candidate answers.

In question answering, there are common steps that you need to follow. To summarise you need first to find the sentence that can answer the question and then compose the final answer.

In the first step you can measure the semantical similarity between Q and A, this a first filter where you can use several deep learning methods. Also, you can define a threshold to validate if the pair QA are enough related.

In the second step, you must extract the answer, if the answer is a fact you can use KB extraction, or if it is a summary, list you can use other DL methods. There is also the possibility that you must infer the answer, also suitable for DL methods.

I suggest to check this paper:

Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks

andres
  • 31
  • 1