Please enable JavaScript.
Coggle requires JavaScript to display documents.
Machine Reading Comprehension(MRC) - Coggle Diagram
Machine Reading Comprehension(MRC)
1 Introduction
The MRC task
Natural Language Processing
Issue 1 The ambiguity of language
Issue 2 Common sense and reasoning skills
Deep Learning
it is used because
if model complexityincreases then efficiently the parameters are modified
powerful feature learning ability
end-to-end learning, avoids the modularity of pipelines schemes allowing for optimization to happen easily
hardware used for DL (GPU) is constantly being upgraded
The community frameworks allow for its ease of use (keras,Tensorflow,Pytorch)
Achievements
BERT 2018
Same level of accuracy as human translator reached in 2018
issues: some DL models are 'black boxes' no one knows how the input and output are related.
Evaluation of MRC
like human reading comprehension, QA and quality of answers
Answer forms
metrics
F1 score
ROUGE
BLEU
Exact Match
MRC datasets
Single paragraph datasets
RACE
CNN/DailyMail
NewsQA
SQuAD
Most InfluentialDataset
CoQA
multirun conversational competition
Multi Paragraph datasets
MS Marco
searches in bing, gets the paragraphs, answers questions 2016
DuReader
extracts full webpages and answers questions using Baidu 2017
QAngaroo
gets the answers from multiple paragraphs, wikipedia and pubmed, 2017
HotpotQA
similar to QAngaroo 2018
promotes reasonign rather than text matching
CorpusBased datasets
AKA open domain MRC because the large text input is not limited in a single topic
AI2 reasoning challenge
Answers multiple choice scientific test questions 2018
How to make your MRC dataset?
Generation of articles and datasets
3 key components: Articles, questions and anwers.
Generating questions from articles
Givena a paragraph: Labelers "artificially" create questions
Generating articles from questions
your google search history , and the result that suits you, may be used to train an MRC model.
Generation of correct answers
Labelers have subjective bias, 100% true accuracy is impossible :warning:
How to build a high quality MRC dataset :question:
high quality is to know how close a model reading ability is to that of the humans
Distinguish comprehension-based and matching-based models
as a standard 35% of unanswerable questions have semantically insignificant matches to answers
Evaluate the reasoning capability
use induction
Assess common sense
the weakest point of NLP and machine learning
Other comprehension skills
List | ennumeration
identify, summarize, and sequentially output related concepts
Mathematical operations
Logical Reasoning
Coreference reasolution
understand the pronouns to answer the question
Analogy
Spatial-Temporal Relations
Common sense reasoning
Schematic|rethorical clause relations
Special sentence structure
Causal relations
2 The basics of NLP
Tokenization
Byte-pair Encoding
usually applied when model needs to generate text
The cornerstone of NLP: word vectors
Word vectorization
One-hot-embedding
Distributed Representation
Word2Vec
Skip-gram
implementaion details
Language models
N-gram model
Evaluation of language models
just use perplexity
Linguistic Tagging
Named Entity Recognition
(to know if a word is part of a category)
Rule-based named entity recognition
Feature-based named entity recognition
NER based on deep learning
Part-Of-Speech (POS) Tagging
To know if a word is a noun,adverb, verb, etc.
Estimate probabilities in a hidden markov model (HMM)
Maximize probabilities in hidden markov model (HMM)
Named entity recognition and part-of-speech tagging in
Python
3 Deep Learning in NLP