Natural Language Processing (AI in NLP (Good progress (Sentiment analysis …
Natural Language Processing
:red_flag:Interactions between computers and human languages
:red_flag:A set of computational techniques for analyzing and representing naturally occurring texts (at one or more levels) for the purpose of achieving human-like language processing for a range of applications.
Requires understanding of
:red_flag:Interpretation of speech sounds within and across words.
:red_flag:Sound might give a big hint about the meaning of a word or a sentence - tonal languages.
:red_flag:Properties of the text as a whole that convey meaning by making connections between component sentences.
:red_flag:Understanding distinct words according to their morphemes (smallest units of meanings).
:red_flag:Understanding everything about distinct words according to their position in the speech, their meanings and their relation to other words.
Non standard English
:red_flag:Analyzing the words of a sentence to uncover the grammatical structure.
:red_flag:The possible meanings of a sentence by focusing on the interactions among word-level meanings in the sentence.
Explains how extra meaning is read into texts without actually being encoded in them.
:red_flag:Studies concepts that directly relate to being, in particular becoming, existence, reality, as well as the basic categories of being and their relations.
AI in NLP
N-gram character model
N letter sequences
Markov chain probability - next depends only on preceding
Prediction on corpus of text
Can be trained
Identify trends in public opinion
Word sense disambiguation
Summarise - xml markup
topic segmentation and recognition
word segmentation - English v Chinese and Japanese (no boundaries)
sentence breaking (beyond punctuation)
NL understanding - text into formal representations - logic structures
NL generation - convert pc database into readable human language
Involves a variety of domains:
time, a little space and physics,
naive psychology, multi-agents.
Replicate human thinking
Reason with knowledge that is true by default
Reason rapidly across a broad range of domains
Tolerate uncertainty in your knowledge