Artificial intelligence, AI
What ?
definition
Lynne Parker
Artificial intelligence refers to "a broad set of methods, algorithms and technologies that make software 'smart' in a way that may seem human-like to an outside observer," said Lynne Parker
categories
Acting Humanly (ponašati se ljudski)
the turing test approach part of RA
What?
total turing test
capabilities
compoter vision
robotics
history
Alan Turing ( 1950 )
was designed to provide a satisfactory
operational definition of intelligence.
capabilities
natural language processing
knowladge representation
automated reasoning
machine learning
Thinking humanly (razmišljati ljudski)
the cognitive modeling approach
brings together
psychology =>
theory of human mind (cognitive science)
computer model from AI
Thinking rationally (razmišljati racionalno)
the "laws of thought" approach part of RA
brings together => intelligent system
logic
AI
history
Aristotel => "right thinking"
Acting rationally (ponašati se racionalno)
the rational agent(RA) approach
limitet rationality
foundation
Philosophy
Mathematics
Economics
theories
utility
Desision theory
combines probability theory with utility theory
Game theory
Neuroscience
neuron
Psychology
cognitive science
computer engineering
Control theory and cybernetics
control theory
history
Norbert Wiener
Ashby’s Design for a Brain (1948, 1952)
elaborated on his idea that intelligence could be created by the use of homeostatic devices containing appropriate feedback loops to achieve stable adaptive behavior
Linguistics
grew up hybrid field
computational linguistics ( natural language proseccing )
theory
click to edit
methods
Machine intelligence
machine learning
Cognitive computing
it is often a complete architecture of multiple A.I. subsystems that work together
definition
"This is a subset of A.I. that deals with cognitive behaviors we associate with 'thinking' as opposed to perception and motor control," Dietterich
more less => marketing trick
Projects
games
examples
deepmind
theory
super mario bro
PySc2 for StarCraft II
OpenIA
new technic of learning
Evolution Strategies, ES
scalable alternative to reinforcement learning(RL)
optimizatio technique
ML
Titanic Data science
ML from disaster => kaggle data project
machine learning
click to edit
steps
preparing the data
training a model
evaluating the model
improving the performance
What?
covering multiple technogies
methods
artificial neural networks, ANN
#
types
deep Q-network, DQN
What?
DeepMind Tehnologies
deep Q learning or deep reinforcement learning
combination of deep learning and reinforcement learning
ANIFIS
What?
combination of NN and fuzzy logic
Deep NN
What?
includes a large system of neurons arranged in several hidden layers
learning
deep learning
About
learning model
lerning methods
supervised
unsupervised
Data comes without labels. We need to find regularities in the data
density estimation ❓
(according to ACM)
banches of AI
click to edit
computer vision (rač. vid )
patern repognition (prepoznavanje uzorka)
click to edit
Understanding and processing of natural and artificial languages
natural language procesing ,NLP
components
NL understanding, NLU
NL generation, NLG
involves
text planing
sentence planning
text realization
click to edit
click to edit
tool
Reverse tuning test
fuzzy logic
about
nastaje na pojmu Fuzzy skupova (Fuzzy sets)
klasični skupovi vs fuzzy skupova
fuzzy skupovi
klasični skupovi
jasno određene granice
nisu jasno određene granice
Membership function ( funkcije pripadnosti )
svojstva
Boundary (Granice )
Height ( Visina )
Support ( Osnovica)
Crossover point ( Polovište )
Core (Jezgra)
def. kao klasičan skup svih elemenata za koje je vrijednost veća od 0
def. kao klasičan skup svih elemenata za koje je vrijednost manja od 1
def. kao klasičan skup svih elemenata za koje je vrijednost manja od 1
def. kao klasičan skup svih elemenata za koje je vrijednost maksimalna
def. kao klasičan skup svih elemenata za koje je vrijednost 0.5
type of membership func.
trokut
trapez
Gaussianova krivulja
Generalizirano zvono
koristi
koristi
fazifikacija
agregacija
Fuzzy interence system, FIS (Fuzzy sustav za zaključivanje)
model
input
je sustav za računanje koji raspolaže s više koncepata znanja, a to su :
teorija fuzzy skupova
fuzzy if-then pravila
fuzzy zaključivanje
lingvistička varijabla
output
fuzzy baza podataka
povezana sa :
dio za zaključivanje # ( Agregacija)
defazifikacija
fazifikacija
više modela :
Mamdani
Sugeno
about
pomoću dva parametra donosi zaključak
" riječ " subjektivnog karaktera
if -then pravila => pravila po kojima fuzzy logika donosi zaključke
jezičnog karaktera
sastoje :
premisa
zaključaka
prvi korak, tansformiranje crisp ulaznih parametara u fuzzy ulazne parametre
crisp ulazni parametri
numeričke vrijednosti koje se pretvaraju u odgovarajuće jezične vrijednosti , za to potreban FIU(engl. fuzzy interferance unit)
suprotan fazifikaciji,
sastoje od elemenata koji po nekom kriteriju pripadaju u taj skup
Agregacija
proces kombiniranja fuzzy rezultata pomoću pravila u kojem dobijemo konačan fuzzy rezultat koji se šalje u proces defazifikacije
svodi na logičke operacije konjukcije ili disjunkcije, odnosno T ili S norma
dobiva se koeficijent
metode
centar površine ili centar najviše sume
Center of gravity, COG (centar suma)
prvi, ili srednji ili visinski maximum
visinska defazifikacija
najčešća metoda
What ?
zadaci koje rješava
problem klasifikacija
predviđanje
vezane uz ulazne i izlazne varijable i probleme optimizacije
raspoznavanje uzoraka, obrada slika, govora, simulacija
obrada podataka nepreciznih i nekompletnih podataka
glavna sastavnica
Node, Process unit (Umjetni neuron, procesna jedinica ili čvor)
sastoji
Weight (težina)
težinske sume ili net
funkcije praga ili aktivacijske funkcije
analogna je somi ili tijelu živčane stanice
analogne dendritima
Treshold ( prag ) ili pomak (offset ili bias )
tipovi :
step func.
funkcija praga
sigmoidalna
analogna je integracijskom dijelu neurona
Umjetna neuronska mreža (engl. artificial neural networks, ANN) čini cjelinu međusobno povezanih čvorova čije veze možemo prilagođavati prema skupu podataka za učenje.Paradigma temeljena na biološkoj neuronskoj mreži ili mozgu znatno je pojednostavljena jer se nemogu programirati fenomeni koji posjeduje živčani sustav. Moć obrade podataka nalazi se u vezama između pojedinih procesnih jedinica.
varijable
ulazna ( Signali ) ili funkcijski signal
izlazna
analogija
sinapse kod neurona
about
su realni brojevi u intervalu [-1,1] ili [0,1]
označavanje
x1, x2, xn
analogija
s aksom
označava s y
daje boolov iskaz
f(x) = {
0, net < 0
1, net ≥ 0
f(net) = {
najčešći prijenos funkcije, daje neku prednost kod učenja NN
net , x < net < 0
1, net ≥ x
0, net < x
f(net ) =
net = w0x0 + w1x1 + w2x2 + ... + wnxn
uzimamo θ = - w0
karakteristike koje NN čine različite
broj slojeva procesnih jedinica
prijenosne funkcije
tip veza između ulaznih i izlaznih podataka
tip učenja
procesne jedinice u NN mogu biti spojene
u ANN procesne jedinice ( node) smještene su u slojeve
kriterij određuje tip učenja => da li je izlazna vrijednost poznata ili ne ?
dvoslojne
višeslojne
sastoje se od dva sloja
ulazni sloj
izlazni sloj
sastoje se
skriveni slojevi
izlazni sloj
ulazni sloj
može ih biti više
inter-slojna veza
intra-slojna veza
za povezivanje čvorova u razl. slojevima NN
služe za povezivanje čvorova unutar jednog sloja
primjer Hopfildova mreža
prijenos informacija kroz NN
FeedForward ( Statički )
FeedBack ( Dinamičke )
informacije su plasirane u više slojeve u jednom prolazu i najvažnije informacija se ne vraća u niže slojeve
informacije se vraćaju u niže slojeve,
informacije se mogu plasirati u dva smjera
ulazne informacije primaju u vremenskim intervalima
iz nije poznata
iz poznata
Nenadgledano (Unsupervised )
suprotno nadgledanom učenju => sustav sam otkriva odnose između podataka prema opisanim značajkama
koriste se
prepoznavanje uzoraka
Supervised (nadgledano )
tip učenja kod kojeg sustav sam podešava parametre na temelju podataka s poznatim ulaznim i izlaznim vrijednostima
tipom učenja koristi se za klastriranje ( clustering )
koliko primjera za učenje pokažemo meži za vrijeme trajanja jedne iteracije
Bactch ili off-line (grupno )
Patern-By-Patern or on-line (pojedinačno)
predočavamo sve primjere u jednoj iteraciji, zbog toga NN uči samo u jednoj fazi u kojoj se prilagode težine, a u ostalim fazama težine su fiksirane
NN uči u vremenskim intervalima, u svakom vremenskom intervalu, tj. iteraciji dodjeljen je jedan primjer za učenje NN pritom se vrše prilagodbe težinskih faktora
faze izrade NN :
- faza učenja ili treniranja
- faza testiranja
neuronsku mrežu nemožemo učiti ako nismo definirali
tip NN
model ( ulazno - izlazne varijable )
karakteristike
podjela prikupljenih podataka u podskupove #
Kako dugo trenirati neuronsku mrežu ?
proces ocjenjivanja mreže
u ovoj fazi težine su fiksirane, te ocjenjujemo kako će se mreža ponašati s novim podacima
diff learning method
Supervised
Adeline
Percepton
Multilayer percepton, MLP (Višeslojni percepton )
sastavljena od percepton čvorova ( nodes )
percepton rule ( pravilo perceptona)
ukoliko se uzorak klasificira ispravno, ne radi korekciju.
ukoliko se uzorak klasificira neispravno, primjeni LMS pravilo
Ciklički uzimaj sve uzorke redom, a postupak zaustavi kada sve uzorke klasificiraš ispravno za redom
temelji na matematičkom modelu McCulloch-Pittsa, poboljšao ga je Frank Rosenblatt
karakteristike
gl. karakteristika
raspoređenih u
više slojeva
izlazni
ulazni
potpuna povezanost njegovih čvorova
signali
vrste
funkcijski signal ili ulazni signal
izlasni signal ( Signal greške )
znači da svaki čvor u svakom sloju povezan sa svim čvorovima predhodnog sloja
signali kroz mrežu kreću s lijeva na desno ( unaprijed )
nastaje u izlasnom čvoru i širi se unatrag sloj po sloj po NN pritom izračunava po svakom čvoru neki tip funkcijske greške
nastaje u ulaznomsloju mreže i širi se dalje do izlaznog sloja te završava kao izlazni signal
skriveni
dva sloja
učenje
backpropaganation algorithm
deep learning
linear algebra
collecting data
About
numerical cumputation
numerical cumputation #
click to edit
fuzzy izlazni parametri pretvaraju se u crisp izlazne parametre
StarCraft AI
agent
what
an intelligent program that acts autonomously in an enviroment
solving several difficult problems
planning
optimization
multiagent control
scene analysis (analiza scena)
click to edit
Accociation of computing Machinery, ACM #
formalisms and methods for knowledge representation
Machine learning
deduction and theorem proving
click to edit
automated programing
Problem solving
Expert systems
Robotics
General AI
cognitive modeling
philosophical foundation
click to edit
Distributed AI (raspodjeljena UI)
Tuning test
history
Alan Tuning
Can machines think?
experiment
The imitation game
Completely Automated Public Turing Test to Tell Computers and Humans Apart, CAPTCHA
completely automated public turing test to tell computes and humans apart
About
click to edit
state space search (pretraživanje prostora stanja)
mathematic model
search problem
consists :
goal state or many goal states
initial state
transitions between states
successor function
test predicate to check if a state is a goal state
can be defined either explicitly
methods
search tree
defining the state transitions
click to edit
control methods ❓
S - set of states (state space)
(basic idea of state space search)
directed graph or digraph (usmjereni graf)
If we also need the transition costs
problems
puzzle
grap vs state space
graph node = states
arcs (directed edges) = transitions between states
may contain cycles
weighted directed graph
By searching through a digraph, we gradually construct a search tree
expanding one node after the other (pojedine čvorove proširujemo)
type of nodes
closed nodes
open nodes or front
nodes that have been generated, but have not yet been expanded
already expanded nodes
using
successor function (operator)
generate the descendants of each node
tree vs state space
Search tree is created by searching through the state space
Search tree can be infinite even if the state space is finite
NB: state space contains cycles ⇒ search tree is infinite
⇒
state vs. node
Node n is a data structure
stores
a state
some sdditional data
node data structure
d - depth of the node in the search tree
s - state
comparing problems vs. algorithms
Problem properties:
Algorithm properties:
search startegies
blind or uninformed search (cro. slijepo pretraživanje)
Heuristic (directed or informed) search (cro. usmjereno pretraživanje)
type
Breadth-first search, BFS
type
Greedy best-first search
A* algorithm
Hill-climbing search
click to edit
click to edit
click to edit
Data is labeled and comes in pairs (input, output)=(x, y). We need to find a mapping f(x) = y
if y is a discrete label(nebrojčana vrijednost)
classification
If y is a number
clustering
novelty/outlier detection
dimensionality reduction
reinforcement learning
Learning an optimal strategy based on trials with a delayed credit
AlhaGo
reinforcement learning
regression
how lear model?
click to edit
learning steps
train
test
About
feature extractor (Ekstraktor značajki)
generates the feature vector x for each data instance
during training the model is fed with data
pairs(x, y)
data instance ⇒ x
labels ⇒ y
prediction
create model
during prediction, the model is presented only data instance x as input ⇒ generates y as output
problem
overfitting ❌
If the model is too complex, it will adapt to much to the data it has been trained on, but give poor predictions on unseen data
solution #
prepare dataset for learning
cross-validation (unakrsna validacija)
split dataset
prunning of decision trees (podrezivanje stabla odluke)
typically
70% ⇒ training set
30% ⇒ test set
because of complexity
split dataset
30% ⇒ validation set
30% ⇒ test set
40% ⇒ training set
We train the model on the train set, use this model to obtain predictions on the test set, and then compute the accuracy
the accuracy
on the test set indicates how well the model generalizes (the model has previously not seen the data from the test set)
algorithms
Na ̈ıve Bayes Classifier
decision trees
click to edit
matrices
vectors
tensors (cro. tenzor)
deep learning
Information theory concepts
:
cross-entropy
decision trees on basic of maximum information gain
Viterbi algorithm
widely used in NLP and Speech
machine tlanslatation => ENCODER - DECODER
technique
clustering
What?
that involves the grouping of data points
Convolutional NN, CNN
used for
effective mechnism used for image recognition
idea
inspired by brain
visual cortex
visual neuron cells in the brain that each act in a different way
simple cells (S cells)
complex cells ( C cells)
What?
activate, for example, when they identify basic shapes as lines in a fixed area and a specific angle
continue to respond to a certain stimulus, even though its absolute position on the retina changes
model
hierarchical NN model
neocognitron
click to edit
history
first
LeNet-5
arhitecture
layers
organised in 3 dimensions
height - H
dept - D
width -W
components
1) the hidden layer/ Feature extraction part
2) the classification part
click to edit
click to edit
scalars
About
What?
About
what?
is an array of numbers, numbers are arranged in order
WHAT ?
2D array of numbers, , so each element is identified by two indices instead of just one
What?
In some cases we will need an array with more than two axes
click to edit
is just a single number
probability & information theory
probability
uncertainty
possible source of uncertainety
2.
3.
1.
Inherent stochasticity in the system being modeled
Incomplete observability
Incomplete modeling
click to edit
click to edit
click to edit
softmax function
often used to predict the probabilities associated with a multinoulli distribution
numerical computation
poor conditioning
Gradient-Based Optimization
an example of a 0th-order tensor
built-in scalar types
int
float
complex
bytes
Unicode ( in Python )
an example of 1st-order tensor
an example of 2nd-order tensors
chatbots
About
an AI software that can simulate a conversation (or chat) with a user in natural language through messaging applications, websites, mobile apps or through the telephone
there are two different tasks at the core of a chatbot
user request analysis
returning the response
2019 ...
Watson Assistant
Bold360
Rulai
LivePerson
Inbenta
Ada
Vergic