Please enable JavaScript.
Coggle requires JavaScript to display documents.
Probability and Markov Chains - Coggle Diagram
Probability and Markov Chains
Uncertainty
Problem-solving and logical agents
Keep track of belief state
Make contingency plans
excessive logic
Catching a flight scenario
Potential issues: traffic jam, petrol runs out,
Planning considerations: leave 2 hours early,
Rational Decision Making
Rational decisions depend on:
Relative importance of goals
Likelihood of achieving them
Typical domains: law, business, design, car repair,
Diagnosis and Uncertainty
Dental Diagnosis
Logic fails due to
Laziness
incomplete medical theories
Practical ignorance
Probability Theory
Degree of belief based on agent's knowledge
Probability Theory
Degree of belief based on agent's knowledge
Logic: TRUE/FALSE
Probability: degree of belief (0 to 1)
Differences between logic and probability
Real-World Application of Probability
Toothache with 80% probability of cavity
Real-world state: TRUE or FALSE
Probability reflects current knowledge
Rational Planning
Catching a flight
Considerations for planning: 2h, 4h, or 24h ahead
Agent preferences regarding outcomes
Decision Theory
chooses action yielding highest expected utility
Combines probability theory and utility theory
Markov Chains
Sequences with probabilistic transitions
Unlike finite state machines (FSMs) which have definite input/output causation
Examples and Applications of Markov Chains
Weather prediction
Current state influences next state
Stock market fluctuations
Blood pressure variations
Markov Chain Monte Carlo (MCMC)
Simulation technique to predict likely outcomes
Example: Monopoly game strategy
Follow game piece around the board
Count landing probabilities
Stochastic Process
Collection of random variables
Indeterminate outcomes
Evolution of system over time
Absorbing Markov Chains
resence of at least one absorbing state
Other states are transient
Example: Games of chance
Chaos Theory
Based on deterministic processes
Tiny changes can have major implications ("butterfly effect")
Example: Raisin in cookie dough
Applications of Markov Chains
Behavior of a kitten
Weather prediction
Stock market
Blood pressure variations
Further Examples
Brownian motion
Random walk