Please enable JavaScript.
Coggle requires JavaScript to display documents.
Uncertainty: Probability and Markov chains - Coggle Diagram
Uncertainty: Probability and Markov chains
Uncertainty in AI
Problem: Too much logic makes planning difficult
Example: Planning for a flight – uncertain variables (traffic, delays, etc.)
Agents track belief states (all possible world states)
Rational Decision-Making
Involves preferences and likelihoods
Agents weigh:
Importance of goals
Probability of success
Diagnosis & Judgment
Common in fields with uncertainty:
Law, business, medicine, etc.
Logic can fail due to:
Theoretical ignorance (gaps in knowledge)
Practical ignorance (lack of data/tests)
Laziness (incomplete info)
Probability Theory
Summarizes uncertainty from laziness and ignorance
Deals with degrees of belief (0 to 1)
Not truth, but estimation based on current knowledge
Utility Theory: assigns usefulness to outcomes
Decision Theory
Combines:
Probability theory
Utility theory
Rational agents choose actions with highest expected utility
Basic Probability Concepts
Sample space = all possible outcomes
Example: Two dice → outcomes and their probabilities
Markov Chains
Memoryless: Next state depends only on current state
Transition Matrix: Rows = current states, Columns = probabilities of next state
Like Finite State Machines but probabilistic
Real-world examples:
Weather
Stock market
Behavior modeling (kitten, blood pressure)
Markov Chain Monte Carlo (MCMC)
Example: Monopoly board strategy
Used in
Physics
Game simulations
Prediction models
Run simulations to estimate probable outcomes
Stochastic Processes
System evolution over time with uncertainty
Used in real-world statistical models
Series of random variables
Hidden Markov Models (HMM)
Markov models where states are hidden (not directly observable)
Applications
Speech recognition
DNA sequencing
Google PageRank
Virology & bioinformatics
Absorbing Markov Chains
Have at least one absorbing state
Example: Games of chance (eventually reaching a final outcome
Chaos Theory
Small changes → Big effects (eg: butterfly effect)
Examples
Weather forecasting
Drug testing models
Deterministic but sensitive to initial conditions
References
https://www.geeksforgeeks.org/probabilistic-reasoning-in-artificial-intelligence/
https://www.geeksforgeeks.org/markov-chain/
https://setosa.io/ev/markov-chains/
https://www.geeksforgeeks.org/hidden-markov-model-in-machine-learning/