Joyous Brands

Artificial Intelligence: A Visual Learner's Series

Part 4: Uncertainty

Author: Christopher DanosAI by Claude

The real world rarely offers certainty. Probability lets an agent reason with partial, noisy information.

Probability as degree of belief

In one line: Probability is a number from 0 to 1 for how strongly you believe something, and it changes as your information changes.

  Before the text:  on time?  ███░░░░░░░  about 30%
  "Leaving now" (25 min away, meeting in 15):
  After the text:   on time?  ░░░░░░░░░░  near 0%

Key terms: prior P(A); conditional P(A | B); joint distribution; marginalizing (summing out other variables).

Watch out: P(A | B) is not P(B | A). Most famous people use social media; most social media users aren't famous.

Bayes' rule: from effects back to causes

In one line: Bayes' rule flips a conditional probability so you can reason from what you observe back to its likely cause.

A positive test for a rare disease is usually a false alarm100,000 people100 have it99,900 healthy99 test positive1 tests negative4,995 test positive94,905 negative0.1%99.9%99%1%5%95%Positives: 99 sick + 4,995 healthy = 5,094. Chance you're sick if positive: 99 / 5,094, under 2%

  • Follow both branches that end in a positive test. The healthy branch is so much bigger that its 5% false alarms swamp the true cases.

Key terms: prior (0.1%), likelihood (99%), posterior (under 2%).

Watch out: Ignoring the base rate is one of the most common reasoning errors, even among experts.

Bayesian networks

In one line: Most variables affect only a few others directly, so a network of small tables can stand in for one enormous table.

Five small tables replace one giant tableArrows point from direct causes to effects; each box stores its chance given only its parentsBurglarySmall earthquakeAlarm soundsNeighbor 1 callsNeighbor 2 callsstores P(Alarm | Burglary, Earthquake)

  • Once you know whether the alarm sounded, the earthquake tells you nothing more about whether a neighbor calls.

  • The full joint is just the five small tables multiplied together.

Watch out: Explaining away. Burglary and earthquake are independent, but once you hear the alarm, learning there was an earthquake makes a burglary less likely.

Answering questions: exact and approximate

In one line: Compute the answer exactly by summing over unknowns, or simulate many possible worlds and count.

MethodHow it worksTrade-off
EnumerationSum over every combination of unknownsExact; explodes on large networks
Variable eliminationSame sums, reusing partial resultsExact; much faster, still hard in worst case
Rejection samplingSimulate, discard runs that don't match evidence, countSimple; wastes runs when evidence is rare
Likelihood weightingFix the evidence, weight each runWastes less
MCMC (Gibbs sampling)Wander between possible worlds in proportion to probabilityHandles big networks; slower to settle

Watch out: Halving sampling error takes about four times as many samples.

Reasoning over time

In one line: When the world changes and you see only clues, predict forward each step, then correct with the new clue.

The weather is hidden; umbrellas are the cluesHidden state: depends only on the day beforeObservation: depends only on that day's stateWeather, day 1Weather, day 2Weather, day 3Weather, day 4Umbrellas?Umbrellas?Umbrellas?Umbrellas?

  • Filtering: today's best guess. Prediction: future days. Smoothing: revising past days with later clues. Viterbi: the single most likely sequence.

  • Kalman filters do this for continuous quantities like GPS position; particle filters use many weighted samples.

Watch out: With no new clues, predictions blur further with every step.

Self-test

  1. Redraw the frequency tree for a disease affecting 1 in 100 with the same test. What's the answer now?

  2. Draw a Bayesian network for why your car won't start (battery, fuel, starter) and the clues you can check.

  3. Sketch the hidden-and-observed rows for tracking whether a store is busy using parking lot photos.