Joyous Brands

Artificial Intelligence: A Visual Learner's Series

Part 5: Decisions

Author: Christopher DanosAI by Claude

Knowing what's probably true is half the job. This part turns beliefs into choices.

Expected utility

In one line: Score every outcome, then pick the action with the best probability-weighted average score.

Take the umbrella: it scores better on average, even if rain is unlikelyUmbrella?Take it: EU 7.7Leave it: EU 7.0Rain, carrying it: 7Dry, carrying it: 8Rain, soaked: 0Dry, hands free: 1030%70%30%70%Take: 0.3 × 7 + 0.7 × 8 = 7.7 Leave: 0.3 × 0 + 0.7 × 10 = 7.0

  • Read right to left: average each action's outcomes by their chances, then pick the higher average.

  • Leaving the umbrella has the single best outcome (10) but also the worst (0).

Watch out: Utilities aren't dollars; the tenth thousand matters less to most people than the first.

The value of information

In one line: Information is worth paying for only if it could change what you'd do.

SituationCould the inspection change your choice?Worth $150?
Used car, might have a $3,000 problem, other cars availableYes, you'd walk awayYes
Same car, you must buy it today regardlessNoNo

Key term: value of perfect information (VPI). It's never negative for an agent that uses information rationally.

Plans for every situation: MDPs

In one line: When today's move changes tomorrow's options, you need a policy, a rule for what to do in every square, chosen for the best total reward.

The best policy steers around the shaft, even when the route is longer+10−10wall+10: loading dock−10: elevator shaftArrow: best movefrom each squareMoves slip sideways 10% ofthe time, so the square underthe shaft steers away from it

  • The robot can't follow one fixed route because slips can land it anywhere, so it gets an arrow for every square.

  • The Bellman equation says a square's value is the best move's immediate reward plus the discounted value of where you land:

Key terms: states, actions, transition model, reward, discount factor γ, policy; solved by value iteration or policy iteration.

Watch out: Make every step very costly and the robot will risk the shaft just to finish sooner.

Deciding when you can't see the state

In one line: When the true situation is hidden, act on your belief about it, including actions that only gather clues.

  Car rattles. Belief:   loose heat shield  ████████░░  80%
                         failing bearing    ██░░░░░░░░  20%
  Action: tap the shield and listen  →  belief sharpens  →  then decide

Key terms: POMDP, observation model, belief state.

Watch out: A 20% chance of something dangerous can matter more than an 80% chance of something harmless.

Deciding alongside other agents

In one line: When others' choices affect your outcome, stable outcomes are those where nobody gains by changing course alone.

Profits (Shop A, Shop B)B charges $5B charges $4
A charges $5Both do well (8, 8)A loses customers (2, 10)
A charges $4A steals customers (10, 2)Both squeezed (4, 4)

  • Whatever the other shop does, $4 pays more, so both end up at $4: a Nash equilibrium, worse for both than $5.

  • Mechanism design sets the rules (like an auction format) so self-interest produces good results.

Watch out: Real people care about fairness and reputation, so predictions can miss.

Self-test

  1. Redraw the umbrella tree with a 10% chance of rain. Which action wins now?

  2. Sketch a 3 × 3 grid with a goal and a trap, and draw the arrows you'd expect.

  3. Fill in a 2 × 2 payoff grid for two roommates deciding whether to clean the kitchen.