Joyous Brands

Artificial Intelligence: A Visual Learner's Series

Part 7: Deep learning

Author: Christopher DanosAI by Claude

From a single artificial neuron to the architectures behind image recognition and language models.

Neurons and layers

In one line: A neural network stacks simple units that each weigh their inputs and pass on a signal; stacking layers lets it represent complex patterns.

Each layer builds on the patterns found by the one beforeEvery line is a weight learned during trainingInputamount, time, placeHidden layer 1simple patternsHidden layer 2combinationsOutputfraud / not fraud

  • Each circle computes a weighted sum, adds a bias, then applies an activation like ReLU:

  • Without the nonlinear activation, all the layers collapse into one linear model.

Watch out: "Neuron" is a metaphor; real brain cells are far more complex.

Training with backpropagation

In one line: Predict, measure the error, then trace backward to see how much each weight contributed and nudge every one to do better.

Predict forward, then send the error backward to every weightForward pass: compute a predictionInputLayer 1Layer 2OutputLoss: how wrong?Backward pass: each weight gets its share of the blame, then a small nudge

  • Repeat over many small batches (stochastic gradient descent); one pass through all the data is an epoch.

  • Residual connections, normalization and ReLU keep the error signal from fading in deep networks; dropout fights overfitting.

Watch out: Learning rate, batch size and architecture can make or break training.

Convolutional networks for images

In one line: Slide small pattern detectors across the image, building up from edges to whole objects.

  pixels ─▶ edges & color blobs ─▶ textures & curves ─▶ eyes, ears ─▶ "cat"
           layer 1               layer 2              layer 3+       output

  • The same filter is reused at every position (weight sharing), so an ear detector works anywhere in the frame.

  • Transfer learning: start from a network pretrained on millions of images.

Watch out: Networks latch onto shortcuts, like learning "snow in the background" means wolf.

Attention and transformers

In one line: Attention lets every word look directly at every other word and weigh which ones matter.

Attention lets “it” look straight at the word it refers toLine thickness shows how much weight “it” gives each word0.700.150.10Thetrophydidn'tfitinthesuitcasebecauseitwastoobig

  • Change "big" to "small" and the heavy line swings to "suitcase."

  • Each word produces a query, key and value; relevance is how well one word's query matches another's key.

  • A transformer stacks layers of self-attention and processes all words in parallel, making huge-scale training practical.

Watch out: Attention cost grows with the square of the text length.

Generative models

In one line: Generative models learn what real data looks like well enough to create new examples.

ModelCore idea
Variational autoencoderCompress to a small code, learn to decode new samples from it
GANA generator and a fake-spotter compete and improve each other
Diffusion modelLearn to turn noise into an image one denoising step at a time
Autoregressive modelGenerate one token at a time, each based on everything before (language models)

Watch out: Fluent output isn't accurate output; generative models can fabricate confidently.

Self-test

  1. Draw a network with 2 inputs, one hidden layer of 3 and 1 output. How many weights does it have?

  2. Sketch the forward and backward passes and say what flows in each direction.

  3. Write a sentence where a pronoun is ambiguous and draw the attention lines for both readings.