Artificial Intelligence: A Visual Learner's Series

Part 11: Frontier models

Author: Christopher DanosAI by Claude

The most capable AI models of the moment: why scale keeps working, how they're built, what they can do, and how they're tested.

What "frontier" means

In one line: The few most capable general-purpose models at any moment, trained with more data and compute than almost anything before.

  2010s:   translate ─▶ model A     sort email ─▶ model B     summarize ─▶ model C
  Now:     translate, sort email, summarize, photos, code ─▶ one foundation model

Scale is measured in parameters, training tokens and training compute (in FLOPs).
Open-weight models publish their parameters; closed models run only on the developer's services.

Watch out: "Frontier" describes a moment. Today's frontier is ordinary within a year or two.

Scaling laws

In one line: Error falls along a straight line on log-log axes as compute grows, so small runs can forecast a big one.

The law: loss ≈ A × compute^−α. Similar laws hold for parameters and data.
Compute ≈ 6 × parameters × tokens. A common rule of thumb for a fixed budget: about 20 tokens per parameter.

Watch out: A trend isn't a law of nature, and lower loss doesn't say which new abilities will appear.

How one is built

In one line: Pretrain on vast data for knowledge, post-train for helpfulness and safety, test, then release.

Post-training methods: supervised fine-tuning, RLHF, AI feedback guided by written principles (Constitutional AI), and reinforcement learning on math and code with checkable answers.
Distillation trains a small model to imitate a big one, spreading frontier abilities to cheaper models.

Watch out: Gaps and biases in the training data carry into the model, however large it is.

Thinking longer

In one line: Spending more computation at answer time, by reasoning step by step or voting over several answers, raises accuracy on hard problems.

  Answer at once:     question ─▶ answer
  Reason first:       question ─▶ steps … check … revise ─▶ answer
  Sample and vote:    question ─▶ 25 answers ─▶ the most common one wins

Majority vote when one answer is right 40% of the time and each of four wrong answers gets 15% (from this part's Python):

Answers sampled	Correct after voting
1	39%
5	56%
25	90%
101	100%

Frontier models are also multimodal (text, images, audio), read long contexts (whole books), and use tools as agents (Part 12).
Their skills are jagged: expert on one task, tripped up by a nearby one that looks easier.

Watch out: A convincing chain of thought can still be wrong.

Testing and rules

In one line: Measure what a model can do, including the harm it could do, and add safeguards before release.

Method	What it checks	Its weak spot
Benchmarks	Standard tests: exams, code, math	Saturate, and leak into training data
Preference comparisons	Which of two answers people prefer	Rewards style as well as substance
Dangerous-capability evaluations	Help with weapons, cyberattacks, escaping oversight	Can miss abilities a test didn't probe
Red teaming	What determined experts can make it do	Only as good as the testers' ideas

Frontier safety frameworks tie stronger safeguards to capability thresholds; the EU's AI Act presumes systemic risk above 10²⁵ training FLOPs.
Open-weight release helps research and independence, but a public model can't be recalled.

Watch out: With 30% of its questions leaked, a benchmark reported 68% for a model whose real skill was about 55% (this part's Python).

Self-test

Sketch a log-log scaling chart from memory, mark the small runs, and draw the forecast for a run 100 times bigger.
Redraw the build pipeline and label which stage uses the most compute and which shapes the most behavior.
Draw the three ways to answer a question (at once, reason first, sample and vote) and note what each costs.
Make a table of four ways to evaluate a model, with one weakness each.