Skip to content

PreviewAll content, scores and forecasts here are illustrative sample data — not reporting, and not measurements.What this means

Agitology
Analysis6 min readSample

How to read the Index without fooling yourself

A single number about AGI progress is either a useful summary or a false precision machine. Which one depends entirely on how you read it.

The Agitology DeskEditorial · Agitology

Any single number describing progress toward AGI is a compression of enormous disagreement into two significant figures. The compression is useful — you cannot hold twelve dimensions and their evidence in your head at once. It is also lossy in specific, predictable ways worth naming.

Four ways this number misleads

  1. It implies a scale where none exists. There is no unit of AGI progress. 63.4 does not mean 63.4% of the way there; it means a weighted position between an explicit floor and an explicit threshold, both of which are editorial choices documented in the methodology.
  2. It implies linearity. Moving from 60 to 70 is almost certainly not the same quantity of progress as 80 to 90. The remaining dimensions are the harder ones, so the last stretch is longer than it looks.
  3. It averages away the constraint. Twelve dimensions from 41 to 82 do not describe any system that exists. The minimum carries more information than the mean.
  4. It implies more measurement than there is. Four of the twelve dimensions carry low confidence. The composite does not visibly distinguish a well-evidenced 74 from a barely-evidenced 59.

What actually carries information

The dimension table, read as a spread rather than as rows. The confidence ratings, which say how much any given score should move your beliefs. The counter-evidence lists, which are the part of the methodology we would most like readers to argue with. And the change over time per dimension, which is more informative than the composite change because it shows where movement is coming from.

What we would change our mind about

The methodology commits to publishing every score revision with its reason, including revisions downward. Two things this year moved scores down: the contamination audit on generalization, and the failed continual-learning replication. Both are in the history. A framework that only moves upward is not measuring anything.

Read the composite last. It is the summary, not the argument.

Continue

More analysis

Every Tuesday

The AGI Brief

Five developments, three papers, one forecast, and what moved the Index.

Every Tuesday. No tracking, no sponsor content in the body. Unsubscribe in one click.