Scoring Public AGI Timeline Forecasts, 2015–2022

Abstract

We collect public AGI-related timeline predictions made between 2015 and 2022, operationalise their resolution criteria, and score them. Capability predictions resolved earlier than forecast; deployment predictions resolved later. Stated confidence intervals were too narrow across nearly all forecasters, and forecaster prominence showed no relationship to accuracy.

Key findings

• 01Capability predictions systematically early
• 02Deployment predictions systematically late
• 03Confidence intervals too narrow across nearly all forecasters
• 04No relationship between forecaster prominence and accuracy

Limitations

Published at equal prominence to the findings. A paper’s limitations are usually the part that determines how much its result should move your beliefs.

• Operationalising vague predictions required judgement
• Selection bias toward predictions that were recorded publicly
• Small sample per forecaster