[ExI] More super forecasting

[John Clark]

*Inspired by Billk's questions to the Chinese AI GLM, I sent the following
question to Anthropic's  Claude Sonnet 5 and set it to "Max":*
*"I want you to think like a super forecaster and give me a probability of
a computer achieving AGI before December 31, 2030. By AGI I do NOT mean a
machine that is capable of performing most tasks as well as the average
human most of the time, I mean a machine capable of performing all tasks
better than the best human all of the time, except perhaps for tasks that
require great manual dexterity. If this isn’t a well-formed forecasting
question then please rewrite it as one that is and check it with me before
proceeding."*


*After about 10 minutes thinking about it Claude very politely informed me
that my question was too fuzzy and suggested that the following would be a
better question: *


*Claude*: "By December 31, 2030, does there exist an AI system, possibly
using external tools, code execution, retrieval, and orchestration of other
AI models but not specialized robotic hardware, that is recognized by broad
expert and scientific consensus as matching or exceeding the best
documented human performance, within roughly comparable time and cost, on
essentially every cognitive task that has an objective or
intersubjectively-agreed success criterion (math, science, programming,
forecasting, strategy games, engineering, legal/medical/financial analysis,
etc.) — doing so at least as consistently as elite human experts — with
tasks gated by legal or biological status, tasks with no objective quality
standard, and tasks bottlenecked by physical embodiment (not just fine
dexterity, but also raw speed, strength, and endurance) all excluded?

*I responded with this: *

*Yes I like the way you rewrote my question, please proceed and answer it
with a precise probability.*

*After another 10 minutes Claude started to answer my question but then it
cut off before giving me a precise probability saying that I had reached my
computation limit and I needed to wait 4 hours before proceeding further.
Being impatient I sent Claude's version of my question to GPT and got a
response almost immediately. Spoiler alert, GPT gave a probability of 29%:*

*GPT:  "*If I were approaching this as a superforecaster, I would try to
avoid both "AI is accelerating exponentially, so it's almost certain" and
"human intelligence is special, so it's very unlikely." I'd decompose the
question into several uncertainties and update from current evidence.

*My estimate: 29%*

That's deliberately more precise than "about one-third," but it should be
interpreted as roughly *25–35%*.

Here's how I arrive at it.
What the target actually requires

The threshold is extraordinarily high. It's not asking whether AI surpasses
humans at many important tasks, but whether there is *broad expert and
scientific consensus* that a system:

   - matches or exceeds the *best documented human performance*
   - on *essentially every cognitive task with an objective success
   criterion*
   - at *roughly comparable time and cost*
   - and does so *consistently*
   - by *December 31, 2030*.

Notice that this is actually a harder target than many people's informal
notion of "AGI." It requires not merely impressive demonstrations but
widespread agreement that the system has reached that level.
Reasons the probability is substantial

Several trends point toward rapid progress.

   - Frontier AI capabilities have advanced much faster than most experts
   expected five years ago.
   - Tool use, retrieval, planning, code execution, and multi-agent
   orchestration dramatically expand what language models can accomplish.
   - AI performance has improved across many unrelated domains
   simultaneously rather than through narrow engineering.
   - Scientific research itself is becoming increasingly automated.
   - Scaling has not obviously hit a fundamental wall yet, although gains
   are becoming more expensive.

If these trends continue, by 2030 AI systems could plausibly outperform
elite humans across mathematics, programming, scientific reasoning,
engineering design, forecasting, and many professional analytical tasks.
Reasons the probability is well below 50%

Several obstacles remain.

*1. Generality is harder than excellence.*

Being world-class at hundreds of domains is easier than being reliably
world-class at essentially *all* cognitive tasks.

Even today, frontier systems exhibit uneven competence across domains.

*2. Reliability matters.*

The question requires performance at least as consistent as elite humans.

Occasional catastrophic reasoning failures are much more damaging than
occasional lack of brilliance.

*3. Cost constraints.*

The system cannot require millions of dollars of compute per difficult task.

Comparable time and cost is an important constraint.

*4. Consensus usually lags capability.*

Experts rarely agree immediately.

There may be years of debate over whether remaining weaknesses are
fundamental or merely edge cases.

Even if a system technically qualifies in late 2030, consensus might not
yet exist.
My decomposition

One way I'd estimate it is:

   - AI capable of surpassing elite humans across nearly every objective
   cognitive domain by 2030:
   *~42%*
   - If such capability exists, probability broad expert consensus
   recognizes it by the deadline:
   *~70%*

Multiplying:

0.42 × 0.70 ≈ *0.29*

Hence the *29%* estimate.
What would move me upward?

I would raise this estimate significantly if, within the next 18–24 months,
we observe:

   - AI making major original scientific discoveries with minimal human
   guidance.
   - Reliable autonomous software engineering over projects involving
   millions of lines of code.
   - Strong long-horizon planning over weeks rather than hours.
   - AI laboratories substantially accelerating their own research through
   AI.
   - Benchmarks increasingly failing because humans can no longer produce
   harder evaluation sets.

What would move me downward?

I would lower the estimate if:

   - Scaling laws flatten despite increasing compute.
   - Reliability plateaus well below expert-human consistency.
   - Autonomous agents remain brittle on long-horizon tasks.
   - Progress increasingly depends on expensive human supervision rather
   than algorithmic improvements.
   - Fundamental reasoning bottlenecks persist despite larger models and
   better training.

Calibration

For context, if I had to forecast several milestones by 2030:
Event Probability
AI exceeds average human on most economically useful cognitive work *85%*
AI exceeds elite humans on most major cognitive domains *55%*
AI meets your very demanding "essentially every objective cognitive task"
definition with broad consensus *29%*
Artificial superintelligence (far beyond the best humans at essentially all
cognitive tasks) *10%*

The key distinction is that your definition demands not just exceptional
capability but near-universal coverage, high reliability, reasonable cost,
and broad scientific agreement—all within about four and a half years.
Those extra requirements reduce the probability substantially compared with
simpler definitions of AGI."

*John K Clark    See what's on my list at  Extropolis
<https://groups.google.com/g/extropolis>*
ttm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260706/8e0e62e6/attachment-0001.htm>