[ExI] More super forecasting

[John Clark]

*Four hours have now passed so I was able to ask Claude the following
question:*

*ME: During our last interaction I ran out of my allowed computation before
you could give me a precise probability to my question, can you give it to
me now?*

*Claude: "*7%, with a range I'd defend of roughly 4–12%.

Here's how that falls out of what I'd already dug up: the reference classes
split into two tiers depending on how hard a bar they're actually pricing.
Samotsvety's superforecasters put standard "AGI" (average-competent-human
across most tasks) at 28% by 2030; Metaculus's crowd comes out around 30%
on the same kind of bar, interpolating their reported 25%-by-2029 and
50%-by-2033 figures. The one data point that asks almost exactly your
question — Grace et al.'s survey of published AI researchers, on the
probability of machines outperforming humans at literally every task —
works out to roughly 15% by 2030 once you interpolate their reported
10%-by-~2027 and 50%-by-2047 figures, and that's before accounting for how
stale it is (it predates the 2025–2026 benchmark acceleration I'd found).

Your definition is a strictly harder version of even that closest match. It
requires beating the single best documented human, not just "workers"
generally; something close to elite-human consistency, not just frequent
success; and broad expert/scientific consensus actually recognizing this
has happened, which historically lags the underlying reality by a year or
more even in narrower, easier-to-adjudicate domains like chess. It's a
little easier in one respect — excluding embodiment and dexterity — but
that buys back less than it sounds like, since proving theorems, writing
novels, and professional analysis were never dexterity-gated to begin with.
That's why I land below that 15% figure rather than above it: the extra
stringency outweighs the staleness adjustment.

That also matches what the capability frontier actually shows: genuinely
superhuman performance where reward signals are dense and verifiable
(competition math, coding, Mythos Preview autonomously chaining novel
exploits), sitting right next to near-total failure on tasks requiring a
world model built from scratch in a genuinely novel setting (ARC-AGI-3 —
humans solve 100%, frontier models were under 1% as of this spring). That
jaggedness, with new gaps reopening each time an old one closes, is exactly
what should keep this forecast well below the "beats humans at most things,
often" number.

The biggest lever in either direction: whether current reasoning/agentic
progress genuinely generalizes to novel-environment planning the way it has
to math and code (pushes up a lot if yes), versus Yann LeCun's bet that
transformer-based LLMs structurally can't close that particular gap without
a different architecture (pushes down hard if he's right) — that's a live,
unresolved disagreement among serious researchers, not a settled question,
and it's doing most of the work in my uncertainty range."

*John K Clark    See what's on my list at  Extropolis
<https://groups.google.com/g/extropolis>*


*61*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260706/d08d9e96/attachment.htm>