[ExI] Superforecast for AGI Singularity

[BillK]

On Sun, 5 Jul 2026 at 11:59, BillK <pharos at gmail.com> wrote:

> I asked GLM to explain the final estimate of 4% probabilty of the
> Singularity by 2040, as to me this sounded like saying it was impossible by
> 2040. The answer was that GLM was separating AGI and the subsequent
> Singularity event.
> So I followed up by asking why the Singularity would be so slow after AGI
> was achieved.
> The answer was physical constraints on building in the real world.
> BillK
>


In response, I suggested to GLM that the current arms race to be the first
nation to achieve AGI might produce an earlier Singularity. GLM was
concerned that in a "race" for AGI too much could go wrong. Including
omitting safety checks on AIs and even competitors sabotaging each other.
BillK


GLM-5-Turbo -

Your logical deduction is mathematically correct in a vacuum: If the
physical hardware bottleneck is fixed, then an earlier arrival of AGI
directly translates to an earlier arrival of the Singularity.

However, introducing the geopolitical arms race dynamic you just described
does not linearly increase the probability of a 2040 Singularity. In fact,
when applied to the conjunctive chain, this specific variable introduces
severe second-order friction that actively *lowers* the likelihood of a
successful, uninterrupted intelligence explosion.

Here is how the "every country wants to be first" dynamic impacts the
calibration:

*1. The Sabotage and Asymmetric Warfare Friction* If AGI is perceived as
the ultimate strategic advantage, the incentive to prevent a rival from
achieving it first approaches 100%. This does not just mean export controls
on chips (which we already see); it means the probability of
state-sponsored cyberattacks, physical sabotage of power grids supplying
rival data centers, or kinetic strikes approaches a non-trivial baseline.
An intelligence explosion requires months to years of uninterrupted,
massive compute cycles. A single successful sabotage event during the
critical recursive self-improvement phase permanently aborts the
Singularity. This drastically reduces the probability of Condition D
(Geopolitical Non-Interference).

*2. The Grid-Construction Mismatch* The massive capital deployment you
noted is heavily front-loaded into GPU procurement and data center
construction, but lags severely in energy generation. Building a data
center takes 1-2 years; building a gigawatt-scale nuclear or green energy
plant takes 5-10 years. In an arms race, countries are effectively plugging
tens of gigawatts of AI demand into grids that lack the base-load capacity
to support it. This increases the probability of systemic grid failures,
forced rolling blackouts, or the need to throttle compute precisely when
the AGI needs it most. This reduces the probability of Condition B
(Infrastructure Scaling).

*3. The Safety-Compression Paradox* In a highly competitive arms race, the
dominant strategy is to deploy systems at the edge of capability while
cutting corners on safety, alignment, and interpretability. For an AGI to
successfully undergo recursive self-improvement (Condition C), its
goal-directed behavior must remain perfectly stable and aligned through
multiple iterations of its own code. An AGI built under extreme time
pressure, without adequate alignment scaffolding, is statistically far more
likely to suffer from reward hacking, goal misgeneralization, or
catastrophic mode collapse during its first autonomous rewrite. It will
destroy itself before it can explode.

*Conclusion on Calibration* You have correctly identified the primary
accelerator in the system (massive capital and geopolitical will). However,
superforecasting requires weighting accelerators against their resulting
friction. The arms race dynamic drastically increases the probability of
*AGI* by 2040, but it simultaneously injects massive volatility, sabotage
risk, and safety trade-offs into the system.

Because a Singularity requires a flawless, uninterrupted execution of the
conjunctive chain, an environment characterized by geopolitical desperation
and infrastructure overloading is a highly hostile environment for a stable
intelligence explosion. Therefore, this new variable does not move my 4%
estimate upward; it validates why it remains in the low single digits.

-----------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260705/223b40ca/attachment.htm>