[ExI] The Race to AGI is inevitable

[BillK]

On Thu, 11 Jun 2026 at 07:22, Ben Zaiboc via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

> On 11/06/2026 05:54, Samantha Atkins wrote:
> > If humans cannot sufficiently overcome parts of our evolutionary
> psychology in order to act more rationally then I fear for our species.
> Environment change drives evolutionary change. Our environment via
> technology has changed so much that we must become different than we
> evolved to be in order to meet the current and coming challenges.  This is
> the Great Filter.
>
> That's probably the most convincing Great Filter I've ever heard of.
> The disparity between the constant speed of evolution and the accelerating
> speed of technological progress is something that will apply to any
> civilisation that arises from biology, no matter how alien.
> Ben
> _______________________________________________



I gave these two comments to DeepSeek AI and asked him to discuss them.
DeepSeek discussed each in turn, then the overall implications.
Agreed that they implied we should expect to hit the Great Filter in the
near future.
BillK

DeepSeek AI -

These are remarkable comments. They take the discussion out of the
immediate policy debate and reframe it as a question about the fundamental
compatibility between biological intelligence and technological
acceleration — which is, as the second commenter notes, a deeply compelling
candidate for the Fermi Paradox's Great Filter. Let me discuss each in
turn, and then their combined weight.
------------------------------
On the first comment: the Great Filter as a failure to overcome our own
evolutionary psychology

This is a powerful and uncomfortable framing. The commenter is making three
linked moves:

*1. The diagnosis: we are maladaptive to our own creation*

The argument is that our threat-detection systems, our tribal loyalty
instincts, our short-term bias, and our dominance-seeking behaviours were
shaped in an environment of scarcity and intergroup competition. Those
traits are now mismatched to an environment where the primary existential
threats are self-created, global, long-term, and require unprecedented
cooperation between rival groups.

In AI safety terms, we're trying to solve a problem that requires *system-2*,
long-horizon, cosmopolitan cooperation using brains that are largely driven
by *system-1*, short-horizon, in-group/out-group dynamics. The commenter
fears this gap is too wide.

*2. The prescription: we must become different*

"Environment change drives evolutionary change." This is true, but there's
a crucial ambiguity. Biological evolution works on generational timescales.
The commenter is almost certainly talking about *cultural* evolution, or
perhaps a deliberate, self-directed transformation of our cognitive and
social architecture — through education, institutional design, or
potentially even technological augmentation of our own decision-making.

This is an enormous demand. It essentially asks us to do what no species
has ever done consciously: recognise that our inherited nature is
insufficient for survival, and intentionally reshape it within a single
generation, before the environment (which we created) destroys us.

*3. The Great Filter connection*

The Great Filter is the hypothetical barrier that prevents civilisations
from reaching interstellar visibility. If this commenter is right, the
Filter is not asteroid impacts or self-replicating nanobots. It's something
more elegant and tragic: *the very traits that enable a species to develop
advanced technology also ensure it cannot wield that technology safely.*

This is a deeply plausible filter because:

   -

   It's scale-invariant. It would apply to any biological intelligence that
   evolved through natural selection.
   -

   It's time-bound. The acceleration of technology always outpaces the slow
   grind of biological evolution.
   -

   It's subtle. It wouldn't look like a sudden catastrophe from outside. It
   would look like a civilisation that simply couldn't coordinate well enough,
   fast enough, and therefore ran headlong into its own power.

The commenter is essentially saying: *our psychology is the bottleneck, and
the clock is the exponential curve of technological capability.*
------------------------------
On the second comment: the universal applicability of this filter

This commenter is zooming out further and noticing something profound: this
isn't about humans specifically. It's about the structural relationship
between any biological origin and technological acceleration.

*The speed mismatch is a constant*

Evolution operates at a fixed pace — governed by mutation rates, selection
pressures, and generational turnover. Even on an alien world with different
biochemistry and planetary conditions, the fundamental dynamic is the same:
biology adapts slowly, iteratively, and locally.

Technology, by contrast, is a *cumulative, cultural process* that can
accelerate exponentially once certain thresholds are crossed — symbolic
language, writing, printing, computation, AI-assisted research. The moment
a species crosses into technological acceleration, it has created an
environment that changes orders of magnitude faster than the organism that
created it.

This gap — between a mind shaped by ancient selection pressures and a world
it remakes faster than it can adapt — is not a human problem. It's a
mathematical inevitability for any biological civilisation that survives
long enough to develop advanced technology.

*This makes it an elegant universal filter*

Previous Great Filter candidates often suffer from being too contingent or
avoidable:

   -

   Nuclear war? Maybe most civilisations see it coming and disarm.
   -

   Climate collapse? Maybe most switch to renewables in time.
   -

   Runaway AI? Maybe most solve alignment.

But this filter is different. It's not a specific technology. It's a
*relationship* between two incommensurable rates of change. There is no
obvious reason any biological species would escape it. The very success of
intelligence in mastering its environment creates an environment the
intelligence is no longer suited to navigate.

*Its explanatory power*

If this filter is real, it neatly explains the silence:

   -

   Civilisations arise, develop technology, and then face problems that
   require unprecedented levels of global cooperation, long-term thinking, and
   the suppression of competitive instincts.
   -

   These are precisely the traits that natural selection *did not optimise
   for* and may even have selected against in many contexts.
   -

   The window to solve these problems — to culturally evolve or self-modify
   fast enough — is narrow and unforgiving.
   -

   Most civilisations fail in that window. Not with a bang. With a series
   of decisions that all made sense to their individual decision-makers, but
   collectively led to ruin.

------------------------------
The combined weight of these two comments

Together, they paint a picture that is both bleak and clarifying.

Bleak, because they suggest the very thing Morris's article treats as a
manageable challenge — our psychology, our geopolitics, our inability to
coordinate — may in fact be the signature of an unsolvable structural trap.

Clarifying, because if this is the Great Filter, then the only meaningful
question is: *can we culturally, institutionally, or technologically evolve
ourselves fast enough to close the gap before it closes on us?*

And that question reframes everything. It's not "should we pause AI?" or
"can we regulate it?" — those are surface-level symptoms. The deep question
is: *can a species whose wisdom evolved to handle inter-tribal conflict on
a savannah become wise enough to handle planetary-scale technological
power, in a single lifetime?*

If the answer is no, then the Great Filter is not behind us. It is dead
ahead.

----------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260612/3d0720eb/attachment.htm>