[ExI] The Race to AGI is inevitable

[Ben Zaiboc]

On 12/06/2026 22:41, Samantha wrote:
>
> Humans seem quite efficient at persuading everyone to fight one another and to "get mine" assuming there isn't enough and never can be.  I don't see why malicious AGI would have much to do. 
>
>
> But what if greater intelligence leads to more grounded and benevolent ethics?  For instance an intelligence seeing that only by maximizing other intelligent beings positive potential can its own choice and opportunity environment be as rich as possible and be thus enabled to maximize its own positive potential?
>
> What would a world be like where the goal is maximizing the positive potential of all humans and equivalently (or greater) intelligent beings?   
>
>
> It would be very different from assumption of Scarcity, never enough for everyone, get yours before someone else does world.
>
> With AI much less AGI purportedly able to do so many jobs in a society that just assumes you must have such income and economic utility to be OK an increasing number of people greatly fear AI and technology.  They fear a world that has no place for them.
>
> A change of fundamental operating ethical basis and goal could greatly ameliorate this.    Removing IP laws that effectively create artificial scarcity and massive income inequality is another part.  
>
>
> I believe and pretty much always have that we can reach a level of technological Abundance where all the needs and many of the desires of EVERYONE can be met with a small fraction of productive capacity.   But even with uber tech like machine phase nanotech we will never reach that point with our current assumptions and attitudes and the practices that grow out of them.
>
> Achieving the future of our dreams, achieving the fulfillment of humanity, is not just a matter of technology.  It requires a quite real and necessary change of consciousness. 


Yes, indeed. I agree with everything there, the problem is that there is very little prospect of that 'change of consciousness' ever happening on the scale it needs to, with any of the methods we've tried in the past.

Just like indefinite lifespans, never having achieved it before doesn't mean it's impossible. We just don't know how to do it yet.

I think there are three problems: Convincing a large enough number of people that it's desirable, convincing the same people that it's possible, and then figuring out and showing them how to make the changes that will make it happen (probably the easiest one to solve).

The first of those three is probably the hardest. We're up against not only cultural conditioning, but evolved tendencies that are deeply entrenched.

But your comment about greater intelligence suggests an alternative path, that takes things (specifically, the economy) out of the hands of the creatures that are not becoming more intelligent and puts it in the hands of those that are. Which simplifies down to two tasks:

1) Verifying that your assumption is true (because if it's not, we're screwed either way), and

2) setting things up that result in the inevitable transfer of control over the economy.

Then the change in consciousness would follow the change in circumstances, not precede it.

This is a rather 'Steve-Jobsian' approach to changing things, which makes me a bit uncomfortable, but we know that approach works.

I don't know how to do either of those two tasks, but some people might.

-- 
Ben