<div dir="ltr"><div class="gmail_default" style="font-family:comic sans ms,sans-serif;font-size:large;color:#000000">uploads etc.</div><div class="gmail_default" style="font-family:comic sans ms,sans-serif;font-size:large;color:#000000"><br></div><div class="gmail_default" style="font-family:comic sans ms,sans-serif;font-size:large;color:#000000">If an upload can program itself to enjoy food, sex, travel -- or anything at any time, where is the contrast? Studies support the idea that we need some average or worse times to enjoy the really good times.</div><div class="gmail_default" style="font-family:comic sans ms,sans-serif;font-size:large;color:#000000"><br></div><div class="gmail_default" style="font-family:comic sans ms,sans-serif;font-size:large;color:#000000">Don't you think that a person who gets all the rewarding things and experiences anytime he wants will get bored? And everybody else can do the same thing - we will all be equal. Can we stand it?</div><div class="gmail_default" style="font-family:comic sans ms,sans-serif;font-size:large;color:#000000"><br></div><div class="gmail_default" style="font-family:comic sans ms,sans-serif;font-size:large;color:#000000">The more we eat at one sitting the less it tastes good and so we quit. Wired into us. The only thing I know that is not susceptible to this effect is stimulating the reward centers - like the rats who kept on stimulating themselves and never stopped to eat and died of starvation.</div><div class="gmail_default" style="font-family:comic sans ms,sans-serif;font-size:large;color:#000000"><br></div><div class="gmail_default" style="font-family:comic sans ms,sans-serif;font-size:large;color:#000000">bill w</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jun 14, 2024 at 1:41 PM spike jones via extropy-chat <<a href="mailto:extropy-chat@lists.extropy.org">extropy-chat@lists.extropy.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">>...> On Behalf Of Keith Henson via extropy-chat<br>
<br>
>...I have watched things develop for over 40 years.<br>
<br>
>...The people who worked on the subject are frighteningly smart. And they have not made notable progress. My suspicion is that it may not be possible for humans to solve the alignment problem, in fact, it may be something that cannot be solved at all.<br>
<br>
>...Part of the reason is that a very well-aligned (but useful) AGI combined with human desires could have undesired effects...Keith<br>
_______________________________________________<br>
<br>
Keith I reluctantly came to the same conclusion about 25 yrs ago at the peak of Eliezer's friendly AI movement. There were a lot of local events talking about that problem in those days, and I used to hang out there, listening much, saying little. Your arguments about why humans go to war had already convinced me by that time you are right. But I also realized understanding the cause doesn't get us to a solution. It only helps us understand why we are screwed: if we understand that nations go to war over scarce resources, then thru some miracle of technology and science manage to get everyone fed and laid, we still haven't solved the underlying problem. It helps temporarily. But if everyone is fed and laid, the population grows and soon they aren't anymore.<br>
<br>
Regarding AGI aligned with human desires having undesired effects: I also came to that conclusion, but that one was easy for me, for a reason. There is a category between conscientious objector (the Amish) and the warrior known as the conscientious cooperator. The conscientious cooperator is the guy who recognizes that nations do go to war, there is nothing we can do to stop that at our level. But if one participates in developing the technology to make attacking another nation more costly than the benefit, then that is a worthwhile pursuit, for it increases the chances that the conflict will be settled over the negotiating table rather than the battlefield.<br>
<br>
Humans are human level intelligence, so we can think of us as biological AGI. We are aligned with human desires and we cause undesired effects. Stands to reason that AGI would do likewise.<br>
<br>
But really it is worse than that. In the last coupla years especially, many of us who have been singularity watchers for three decades have become convinced that now we really are getting close to that time, and that (as we feared) AI is being used by governments as a super weapon. We are in the middle of a huge AI arms race. Eliezer was right all along, or partly right. He warned us this would happen, but he was convinced there was a way out. I don't know that there is, and reluctantly conclude that this is one example of an element of the Great Filter which explains why the universe is not humming with artificial signals.<br>
<br>
I know this is the ExI list, so my apologies for what must look like a terribly negative post, but I will end on a positive note, as is my wont, and simultaneously gives me a chance to use the funny-sounding word wont. I believe there is hope. I recognize humanity is in grave danger, but I firmly believe there is a chance we can prevent or avoid slaying ourselves. I have not given up on us. I would offer a bit more detail on that if I knew any, but suffice it to say I firmly believe there is a way, and we might find it in time. This is my take on Dynamic Optimism: I live and work towards a version of AGI which peacefully coexists with the descendants of mankind.<br>
<br>
spike<br>
<br>
<br>
<br>
_______________________________________________<br>
extropy-chat mailing list<br>
<a href="mailto:extropy-chat@lists.extropy.org" target="_blank">extropy-chat@lists.extropy.org</a><br>
<a href="http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat" rel="noreferrer" target="_blank">http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat</a><br>
</blockquote></div>