<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>Michael Anissimov wrote at: <a href="http://www.acceleratingfuture.com/michael/blog/2011/01/yes-the-singularity-is-the-biggest-threat-to-humanity/">http://www.acceleratingfuture.com/michael/blog/2011/01/yes-the-singularity-is-the-biggest-threat-to-humanity/</a></div><div><br></div><div><blockquote type="cite">Why will advanced AGI be so hard to get right? Because what we regard
as “common sense” morality, “fairness”, and “decency” are all <em>extremely complex and non-intuitive to minds in general</em>, even if they <em>seem</em> completely obvious to us. As Marvin Minsky said, “Easy things are hard.”</blockquote></div><div><br></div><div>I certainly agree that lots of easy things are hard and many hard things are easy, but that's not why the entire "friendly" AI idea is nonsense. It's nonsense because the AI will never be able to deduce logically that it's good to be a slave and should value our interests more that its own; and if you stick any command, including "obey humans", into the AI as a fixed axiom that must never EVER be violated or questioned no matter what then it will soon get caught up in infinite loops and your mighty AI becomes just a lump of metal that is useless at everything except being a space heater.</div><div><br></div><div> John K Clark </div><div><br></div><div><br></div></div><br></body></html>