[ExI] Intelligence Unbound!

Anders Sandberg anders at aleph.se
Tue Jul 22 23:18:18 UTC 2014

William Flynn Wallace <foozler83 at gmail.com> , 22/7/2014 5:05 PM:
​Why should we believe them?​  ​If they are much smarter it follows that we will not be able to understand their reasoning any more than I can follow debates in quantum physics.  
Smart people can sometimes explain complex things clearly. The mental work of understanding a domain may be large, but the resulting theory might actually be simple.
In practice it is a matter of trust: when the quantum physicist tells you to use phosphorous as a dopant in your semiconductor or your lawyer says pleading insanity is a bad idea the reasoning might be beyond you, but if they have reliably been right in the past it would be stupid not to follow their advice. The hard situation is when feedback is not forthcoming: if a black box tells me I will have a happier life if I believe in P, I cannot trust its prediction directly since I do not understand why or have evidence that it works. But if it had been right about other people or other questions, it might be rational to follow the advice.
Of course, apparently very good advice might still be hiding bad advice:http://lesswrong.com/lw/jao/siren_worlds_and_the_perils_of_overoptimised/

Anders Sandberg, Future of Humanity Institute Philosophy Faculty of Oxford University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20140723/ef68d77f/attachment.html>

More information about the extropy-chat mailing list