<html><head></head><body><div><span data-mailaddress="foozler83@gmail.com" data-contactname="William Flynn Wallace" class="clickable"><span title="foozler83@gmail.com">William Flynn Wallace</span><span class="detail"> <foozler83@gmail.com></span></span> , 22/7/2014 5:05 PM:</div><blockquote class="mori" style="margin: 0px 0px 0px 0.8ex; border-left-width: 2px; border-left-color: blue; border-left-style: solid; padding-left: 1ex;"><div class="mcntgmail_default" style="font-family: 'comic sans ms', sans-serif; font-size: small;"><br></div><div class="mcntgmail_extra"><div class="mcntgmail_quote"><blockquote class="mcntgmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div class="mcntHOEnZb"><div class="mcnth5"><div class="mcntgmail_default" style="font-size: small; font-family: 'comic sans ms', sans-serif; display: inline;">Why should we believe them?</div> <div class="mcntgmail_default" style="font-size: small; font-family: 'comic sans ms', sans-serif; display: inline;"> If they are much smarter it follows that we will not be able to understand their reasoning any more than I can follow debates in quantum physics. </div></div></div></blockquote></div></div></blockquote><div><br></div>Smart people can sometimes explain complex things clearly. The mental work of understanding a domain may be large, but the resulting theory might actually be simple.<div><br></div><div>In practice it is a matter of trust: when the quantum physicist tells you to use phosphorous as a dopant in your semiconductor or your lawyer says pleading insanity is a bad idea the reasoning might be beyond you, but if they have reliably been right in the past it would be stupid not to follow their advice. The hard situation is when feedback is not forthcoming: if a black box tells me I will have a happier life if I believe in P, I cannot trust its prediction directly since I do not understand why or have evidence that it works. But if it had been right about other people or other questions, it might be rational to follow the advice.</div><div><br></div><div>Of course, apparently very good advice might still be hiding bad advice:</div><div>http://lesswrong.com/lw/jao/siren_worlds_and_the_perils_of_overoptimised/<br><blockquote class="mori" style="margin: 0px 0px 0px 0.8ex; border-left-width: 2px; border-left-color: blue; border-left-style: solid; padding-left: 1ex;"><div class="mcntgmail_extra"><div class="mcntgmail_quote"><blockquote class="mcntgmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div class="mcntHOEnZb"><div class="mcnth5"><div class="mcntgmail_default" style="font-size: small; font-family: 'comic sans ms', sans-serif; display: inline;"></div></div></div></blockquote></div></div></blockquote><br><blockquote class="mori" style="margin: 0px 0px 0px 0.8ex; border-left-width: 2px; border-left-color: blue; border-left-style: solid; padding-left: 1ex;"></blockquote><br>Anders Sandberg,
Future of Humanity Institute
Philosophy Faculty of Oxford University</div></body></html>