[ExI] Morality research

[Anders Sandberg]

Eugen Leitl wrote:
> On Mon, Nov 14, 2011 at 11:09:10AM +0000, Anders Sandberg wrote:
>
>   
>> One of the interesting things to watch is how people who actually work  
>> on 'friendly AI' reinvent or rediscover chunks of old philosophy. The  
>>     
>
> You don't consider Friendly AI meaningless? 
>   

I think it is unlikely to work ('it' being safe-by-design AI; 
friendliness was just the first iteration, CEV the second, and now we 
have a plethora in Nick's manuscript). But it remains an important 
question in the fledgling field of superintelligence studies. We should 
pray that intelligence takeoffs are soft, given the difficulty of AI 
safety.


>> The problem happens when you don't think there is anything worth looking  
>> at in the old discipline, since obviously your fresh approach is going  
>> to cut through all that messy stuff. After all, they can't have thought  
>>     
>
> How did the old philosophers deal with the idea of some (but not all)
> humans becoming literal gods?
>   

I don't know the actual stoic or epicurean writings, but I think they 
actually discussed the topic. I think Lucretius would say it would be a 
very good thing if humans could become gods (who live in a sort of 
tranquil Nirvana that humans should aspire to). The ancient Greeks of 
course felt that one could achieve immortality (not necessarily just 
figurative) through excellence and heroism. Romans even had a special 
category divus for ex-mortals, separate from the deus category of a god.

We might of course care more about the rights and obligations of godhood 
than the virtues. But even here the classical ethics arguments seem to 
apply: Plato's Euthyphro dilemma would seem to imply that unless you 
manage to become a very metaphysically powerful god that can "make" 
morality as you wish, you would still be bound by universal laws of 
morality (or social contracts). So the Voltaire-Uncle Ben dictum "with 
great power comes great responsibility" might indeed apply...

(And let's not forget Empedocles memorable attempt at apotheosis...)

-- 
Anders Sandberg,
Future of Humanity Institute 
Oxford Martin School 
Faculty of Philosophy 
Oxford University