[extropy-chat] Proof that a paperclip maximizer cannot be a general intelligence

[Eliezer S. Yudkowsky]

justin corwin wrote:
> On 8/18/05, Marc Geddes <marc_geddes at yahoo.co.nz> wrote:
> 
>>Is the following sentence 'True' or 'False'?
>>
>>"Either I did not just carry out the goal of
>>understanding this sentence or the goal 'Maximize
>>paperclips' is not the goal with the highest utility.
> 
> 
> "Either I did not just carry out the goal of understanding this
> sentence, or I am a weevil"
> 
> *poof* instant weevil?

"If this sentence is true, then Santa Claus exists."

Suppose the sentence were true.
Then the antecedent would be true, and the conditional would be true, so the 
antecedent would be true and Santa Claus would exist.
Thus if the sentence is true, Santa Claus exists.
But this is precisely what the sentence asserts, so it is true.
Therefore Santa Claus exists.

Lob's Theorem in action.

See also http://www.machall.com/index.php?strip_id=189

-- 
Eliezer S. Yudkowsky                          http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence