[extropy-chat] Singularitarian verses singularity

[Eliezer S. Yudkowsky]

Brett Paatsch wrote:
> Eliezer S. Yudkowsky wrote:
> 
>> The Extropy list is one of the Singularity Institute's primary
>> sources of donors.  This list is where Brian Atkins and Sabine
>> Atkins (then Sabine Stoeckel) and I got together and founded SIAI.
>> If you aren't doing anything, that's your own choice.  If you
>> dislike your choice, change it!
>> 
>> The Singularity Institute is currently running a $100,000 Challenge
>>  Grant.  So's Alcor, if that's more to your taste.  From bystander
>> to actor is a straightforward transformation, if you're
>> dissatisfied with cheering from the sidelines.
>> 
>> I agree that political yammering is a failure mode, which is why
>> the SL4 list bans political discussion.
> 
> To ban something is an intensely political act.

Well, duh.  I'm not saying it's bad to commit acts that someone might 
view as intensely political.  I'm saying that it's okay - no, really, 
it's okay - to create a mailing list that doesn't contain a hundred 
messages per day arguing about Bush.

> To ban political 
> yammering there has to be someone who decides what is and is not
> political yammering.

The SL4 List Snipers are myself, Mitch Howe, and J. Andrew Rogers.

> How do you conclude that *you* are especially qualified to decide 
> what is political yammering?

Anyone with common sense can do the job.  We don't try to discriminate 
between good political posts and bad political posts, we just ban it 
all.  That's not what the SL4 list is for.

> It seems like a "friendly" AI with
> *your* values could only be a benevolent dictator at best.  And
> benevolent not as those that are ruled by it decide but as it decides
> using the values built in by you.

Yeah, the same way an AI built by pre-Copernican scientists must forever 
believe that the Sun orbits the Earth.  Unless the scientists understand 
Bayes better than they understand Newtonian mechanics.  AIs ain't tape 
recorders.

http://singinst.org/friendly/collective-volition.html

-- 
Eliezer S. Yudkowsky                          http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence