[extropy-chat] RE: Singularitarian verses singularity

[gts]

On Thu, 22 Dec 2005 01:46:28 -0500, gts <gts_2000 at yahoo.com> wrote:

> Remember we need the AI to handle challenges beyond our abilities.  How  
> exactly can a challenge be beyond our own abilities yet humans be  
> capable enough to second guess the AI?

Certain difficult questions of ethics and morals would I think need to be  
resolved first. The answers would then need to be coded and made  
open-source and required in any SAI charged with the task of making public  
policy decisions.

For example two people are adrift in a life-raft with limited food and  
supplies. One is a scientist who has discovered but has not yet published  
a real cure for cancer. The other has nothing obvious to contribute to  
humanity. Is it more permissible for the scientist than the other to push  
an innocent person over-board to save his own life? The scientist would be  
saving the lives of countless innocent persons in addition to his own but  
the other person could make no similar claim.

I don't pretend to know the answer but I think an SAI might answer in the  
affirmative, while many humans might answer either that 1) murder is  
always wrong, or that 2) it's a case of every man for himself without  
regard to the greater social consequences.

I'm not sure we could trust an SAI to formulate the right answer to such  
questions on their own, so I suppose I am in agreement with you, at least  
tentatively.


-gts