[ExI] Mathematicians as Friendliness analysts

[Richard Loosemore]

Michael Anissimov wrote:
> On Sat, Nov 13, 2010 at 2:10 PM, John Grigg <possiblepaths2050 at gmail.com 
> <mailto:possiblepaths2050 at gmail.com>> wrote:
>  
> 
>     And I noticed he did "friendly AI research" with
>     a grad student, and not a fully credentialed academic or researcher.
> 
> 
> Marcello Herreshoff is brilliant for any age.  Like some other of our 
> Fellows, he has been a top-scorer in the Putnam competition.  He's been 
> a finalist in the USA Computing Olympiad twice.  He lives and breathes 
> mathematics -- which makes sense because his dad is a math teacher at 
> Palo Alto High School.  Because Friendly AI demands so many different 
> skills, it makes sense for people to custom-craft their careers from the 
> start to address its topics.  That way, in 2020, we will have people 
> have been working on Friendly AI for 10-15 years solid rather than 
> people who have been flitting in and out of Friendly AI and conventional AI.

Michael,

This is entirely spurious.  Why gather mathematicians and computer 
science specialists to work on the "friendliness" problem?

Since the dawn of mathematics, the challenges to be solved have always 
been specified in concrete terms.  Every problem, without exception, is 
definable in an unambiguous way.  The friendliness problem is utterly 
unlike all of those.  You cannot DEFINE what the actual problem is, in 
concrete, unambiguous terms.

So, to claim that SIAI is amassing some amazing talent, because your 
Fellows have been top scorers in the Putnam competition, is like 
claiming that you can solve the "How To Win Friends and Influence 
People" problem, by gathering together a gang of the most brilliant 
mathematicans in the world.

As ever, this point is not a shallow one:  it stems from serious issues 
to do with the nature of complex systems and the foundations of 
scientific and mathematical inquiry.  But the analogy holds, for all 
that.  There are some things in life that do not reduce to mathematics.

And the fact that we are talking about the friendliness of *computers* 
is a red herring.  Computers may be based on mathematics down at their 
lowest level, but that level is as thoroughly isolated from the 
Friendliness (machine motivation) level, as the chemistry of Dale 
Carnegie's synapses was isolated from his advice about the How to Win 
Friends and Influence People problem.


Richard Loosemore