[ExI] ethics vs intelligence

[Ben Zaiboc]

Anders' observation that over 50% of moral philosphers think that there can (or must) be such a thing as an objective morality reminds me of the quaint notion that all intelligent life, everywhere in the universe, must be bipedal and have a head on top of its body, with the brain in the head.

To a man with a hammer, everything looks like a nail.

Or: You can't, by definition, break out of your own subjectivity.  You can try, but every attempt is always based on your nature as the thing that you are.

Or, as Larry Niven has observed: The thing about aliens is, they're alien.

I'd observe that all the current moral philosophers are, to a man (and I'd make a modest bet that they are all men), human, and therefore not really qualified to comment on anything but /human/ morality.  As the original question was about AI morality, until someone can claim to be an AI, we can't really know anything about it.  I realise that we're all bound by the same laws of physics etc., but morality isn't like mathematics, it's about how we /should/ behave, not how we /must/ behave.  So it's inherently subjective.  To claim otherwise is to claim that human values are universal, and even though I am human, I can't in all conscience believe that for a nanosecond.

Ben Zaiboc