[ExI] Isn't Bostrom seriously bordering on the reactionary?

[Anders Sandberg]

I often joke that we at FHI are turning into transluddites.

But there is a method to the madness: given the astronomical waste 
argument (the potential posthuman future is worth *a lot*) 
consequentialists should be *much* more concerned about existential 
risks than they commonly are, and the maxipok principle means that we 
should try to maximize the chances of surviving - even if that means 
being less gung ho about radical new technologies. Similarly, 
considerations about the documented limits and biases of human 
decisionmaking, together with the realization that the importance of 
different priorities/goals may differ by orders of magnitude, makes 
ensuring that we focus on the *right* targets a lot more important than 
most people realize.

I disagree with Nick mainly about the utility of individual or 
collective rationality. He seems to think that rational decisionmaking 
can solve key problems by finding optimal solutions; I am not entirely 
convinced. I think distributed approaches (let a thousand flowers bloom, 
Postrel's dynamism, trial and error) have better chances to get good 
outcomes, including inventing regulatory and protective systems against 
some of the threats that are emerging. We both agree that finding ways 
of reducing the bias and other stupidity-inducing factors have a high 
priority.

Giulio Prisco wrote:
> Oh, nothing. Not reactionary, just boring. Nick used to be fun, he
> must be aging fast.
>   

Well, he got troublesome employees like me :-)

(Sometimes reactionary transhumanism is good. Nick's paper on human 
dignity for the Presidents Council on Bioethics ruffled a few feathers 
by arguing for posthuman dignity from a very conservative standpoint - 
they had not expected being forced to criticise their *own* position)

-- 
Anders Sandberg,
Future of Humanity Institute 
James Martin 21st Century School 
Philosophy Faculty 
Oxford University