[extropy-chat] Would You Enjoy Knitting?

[Robin Hanson]

On AM 6/27/2006, Lee Corbin wrote:
>... we already know what we like, and so what good would be the 
>freedom to choose what we like? Well, I hope that you won't 
>automatically close down on the possibility that not only ought we 
>have such freedom, it should be used in the most imaginative ways we 
>can short of inflicting harm on ourselves or others.
>Yet another upcoming freedom is more terrifying still. It is the 
>freedom to determine one's own levels of happiness, satisfaction, 
>enjoyment, contentment, and fulfillment by artificial means.

It seems clear to me that, if allowed, some people will use such 
freedoms, and explore the space of possible preferences and emotional 
reactions.   It also seems clear that many people will not choose to 
explore this space, preferring to retain their current preferences 
and emotional reactions.

So what will future people want and enjoy?  The result will be some 
combination of momentum and selection.   Momentum is the tendency of 
people to want to retain their preferences, and selection is their 
environment rewarding some preferences relative to others.

If selection were strong enough, what we want now would not 
matter.   Consciousness and perhaps even intelligence would be 
selected away in many environments.   I analyzed a scenario like this 
in "Burning the Cosmic Commons."

What we want now may well matter.   It is possible that it will 
matter because we create some grand international organization to 
enforce our current wishes, or because we initialize a seed AI with 
what we want, though these scenarios seem unlikely to me.   Or some 
people may live forever and continue to accumulate wealth and spend 
to get what they want.

It is not clear to me which will be more important: what people now 
want, or what they now want to want.

People now do seem to have preferences they want to have, that differ 
from the preferences they do have.   Many religious people, for 
example, want to be more committed to their religion.   Many married 
people want to be less tempted to betray their marriage.   Many 
somewhat ambitious people would like to be more devoted to their 
ambitions.  Many somewhat altruistic people would like to be more 
truly altruistic.  Many soldiers would like to be more courageous in 
battle.  And so on.

In general humans want to be more "ideal" than they are, and this is 
due to a generic self-deception whereby people believe they are more 
ideal than they are, in order to convince others to admire and 
affiliate with them.   "Be careful what you wish for" is apt here - 
many bad things will happen when people get their wish to be more 
ideal.   And because deep down people know they really don't want to 
be as ideal as they say they do, many people will find excuses to 
back away from their professed ideals.

It is not even clear that people who can change themselves will 
become more consistent, wanting to want what they do want.   Perhaps 
they will instead become better at self-deception.



Robin Hanson  rhanson at gmu.edu  http://hanson.gmu.edu
Associate Professor of Economics, George Mason University
MSN 1D3, Carow Hall, Fairfax VA 22030-4444
703-993-2326  FAX: 703-993-2323