[extropy-chat] limits of computer feeling

Stathis Papaioannou stathisp at gmail.com
Tue Mar 13 08:37:55 UTC 2007

On 3/13/07, Anders Sandberg <asa at nada.kth.se> wrote:

Besides, how easy is it to set one's desires? Even if it is technically as
> simple as in Greg Egan's _Reasons to be Cheerful_ it is obvious to anybody
> with half a brain that there are risks involved, some of which are very
> subtle. Most likely desires are very complex systems few will have the
> skills to adjust well. Hence there is a market for services related to
> desire management. It will range from desire consultants and designers
> over desire monitors to the psycops and mind auditors who occasionally
> check that you are not stuck in a bad mental state, insanity and
> addiction. Compare to the role of the House AIs in Wright's golden
> transcendence books. Clearly there is a huge market in a desire economy!

No doubt, someone will try to regulate self-modification, but it is
interesting to speculate as to what would happen if it were allowed to
develop unfettered. We would have to consider not only second order desires
but also third and higher order desires. For example, if someone felt
uncertain as to what they should desire, instead of seeing a desire
consultant they might simply nullify the uncomfortable aspect of the
uncertainty, rendering themselves confident and happy with any decision they
might make. I think the equilibrium point in this process will ultimately be
a computer heaven in which the inhabitants experience maximum pleasure for
no effort, as John K Clark has suggested. However, it may be a long process
before someone gets to heaven, and there will always be at least the
eccentrics who deliberately limit their self-modification (perhaps by
modifying themselves so that they are never again tempted to modify
themselves). Moreover, even if everyone is in the heaven computer, there
will be either servitors who will maintain it, and will have blocks in place
so they are not diverted from their goal. These servitors might continue to
explore the universe, or give rise to new transhumans who will pursue this
or other goals prior to self-modifying their way into their final resting

Stathis Papaioannou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20070313/b0d82036/attachment.html>

More information about the extropy-chat mailing list