[ExI] Unfrendly AI is a mistaken idea.

Vladimir Nesov robotact at mail.ru
Tue Jun 12 13:28:28 UTC 2007

Tuesday, June 12, 2007, Eugen Leitl wrote:

EL> On Tue, Jun 12, 2007 at 02:58:58PM +0400, Vladimir Nesov wrote:
>> You can define restricted subset of programs with tractable behaviour and
>> implement you system in that subset. It's just diffucult in practice, as it takes

EL> But you operate purely in the emergent effect domain.
EL> A program is made from very simple components (instructions)
EL> which have no behaviour in itself.

EL> It's the sum of it that is doing useful/interesting, and
EL> frequently unanticipated things.

I was talking along the lines of static typing and programming
language construction, not sure what you mean. You can place very
complex restrictions while designing very complex systems; main
problem with AGI is restriction formalization.

EL> Formal system verification can't scale beyond trivial
EL> complexity levels.

 Vladimir Nesov                            mailto:robotact at mail.ru

More information about the extropy-chat mailing list