[ExI] Unfrendly AI is a mistaken idea.

[Vladimir Nesov]

Tuesday, June 12, 2007, Eugen Leitl wrote:

EL> On Tue, Jun 12, 2007 at 02:58:58PM +0400, Vladimir Nesov wrote:
>> You can define restricted subset of programs with tractable behaviour and
>> implement you system in that subset. It's just diffucult in practice, as it takes

EL> But you operate purely in the emergent effect domain.
EL> A program is made from very simple components (instructions)
EL> which have no behaviour in itself.

EL> It's the sum of it that is doing useful/interesting, and
EL> frequently unanticipated things.


I was talking along the lines of static typing and programming
language construction, not sure what you mean. You can place very
complex restrictions while designing very complex systems; main
problem with AGI is restriction formalization.

EL> Formal system verification can't scale beyond trivial
EL> complexity levels.

-- 
 Vladimir Nesov                            mailto:robotact at mail.ru