[ExI] Unfrendly AI is a mistaken idea.
eugen at leitl.org
Tue Jun 12 12:37:29 UTC 2007
On Tue, Jun 12, 2007 at 02:58:58PM +0400, Vladimir Nesov wrote:
> You can define restricted subset of programs with tractable behaviour and
> implement you system in that subset. It's just diffucult in practice, as it takes
But you operate purely in the emergent effect domain.
A program is made from very simple components (instructions)
which have no behaviour in itself.
It's the sum of it that is doing useful/interesting, and
frequently unanticipated things.
> many times over in work, training on the level you can't supply in
> large quantities, and slower resulting code. And it probably can't be
> usefully applied to complicated AI (as too much is in unforeseen data, and
> assertions you want to check against can't be formulated).
Precisely. Formal system verification can't scale beyond trivial
complexity levels. Formal system verification is absolutely
useless in real-world AI, unless you're operating on the formal
domain to start with.
More information about the extropy-chat