[ExI] How could you ever support an AGI?

[Henrique Moraes Machado]

Richard Loosemore> Unfortunately, I think I was not clear enough, and as a 
result you have
> misunderstood what I said in rather a substantial way.
> When you build an AGI, you *must* sort out the motivation mechanism
> ahead of time, or the machine will simply not work at all.  You don't
> build an AGI and *then* discover what its motivation is.

Let me throw more wood into the fire then.
I am not afraid of AGIs at all. But let's imagine a scenario where some 
humans begin to worship the AGI as some sort of deity (yes, humans can be 
that messed up). A sect could quickly evolve around the AGI and THAT is 
something I consider dangerous. Not the AGI itself, but those possible post 
modern religious fanatics. The difference between these new fanatics and the 
present day fanatics that we all know and don't love is that their god 
really exists (I'm sure there's already some distopic scifi novel that has 
already addressed this issue).