[ExI] Artificial Battles

[Anders Sandberg]

In Sweden you also have a real thesis defense, where not only a board of 
professors judge the dissertation and your explanations, but an opponent 
who at least in theory will try to ask hard questions. Of course, this 
being modern Sweden, this is not too adversarial - you don't go up for a 
defense unless you have a sufficiently good thesis, and the opponent 
usually doesn't go for the jugular. Back in the day it could be far more 
tough, and in the old system you also had a "second opponent" to give 
you help and sometimes a "third opponent" to make jokes to keep people 
awake (kid you not).

There is something relevant here: in order to bring out quality, you 
need not just encouragement but also honest criticism - especially of 
important weak points. And that requires somebody who has incentives to 
be honest and cut to the bone, not just keep people happy. Security is 
best tested by red teams who actually try to break it, science is 
advanced by people testing theories to the breaking point.

The problem is the incentive structure. Most bosses do not like being 
shown their mistakes by red teams, and academics will often build 
strange coalitions to review papers nicely. The institutional frameworks 
we build should be aware of *why* they are supposed to reward artificial 
battles, so that they can try to avoid accidentally rewarding the wrong 
behavior.

Markets are typically not *intended* to produce innovation and 
efficiency, it is just that they tend to do it under a range of 
circumstances. We can set up institutions to favor beneficial 
competition (patents, SEC, antitrust regulations, etc), but again these 
institutions need to be kept on the track. That in turn requires further 
criticism and artificial battles - and hence other institutions. It is 
checks and balances all the way down.

As for academia, I wonder if we shouldn't have a duty to act as 
wandering opponents who show up unnanounced at random offices and ask 
pertinent questions from time to time. Eric Drexler has suggested that 
we might have a "duty to respond" to claims we hear, making sure that 
each intellectual encounter leaves a bit of logged trace data.

-- 
Anders Sandberg,
Future of Humanity Institute
Philosophy Faculty of Oxford University