[ExI] new testing with internet access

[Adrian Tymes]

On Sat, Feb 11, 2012 at 10:27 AM, Mike Dougherty <msd001 at gmail.com> wrote:
> Take the cleverer parts of proprietary code written by your existing
> pool of talent, ask applicants to explain it to you.  Take the top N
> candidates that did a good job explaining the code and ask them to
> introduce new features to the existing code.  After they've done this,
> let them find (and describe) flaws in the other applicant's code.
> When they're finished, you have a decision to make whether you hire
> the person who wrote flawless code or the person who identified the
> most potential problems in others' code.

Aye.  Don't ask people to do stuff that, in a real situation, they really
would be looking up (like, "describe how (language-specific feature)
works," when you're really trying to find out if they can solve problem
X with the language in question).

Problem is, you need to overcome a psychological hurdle of, "the
best candidate will have memorized all the definitions as well as
know off the top of his/her head how to answer any question".  This
is not something that even the interviewers themselves will be able
to do; they excuse it by looking for someone "better" than
themselves.

> If your test isn't for programmers, you might have a different puzzle
> to solve.  Time is probably the most realistic measure of information
> synthesis - sure you can have internet access, but do you know how to
> get the answers more quickly/easily than anyone else?  I've seen
> people browsing through pages of results rather than issuing a better
> search.  That's an unproductive use of clock-time.

That measures searching itself - which, yes, is a job skill, but can be
seen to not measure knowledge of the field in question.