[ExI] Terrorist? Who can tell?

[Harvey Newstrom]

"Lee Corbin" <lcorbin at rawbw.com> wrote,
> Harvey writes (forgive me for chopping up your email and replying to
> various parts, perhaps out of order)

That's what e-mail is for.  :-)

> Well, to keep the issues straight, you are suggesting that
> if a known criminal X (of whatever race) passes through
> an airport that today's present face-recognition programs
> are pretty much useless?

Yes.  Because of the statistics I exemplified, the system will not only spot 
the known criminal, but a large number of people who are not wanted.  They 
will waste their time with 10,000 unwanted people for every watned one. 
Over the course of a year, that means they will detain 27 innocent people 
every day for a year before they get one wanted guy.  The system won't last 
that long. They will yank it and declare it doesn't work.

I'm not just theorizing this.  We tried to use such a system in Florida at 
our Bowl game a few years ago to catch wanted criminals and deadbeat dads 
(not paying child support).  Bowl security was swamped with hundreds of 
false positives.  They did not find a single wanted person, even though 
there probably were some in the crowd.  They then theorized that pre-bowl 
publicity kept criminals from going to the game.  So they deployed this in 
Tampa's Ybor district and used it for a whole year.  Not one real 
recognition was made.  All they got were false positives.

Google for "florida super bowl face recognition"
<http://www.google.com/search?q=florida+super+bowl+face+recognition>

> (Whereas human spotters, I presume, are not at all useless.)

I don't have evidence for this, but anectodally, humans were not much 
better.  The system matched exact facial dimensions, but got gender, height, 
weight, race, and other obvious traits wrong.  But if the criminal had 
altered their appearance, with beard, hair colr, different hair-cut, 
glasses, etc., the computer would still match it but the human double-check 
would likely think it was a poor match.  In general, I don't think facial 
recognition (human or computer) works that well.  Detecting fingerprints, 
DNA, electronic IDs, vehicle registrations, RFID tags, and other exact 
matches would probably work better.  As well as actually trying to track 
down a suspect to their actual location rather than randomly looking around 
and expecting them to walk by.  Again, it's all statistics.  The number of 
non-matches just overwhelms the (human and machine) systems.

> Or, contrariwise, do you mean terrorist-spotting software,
> e.g. which for argument's sake say are quite effective at
> distinguishing Middle-Eastern young men from Indian young
> men, are useless because of these statistical facts you adduce?

Yes.  Such software is notoriously poor at recognizing race, which cannot be 
precisely defined between groups.  Even a small 1% overlap between racial 
groups can cause the statistical large number of false positives.  I can't 
find the rigorous studies I have seen, but I found a couple of examples new 
to me that describe the difficulties.
http://www.anonequity.org/weblog/archives/2007/06/are_biometrics_raceneutral.php
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1770976

> And (on the same point as this last paragraph) terrorist-spotting
> humans at, say the Tel Aviv airport---using every clue they can
> ---get way too many false positives to be of any use?

Tel Aviv does not rely on racial profiling or other profiling like the US is 
trying to do.  (They may use it, but it isn't key.)  They screen every 
single bag.  They screen every single passenger.  They lock passengers out 
of the cock-pit.  They have strong security unstead of underpaid TSA 
contractors.  And most importantly, they profile behaviors or dangerous 
materials they want to detect, and not secondary characteristics like race 
or religion that a very large number of innocent people have.  Thus, they 
don't get this massive list of false positives.

Tel Aviv has the strategy of keeping dangerous materials off planes by 
looking for those materials, and keeping suspicious people off planes by 
detecting suspicious behavior.  The U.S. strategy seems to be trying to tell 
who is guilty by their past credit histories and what they look like, while 
doing a poor job searching their bags or watching their current/actual 
behavior.  The former strategy is more direct to the goal and more 
objectively measurable than the U.S. strategy.

> Also, let me weaken that entirely separate claim a little bit to two
> questions, A and B:
>
> A: "Are telling me that a row of six or more recent convicted
> terrorist bombers could not be distinguished at the ninety-percent
> level of confidence from a numerically similar row of Londoners
> picked at random"  Surely you agree that I'm right about *that*,

No, I don't agree.  As I showed with my statistical correlation, that 
false-positives overwhelm the systems a thousand times over.  How would you 
distinguish them?  Are you saying they look different?  Are you saying they 
act different?  Do you suppose there are forensics traces of dangerous 
materials in their clothes?  What do you claim to be looking for that could 
be distinguished?  Race isn't enough, because there are a lot of 
middle-easterners in London.  Religion, language, even hatred toward other 
groups is not a good predictor of terrorism.  Could you be more specific 
about what exactly you think you could see?  There is a reason we have 
trials with evidence and just don't convict people because they "look" 
guilty.

> B: "are you telling me that a row of six or more recent convicted
> terrorist bombers could not be distinguished at the ninety-percent
> level of confidence from a numerically similar row of Londoners
> of completely matching age and sex?"

I explained how a 9.9999%  accurate system with only 0.0001% error rate 
still isn't good enough.  Your 99% or 90% aren't even in the ballpark.  I 
know it sounds counter-intuitive (as with much of Baysean statistics).  But 
if you work the numbers out of your random sampling, you simply won't get 
correct answers more than you get wrong answers.  The numbers just don't 
work out most of the time.

(Being in the security field, I really am interested in new ideas and 
schemes that might work.  But many of the "obvious" schemes have already 
been tried and failed.)

--
Harvey Newstrom <www.HarveyNewstrom.com>
CISSP CISA CISM CIFI GSEC IAM ISSAP ISSMP ISSPCS IBMCP