Tuesday, December 29, 2009

How About This?

After reading far too many op-eds righteously urging us to like having strangers look at our naked bodies, harumph for the good of the nation harumph, and after reading many comments from presumably ordinary people saying that the additional requirements are convincing them never to fly again;

And after considering that those scanners are going to cost a great deal of money and that we are spending much less to prevent 40,000 or so deaths a year in automobile accidents;

And after wondering why metal-detector wands are being used in an attempt to detect explosives, which they do not, and why a terrorist cannot blow up an airplane 70 minutes before landing, and whether the transatlantic planes will be warm enough for me to sleep without a blanket;

Now therefore I do suggest that we suck it up (in the phrase of those righteous harrumphers) and accept the fact that there is a 1 in 10,408,947 chance that we will be involved in a terrorist incident as we fly. Not that we will be killed, because Nate Silver's statistics include the shoe bomber and the ball bomber. Just that you might have a nut on your plane who sets his privates on fire. That's a lot better odds than that you'll die driving to the store today to buy some New Year's cheer.

15 comments:

Andy said...

So terrorism can be reduced to mere probability? The problem with that is that terrorists get a vote and if one doesn't adapt when they find a weakness in your security then your rate of incidents is going to jump.

Cheryl Rofer said...

What's wrong with probability? And what's wrong with identifying the weakness that allowed this incident instead of locking everyone in their seat for the last hour?

This guy was in his seat when he set himself on fire. And I suspect he could have done that at the seventy-minute mark. So locking everyone in their seat puts them in the position he was in, and the time limit is as irrelevant as the erroneous belief that TATP can easily be mixed up in the restroom.

Probability matters because the last 20% of precaution usually costs as much as the first 80%. And tracking down that last one-in-ten-million costs a lot more. Are more delay, more wands that don't detect explosives, more people sniffing through your luggage going to prevent another incident? Probably not.

Andy said...

Cheryl,

Probability is a good tool for determining the likelihood of random events but terrorism is not a random event. Dave Schuler explains it well I think.

I agree that many of the reactionary security measures are stupid - made by people who feel it's politically necessary to be seen doing something - but that doesn't mean we can simply declare that terrorist attacks against aircraft are low probability events and therefore require no changes in security.

Cheryl Rofer said...

It's not clear how much of a strategy al-Qaeda is using any more. And if we don't know their strategy, probability is as good as we've got. Certainly for any airline passenger, it's as reliable as figuring the probability that someone who doesn't care to slow down for snow will destroy me and my car today.

The 80-20 rule still holds. A number of people have pointed out that we will have no terrorist incidents on airplanes if we stop flying airplanes. I think most of us are not willing to pay that price.

A rational assessment of al-Qaeda's strategy (or whether this was an individual nut or whether he is from some other organization) and response makes sense. What TSA has been doing the last few days doesn't.

Andy said...

Cheryl,

It doesn't appear to me AQ's strategy has changed much over the years WRT to targeting airplanes, but when discussing security measures, it's the tactics, not strategy, that matter. When an adversary finds a tactic that works, they are likely to use it again which is why security measures need to be flexible and adaptable. If security flaws aren't addressed then your adversary will exploit them, resulting in more attacks and throwing simplistic probability calculations out the window.

That's why the comparison to car accidents is a very bad one. The person who may speed and hit you on a snowy road isn't intentionally targeting you. And, using simple probability, the chance of a 9/11 style attack before 9/11 was zero. It's for these reasons and others that security decisions do not and should not depend on simplistic probability calculations.

Cheryl Rofer said...

It's not an either-or situation.

There is a long list of oversights and mistakes already identified with regard to letting the underpants bomber through. Until al-Qaeda gives us their playbook, dealing with those should be a step forward.

But if you're a government official trying to allocate limited resources, you have to ask whether they will give a better effect in one place or another. So statistics are part of that: more lives saved by better traffic management or by installing see-you-nekkid machines in airports?

Trying to prevent every terrorist act is stupid. It leads to the very hysteria and disruption that the terrorists are trying to promote.

Andy said...

Cheryl,

You say it's not an either or situation then then you say this:

"But if you're a government official trying to allocate limited resources, you have to ask whether they will give a better effect in one place or another. So statistics are part of that: more lives saved by better traffic management or by installing see-you-nekkid machines in airports?"

...which is and either-or argument. Also, consider that the same flawed argument could be used regarding murder or child rape or any number of other "low-probability" crimes in comparison to traffic accidents or heart disease or cancer.

And personally, given the choice between a machine that creates weirdish "nekkid" picture of me and having an actual person pat me down and grope me (which has happened to me a couple of times in airports) I much prefer the former.

Peter said...

And, using simple probability, the chance of a 9/11 style attack before 9/11 was zero."

Not true. Look up Bojinka.

Andy said...

Peter,

Except the Bojinka plot failed. Even if one includes it in a probability analysis, how can you calculate a reasonable probability for a 9/11-style attack when your dataset consists of one incident? You can't, which is kind of the point.

Cheryl Rofer said...

Andy - It's not an either-or argument as to whether one uses statistics or tries to figure out the strategy. If you have limited resources, yes, then it's an either-or decision as to what to do with them. But those are two different either-ors.

Terrorism is a shrewd strategy because if the victim doesn't do anything, then the terrorists win. If the victim is terrorized, then the terrorists win. So the one-percent fallacy, that ultimate and complete protection is the goal, is a win for the terrorists because it sucks up resources and terrorizes citizens.

The smart response to terrorism is prudent protection and rationality. That sometimes involves figuring up the probabilities and acting on them.

Andy said...

I guess where we disagree is that you believe that such probabilities can be calculated whereas I believe they can't. For example, we can predict with reasonable precision and accuracy the number of traffic-related deaths from year to year based on probabilities because we have a large dataset and factors that affect the probability are relatively transparent.

Conversely, we have no idea based on probabilities if there will be an attack, for example, in 2010, much less where it will occur, what form it will take or how many people it will kill. History is not a reliable indicator, there is not enough data, terrorist planning is opaque, and, most important, there is a fundamental difference between accidents and criminal intentions in terms of probability. Probability is, therefore, of extremely limited utility for determining what is prudent and rational because probability calculations regarding terrorism are essentially guesstimates. Using them to form policy is wishful thinking. And that's another reason why terrorism is a shrewd strategy - it's unpredictable.

Peter said...

Except the Bojinka plot failed. Even if one includes it in a probability analysis, how can you calculate a reasonable probability for a 9/11-style attack when your dataset consists of one incident? You can't, which is kind of the point.
First of all, I'm not at all familiar with statistical techniques, nor am I a professional analyst or anything like that. So take my comments with a generous pinch of large-crystal sea salt.

Just because Bojinka failed (or failed to be realized, which is another matter), doesn't mean it didn't have an impact on the future moves of the opponent (to co-opt the lingo of game theory). In the same way that security policy adapted in response to the 'failed' attempt of Richard Reid, the opponent adapted as well: Don't even try to ignite a spark on a plane - too many people will notice too quickly. Result? They go the binary chemical route.

The point is, you can't ignore 'failed' attempts in an analysis because both sides learn from them, just as both sides learn from 'successes'. Again - I'm no nothing at all about statistics or threat analysis, but it seems to me that both sides learn from failures as well as non-failures.

So I'd imagine that if one were to try to calculate the probability of a another plane-based act of terrorism, one could not logically exclude ShoeBomberGuy, UnderwearBombGuy, 9/11 techniques, nor even Bojinka. There's a reason they hardened cockpit doors, after all.

Cheryl Rofer said...

Peter makes a good point: hardening cockpit doors and vigilant passengers are defense against many possible tactics. And they've worked so far, aided by terrorist incompetence.

So you don't need to know details of terrorist strategy.

And those numbers of Nate Silver's are pretty big; I don't see why they're not just as good as the automobile statistics. After all, there's been air terrorism since the 1970s.

You can consider both air terrorism and automobile accidents from a probabilistic viewpoint, as I've done, or you can consider whether Mr. A, who believes he is invulnerable in his big SUV but doesn't realize that it doesn't brake well on snow will be on the road today, which is the sort of thing you're advocating for judging the likelihood of terrorist attacks, Andy, and saying that's the only way to judge terrorist attacks but probability is fine for automobile accidents. Both situations can be analyzed either way.

It's this idea that there is something special about terrorist attacks that encourages the sort of overreaction we've seen from the TSA these last few days.

Andy said...

Peter,

I agree with your comment - that's real analysis and is far different from using fictional probabilities. Game theory is a valid analytical tool in this case. My point is that probability in this case is, at best, deceptive and, at worst, completely meaningless.

Cheryl,

You can consider air terrorism from a probabilistic viewpoint, but don't delude yourself into believing it is an accurate predictor of the future or what the "real" odds are. To begin with, there simply haven't been enough attacks to provide any kind of reliability - Nate uses 6 total which is not nearly enough. To make probabilistic predictions one needs sufficient quantitative data to establish a pattern. If you can explain how it's possible to make an accurate quantitative probability forecast based on six data points then I'd like to hear it!

It's akin to predicting war - tell me, what is the probability that the US will be involved in another major war next year? Well, simple probability tells us it's about 5% (12 major wars in 233 years). That would be fine if wars were random events, but they're not. Neither are terrorist attacks. There is a reason we use indicators to predict war and not quantitative probability.

Here is an equally flawed probabilistic prediction: There will be 7 terrorist attacks on US commercial aircraft during the next decade based on the fact that we had six attacks on aircraft in the last eight 8 years, or a 75% chance per year. 2/3 of those will be successful attacks (since 4 out of the six since 2001 were successful), so next decade we can expect to lose 4-5 US airliners. Can you see how flawed such analysis is?

And this is leaving aside the fact that people have a lower tolerance for acts of intent (crime, terrorism) than they do for random acts like lightning and car accidents. In other words, people aren't strict utilitarians who see death by car accident and death by terrorist attack as equal, so resource allocation is never going to match statistical probability.

Cheryl Rofer said...

The trouble with treating terrorism as a special case is that's exactly what the terrorists want.

And without trying to quantify, we leave the argument open to the emotionalism of the one percent doctrine - that we must expend all our resources to prevent one improbable occurrence.

So I've made all the arguments I can think of. We disagree on this, Andy.