October 1996 € Volume 23 € Number 5

Flying Scared
How Much Spending on Safety Makes Sense

By Robert E. Machol

The July 28, 1996, issue of Time magazine shows heart-rending photographs of 38 of the 230 people who died when TWA 800 exploded. It is not inappropriate to mourn these people; but to put their deaths in perspective, note that on any given weekend we kill that many people on our highways and injure thousands more. Life is risky. Millions of Americans die every year. But America has a morbid fascination with any incident that involves a lot of people dying in one place at one time; and if the incident is an airplane crash, America as a whole becomes hysterical. It is not the purpose of this article to be cynical, but rather to take a rational, analytic view toward the questions that are facing us all: Should we continue to fly? Should we do more about preventing aircraft accidents? What should we do about terrorism?

A long time ago I published in the OR/MS journal Interfaces an article entitled "How Much Safety?" [Machol 1986]. I strongly commend that article to readers of this one &emdash; I'm not much for higher mathematics, but there is a lot of interesting arithmetic in it. The Interfaces article began with the following quote from Airline Pilot magazine, a journal published by the Air Line Pilots Association:

"The issue is not whether you have 10 B-747s operating in or out of an airport in one hour or one that comes in once a week. CFR (Crash/Fire Rescue equipment) requirements should be based on the need to protect passengers on the largest aircraft operating into that airport [Moorman 1986]."

In my opinion, the assertion is wrong. As I wrote in the Interfaces article, "(The assertion) stems from the feeling that human life is priceless, and therefore no expense is too great if it has any possibility of saving lives. Alas, though many people feel that way, it is not a viable approach to system design in a world of finite resources."


$1 Billion for Every Life
Some of the arithmetic in the Interfaces article shows that, unbeknownst to himself, Moorman was recommending an expenditure in excess of $1 billion for every life saved. I feel that such an expenditure is not justified. It should be clear that we are talking about a statistical life, not yours, or mine or any one in particular. If little Suzie is stuck in the bottom of a well and in danger of her life, we will willingly spend millions of dollars and get three people killed getting her out.

Three years ago, I sent a memo to the administrator of the FAA saying: "You have stated that safety is the most important consideration for you as administrator of the FAA, and that it cannot be compromised for other considerations. Every other administrator has said the same thing, as have DOT secretaries, airline presidents, etc. None of you has any choice about saying this. The danger is that you might believe it."

It was made clear to me that he was not amused.

So, for a start, what should we do about airport safety and the detection of explosives? Some years ago we had a lot of hijackings; we solved that problem, mostly with a simple technology that prevents people from getting guns onto airplanes. So why do we not have equipment that prevents people from getting bombs onto airplanes?

Nearly 10 years ago, largely through FAA funding, a company developed an explosive-detection system called TNA, for Thermal Neutron Activation. Later it was realized that "Activation" in connection with nuclear energy might scare people, so the name was changed, but to save the acronym the name chosen was Thermal Neutron Analysis. (If the FAA would give as much attention to real safety as to this kind of PR, we might be better off.)

It was a big device weighing many tons, and costing about a million dollars per device. It had high levels of nuclear energy contained by heavy shielding. It could detect nitrogen, and there are essentially no useful explosives that do not have a great deal of nitrogen. One of the problems is that lots of other things &emdash; woolen sweaters, for example &emdash; also contain lots of nitrogen. Nonetheless, the FAA was on the verge of requiring every airport to install TNA equipment.

I went to the California factory where the TNA was being developed and took along live explosives to perform an adversarial test. As every OR worker knows, the best way to prove that something works is to try as hard as you can to prove that it doesn't work, and hope that you fail. We put the explosives into several dozen of some hundreds of pieces of typical luggage &emdash; nobody but us knew which pieces &emdash; and put them through the TNA. It correctly detected about 85 percent of the explosives, and had some 15 percent false alarms (a "false alarm" means that you think you have detected explosives when no explosives are present). It might be assumed that further work would improve these numbers, but they are nowhere near good enough. I personally didn't worry too much about not detecting all the explosives. If you've got a system that detects explosives most of the time, the bad guys won't try to pack bombs in their luggage, assuming we continue to match baggage to passengers as we now do on international flights, and as we probably will on domestic flights by the time this article is published. (This is a case where we willingly accept extra expense and extra delay in exchange for extra safety &emdash; but as OR people know and many others do not know, the key questions are: How much expense? How much delay? How much safety?)

I do worry about the false alarms. Suppose you get false alarms down to 5 percent. If you have a 747 going out with 600 bags &emdash; then you will "detect" explosives in 30 of them; at which point you must track down the owners and have them open their bags. And where will you do this opening? Right there by the gate when you think there is a bomb in the bag? Or take all 30 bags out of the airport to open them while the plane waits?

The manufacturer of TNA asserted, and had demonstrated, that it could deal with 600 bags an hour. In practice, we found that it dealt with considerably fewer because, for example, big bags are mixed with little bags, some bags don't have handles, and so on. (This kind of practical observation will not come as a surprise to anyone who has actually done OR.) How many TNAs do you need at an airport like JFK that can put out 20 or 30 747s an hour during the evening? Remember, each is enormous, weighs tons, has a lot of radioactivity, and costs a million dollars. And you probably can't examine Delta bags at an Air France counter, or vice versa. (Queue theory is relevant here.)

So if anybody asks you why we don't have explosive-detection equipment in our airports today, you can tell them that Machol is partly responsible.


New, improved techniques
Since that time the FAA and some foreign countries have done excellent work in developing new and improved techniques for detecting explosives. One of these techniques is now being tested in a couple of U.S. airports, and several are in use in other countries. This is not the place to describe these techniques or evaluate their efficacy; that efficacy will improve with time. X-rays may be useful, especially if combined with an efficient CAT scan system, but X-rays cannot distinguish explosives from other materials. Most explosives give off vapors that can be detected with appropriate sniffers if the explosives are not very tightly wrapped and if there is some suction or blower that brings the vapors to the sniffer. Some airports put checked luggage in a vacuum chamber to simulate going to altitude &emdash; since the easiest way to fuse a bomb for destruction of a jet is to have it go off when the jet reaches altitude &emdash; but it isn't very hard for a bad guy to set the fuse to go off the second or third time the plane gets to altitude.

Just remember that there are difficulties with these techniques, some of which are indicated above (e.g., how do you deal with false alarms?); remember that the fact that a technique is in use in some other country doesn't prove that it is actually useful; and remember that it probably would be a mistake to rush into the installation of billions of dollars worth of doubtful equipment to help some congresspersons get re-elected. The public is clamoring for action, and politicians tend to pander to this kind of thing. Note that Congress passed a law a few years ago mandating bomb-detection equipment by a certain date, but the FAA proved (predictably) unable to invent on demand.

Apart from airport security, how should we assess the safety of the aviation system, and how should we assess the FAA's performance? In the "How Much Safety" article in Interfaces [Machol 1986], we boasted that we had fewer than one fatality per billion passenger miles; I now realize that this is not a good measure. Barnett [1990] shows that the probability of getting killed on a randomly selected flight is the best measure. It comes out to about one in five million for scheduled commercial jets of any of the developed countries. For the undeveloped countries it may be 10 times as great (personally, I would be willing to face a probability of death of one in 500,000 if I were in a hurry, but many people might not feel that way). Nonscheduled and propeller-driven aircraft may be more dangerous; certainly very small aircraft such as air taxis and private planes, especially private planes flown by bold pilots, are more dangerous. (There is an old saying in aviation: There are old pilots and there are bold pilots, but there are no old, bold pilots).

Scheduled commercial domestic jets fly roughly 500 million people a year and kill about 100 of them, or one in five million (1996 has been an outlier &emdash; although TWA 800 was technically not a domestic flight). There are very few activities you can undertake that are as safe. More than 100 people a year drown in bathtubs. (I don't have an actual source for that number, but I've heard it for years, and it does give pause). Most importantly, flying is much safer than driving. And that's why I am strongly opposed to the proposed infant restraint-seat rule.


Infant Restraint-Seat Rule
The FAA now requires children over two years of age to have a separate seat, but those under two are permitted to be held on the lap of an adult, thereby saving the cost of an extra seat. The deceleration in a crash such as the DC-10 at Sioux City in 1989 causes so much force that the adult cannot hold the child. The child may be torn from the arms of the adult and slide down an aisle into a part of the plane which is afire. One child did, indeed, die needlessly in that crash, and now there is enormous political pressure to require every child to be strapped into an infant-restraint seat in his/her own private seat. Never mind that a tiny baby is going to be happier in a parent's lap.

The best analysis I know showed that if this requirement were enacted, 80 percent of the two million under-two children who now ride free each year would fly in a purchased seat, but the other 20 percent would drive, along with their families, in order to save money. That's an extra million people a year driving. And because driving is so much more dangerous than flying, a dozen people would die and many more would be injured for every infant life saved on an airplane.

Pro football TV analyst John Madden is well known for traveling back and forth across the country by bus because he is afraid of flying, although driving is more dangerous than flying. But comparing the safety of flying and driving is a very complicated business [Barnett, 1991], and all tied up with emotional considerations. I have always defined "risk" as a measure of the probability and severity of harm to human health, while "safety" is a subjective assessment of risk. If Madden is more comfortable driving, then maybe for him it is safer even if it is more risky. Furthermore, his large vehicle, driven by a professional chauffeur, is doubtless much safer than an ordinary auto. Another point: danger in driving is roughly linear with distance traveled, while in aviation danger is virtually independent of length of flight. In driving, danger depends on day vs. night, on age of driver, on speed, on size of car, on alcohol, and on and on. One needs to look at the OR/MS literature &emdash; e.g., articles by Barnett &emdash; to better understand these issues.

Of course the system should be as safe as possible, and I have had quarrels with the FAA about things I thought should be done to increase safety. For example, for years I urged the FAA to increase the distance of smaller planes behind B-757s because I predicted that otherwise accidents would occur when the trailing plane was caught in the wake vortex of the 757. Only after two accidents (killing 13 people) occurred exactly as predicted did they finally increase that separation. I still feel that the wake-vortex issue is not being handled optimally. But this is not the place to take up such questions.

On the overall question of safety, people must realize that if you put enough weight and cost into making it safer (for example, requiring a defibrillator on every flight in case a passenger suffers a heart attack, as has been seriously suggested) eventually the aircraft becomes a "lead sled" which is perfectly safe but is too heavy to take off unless you remove all the passengers.


White-knuckle fliers
It is of interest to note that white-knuckle fliers tend to worry about the wrong things. In the past they have worried about mid-air collisions, and today they also worry about sabotage. Mid-air collisions are common enough between small aircraft at untowered airports [Machol 1979], but they just don't occur with domestic scheduled commercial jets &emdash; we haven't had such an accident in decades. (There was a midair collision over Cerritos, Calif., in 1986 involving a Mexican commercial jet.) The reason we don't have midair collisions is because our Air Traffic Control (ATC) system is extremely conservative and extremely effective. Sabotage is also very rare; we've had PanAm 103 and (possibly) TWA 800, but no accident that we know of to a scheduled domestic commercial jet due to sabotage. Mechanical failure, which people also worry about, is also rare.

The most common cause of accidents to scheduled commercial jets is "controlled flight into terrain" in which the plane is flying along and, without mechanical failure or trauma to the pilot, or for any one of a dozen different reasons, slams into the ground [Machol, 1992]. Because the pilot doesn't think he is that close to the ground in such cases, the plane tends to be going at high speed &emdash; much higher than for landing &emdash; and so there tend to be lots of fatalities.

Transportation Secretary Federico Pena recently decreed for the first time that small planes (technically those with less than 30 passengers) must meet all the safety requirements of large planes. This may not make good sense. If you require, say, extra training for a pilot, this might make cost/benefit sense if the cost is split among 150 passengers, but might not if it is split among 15 passengers, each of whom must pay 10 times as much for this increment of safety.

I'm not trying to analyze here how much training a pilot needs, but pointing out some elementary principles of cost/benefit analysis. And apart from that, it has been proposed that people should be given a choice between low-priced safe airlines and high-priced ultrasafe airlines. I think this is a reasonable topic for debate.


Grading the FAA
And what kind of grade should we give the FAA? The FAA does have a sizable OR group; unfortunately, under the present administration, it has been used for other purposes and has not done a lot of OR. It should be noted that the administrator and the deputy administrator, the two top officials of the FAA, are nominated by the president and confirmed by the Senate, so they are both political appointees. However, some administrators have politicized the agency to the extreme, while others have tried to keep their activities as nonpolitical as possible.

Every administrator would prefer not having an accident to having an accident. The question is: Where on his priority list does safety rank compared to pleasing Congress and getting good treatment in the media? This has varied from one administrator to another. But regardless of the administrator, there are two fundamental questions that must be answered:

1. Which agency is ultimately responsible for aviation safety?
2. To what extent should the FAA be responsible for advancing and supporting the aviation industry (especially airlines and aircraft manufacturers)?

The latter question has been raised repeatedly lately because of the assertion that promoting the industry is incompatible with maximizing safety.

In 1935, a TWA DC-2 crashed, killing, among others, a U.S. senator. The CAA (predecessor of the FAA) investigated the accident and concluded that it was TWA's fault. TWA investigated the accident and concluded that it was CAA's fault. The Senate, which had a special interest because of the loss of one of its members, investigated the accident and decided TWA was right. Conclusion: You cannot have the agency that operates the ATC system investigate accidents which might involve fault in the ATC system. Accident investigation was subsequently given to the Civil Aeronautics Board (CAB).

Today we have the National Transportation Safety Board (NTSB) which investigates accidents. I have enormous respect for the NTSB. They make safety recommendations, but it is up to the FAA (which must worry about efficiency and cost, as well as safety) to choose whether or not to accept these recommendations. Usually it does, but if it fails to implement an NTSB recommendation it is often subject to severe criticism. The FAA should not be expected to implement every NTSB recommendation &emdash; ultimately the FAA, not the NTSB, is responsible for Air Traffic Control.


Dividing UP responsibilities
There is no ideal solution to this question of optimal division of responsibilities, but I feel that the present set-up is about right. Similarly, there is talk of splitting the FAA into two organizations, one of which would be responsible for ATC while the other would be responsible for safety. Note that the FAA has some 20,000 controllers (a few in towers, but many more underground looking at radar scopes), but it also has thousands of employees who certificate aircraft, pilots, spare parts, maintenance and dozens of other things, operate and maintain weather equipment, do R&D, and perform other safety-related functions. Once again, there is no ideal way of dividing up these responsibilities. I think that under a competent, apolitical FAA administrator, the present set-up is as good as any.

There are alternatives as to who does the research on such things as aviation safety. In the United States, a great deal of this is done by the National Aviation and Space Administration (NASA) rather than the FAA. Finally, there are different relationships of civil to military aviation. The FAA reports to a civilian, the secretary of Transportation, but in time of war, the FAA automatically comes under control of the Department of Defense. In other countries, the civil and military are much closer; in the United Kingdom, for example, much civil-aviation research is done in military facilities, and the controller of the CAA (the British equivalent of the FAA) was recently an air marshal in the Royal Air Force. In most of these cases there is no obvious optimal way to deal with such matters.

In conclusion, I think that flying is remarkably safe. I warn against hysteria stimulated by TWA 800. I think the FAA is spending about the right amount on safety, although I don't always agree on exactly how it is being spent. And I caution against rushing into poorly thought-out solutions which may do more harm than good.


References
1. Barnett, A., 1990, "Air Safety: End of the Golden Age?" Chance, Vol. 3, No. 2, pp. 28-32.

2. Barnett, A., 1991, "It's Safer to Fly," Risk Analysis, March.

3. Machol, R., 1979, "Effectiveness of the Air Traffic Control System," Journal of the Operational Research Society, Vol. 30, pp. 113-119.

4. Machol, R., 1986, "How Much Safety?" Interfaces, Vol. 16, No. 6, pp. 50-57

5. Machol, R, 1992, "Natural Hazards to Aviation," OR/MS Today, Vol. 19, No. 6, pp. 30-38.

6. Moorman,R., 1986, "Toward Safer Airports," Airline Pilot, No. 55, Vol. 2, pp. 10-13.


Robert E. Machol recently retired as chief scientist of the FAA. A frequent contributor to OR/MS Today, Machol is a past president of ORSA and a winner of its prestigious Kimball Medal for distinguished service to the society and to the profession of operations research.

For more information, put the number 3 in the appropriate space on the
Reader Service Form