The Planes That Came Home
In 1943, the military brought a refugee mathematician a map of bullet holes and asked where to add armor. He gave them the only answer in the room that could not be seen: armor the places without holes. Eighty years later, we still make his enemies’ mistake every single day, in hiring, in investing, in the stories we tell about success.
The office was on West 118th Street in Manhattan, a few blocks from Columbia University, and the men inside it were fighting the Second World War with pencils. The Statistical Research Group was one of the strangest weapons America ever fielded: a room of mathematicians and economists, several of whom would later win Nobel Prizes, doing ballistics and bomb-pattern arithmetic for the military the way other men riveted wings. Milton Friedman worked there. So did a quiet Austrian named Abraham Wald, and Wald’s presence in that room was itself a small miracle of escape. He had been a rising star in Vienna, a Jewish mathematician in a city that was about to make that combination lethal, and when the Nazis annexed Austria in 1938 he lost his position, his country, and very nearly his life, finding his way to America while most of his family did not find their way out at all. Five years later, the government of his new country handed him a problem soaked in exactly the kind of data everyone trusts most: data written in bullet holes.
The problem, as the military framed it, was armor. Bombers were coming back from Europe shredded, and armor is heavy; plate everything and the aircraft can barely climb. So the services did the obvious, rigorous thing. They surveyed the returning planes, mapped every hole, and produced a picture of where aircraft get hit: the wings peppered, the fuselage riddled, the tail chewed up. The engines, curiously, were nearly clean. The recommendation practically wrote itself, and it is the recommendation almost every person alive would still write today: reinforce where the holes are. Put the steel where the damage is. Follow the data.
Wald’s reply has become the founding legend of survivorship bias, and like most legends it is usually told too fast. The holes, he pointed out, had been counted only on planes that made it home. The survey was not a map of where bombers get hit; it was a map of where a bomber can be hit and still fly. German flak and fighter fire did not somehow avoid engines. The planes hit in the engines were resting at the bottom of the English Channel, invisible to the clipboard men, unable to contribute their holes to anyone’s chart. The clean patches on the diagram were not evidence of safety. They were the outline of the morgue. So Wald inverted the recommendation, armor the engines, armor the places the surviving planes never seem to be hit, and then he did something more impressive than the insight itself: he built the mathematics to estimate, from survivors alone, the vulnerability of the parts you never get to inspect, working through the arithmetic of downed aircraft in a series of memos that military analysts were still using decades later, in Korea and Vietnam.
What I love about this story is not that Wald was clever. It is that everyone else in it was also smart, careful, and diligent, and still walked straight into the trap, because the trap is not made of stupidity. It is made of visibility. The men with the clipboards did everything right except ask a single question, the question that sits underneath all honest analysis: what data am I not seeing, and why am I not seeing it? Their dataset was perfectly accurate and catastrophically incomplete, and the incompleteness was invisible precisely because the missing planes were, well, missing. No cell in a spreadsheet announces its own absence. The silence where evidence should be does not look like silence; it looks like a clean engine cowling, like good news, like a place you can save some steel.
Once you have seen this shape, you cannot stop seeing it, because most of what we call wisdom about success is a survey of returned aircraft. The business press studies companies that became huge and reverse-engineers their habits into commandments, without visiting the graveyard of companies that had identical habits and died, unphotographed. Twenty college dropouts built empires, so the dropout becomes a legend; the several million dropouts for whom it went otherwise hold no press conferences. Every fund that markets a dazzling ten-year track record is, by definition, a fund that survived ten years, and the funds that gambled identically and blew up in year three have been quietly merged, closed, and scrubbed, leaving an industry-wide performance record that flatters itself automatically, no dishonesty required. LinkedIn is a fleet of returning bombers. So is every conference stage, every memoir, every panel of founders explaining how conviction and courage carried the day. Conviction and courage are also the last recorded sentiments of a very large number of planes at the bottom of the Channel.
The financial version of this trap deserves special respect, because in markets the selection effect compounds. Strategies that quietly take enormous tail risk, selling insurance against rare disasters, in one form or another, look brilliant for years precisely because the disaster is rare. The surviving managers get bigger allocations, lever the lesson, and train a generation of imitators on a dataset from which every crash casualty has been politely removed. Then the engine takes its bullet. The remarkable thing is not that this happens; it is that after each blowup we return, almost immediately, to studying the survivors, because the survivors are available for interviews and the dead are not. Nobody sits down with the seventy-year-old who ran the same strategy as the legend, with the same discipline, and got hit in a different spot. His holes are the ones we most need mapped, and his is exactly the plane that never comes home to show us.
There is a discipline hiding in Wald’s memo, and it costs nothing but comfort. When you find yourself studying any collection of winners, a portfolio, a shortlist, a genre of advice, force yourself to write one sentence describing the process that assembled the collection, and then ask what that process filtered out before you arrived. Interview your rejected candidates a year later, the ones who thrived elsewhere are your clean engine cowlings. When a backtest looks flawless, ask which funds, which years, which delisted tickers the database quietly dropped. When a habit is credited for a fortune, go looking, deliberately, for the bankrupt who shared it. None of this comes naturally, because evolution built us to learn from what is in front of us, and the whole lesson of the bullet holes is that what is in front of us has already been chosen, by a process with its own agenda, before we ever started counting. The data never lies, exactly. It just only ever tells you the story of the ones who lived, and it tells that story in a voice so confident you forget anyone else existed.
Wald never saw his idea become famous. In 1950, at the height of his powers, he and his wife died when their plane crashed into the Nilgiri Hills in southern India, on the way to a lecture tour, an irony so heavy that writers on this subject tend to either lean on it too hard or omit it in embarrassment. I will simply say this: the man who taught the world to think about the planes that do not come home became one, and the discipline he left behind remains the cheapest edge in analysis, available to anyone willing to sit with the least natural question in the world. Not what does the evidence say, but who never made it into the evidence, and what were they trying to tell us on the way down.
Notes & sources: Wald’s wartime work with the Statistical Research Group at Columbia is documented in W. Allen Wallis’s 1980 retrospective in the Journal of the American Statistical Association and in Wald’s reprinted memoranda, “A Method of Estimating Plane Vulnerability Based on Damage of Survivors” (Center for Naval Analyses, 1980); biographical details from standard accounts of Wald’s life, including his 1938 flight from Vienna and his death in the 1950 Air India crash near the Nilgiri Hills; the survivorship diagram is by Martin Grandjean, McGeddon, and Cameron Moll, CC BY-SA 4.0; fund-survivorship effects are treated in the mutual-fund literature beginning with Elton, Gruber & Blake (1996).