How Small Is Small? – Scientific American

Quantifying things is an overarching goal in science, but recent events have left me pondering just why that is. After all, we’ve long known that numbers can be precise but inaccurate. The textbook exemplar is Lord Kelvin, the 19th-century British physicist who insisted that unless you could quantify a thing, your knowledge of it was of a “meager and unsatisfactory kind.”

Yet he was responsible for one of history’s most infamous examples of a fallacious quantitative argument. It concerned the geologic principle of uniformitarianism, which states that processes we can observe can be used to interpret Earth’s history. Kelvin insisted it was wrong because it had led geologists to conclude that Earth was billions of years old. Kelvin’s own calculations ended up placing the age at no more than 20 million to 40 million years; ergo the methods geologists had used must be faulty.

Kelvin’s paper on the topic is perhaps the most arrogant in the history of science: the body of The “Doctrine of Uniformity” in Geology Briefly Refuted is one paragraph long, with one additional paragraph laying out his calculations. But it was Kelvin who ended up being refuted. He had made (at least) two faulty assumptions: one, that Earth began in a molten state and has been cooling ever since, and two, that there is no source of heat other than that left over from the planet’s formation. (Today scientists think our planet began as gas and dust; we know that radioactive decay is a major source of heat and that Earth is 4.5 billion years old.)

Calculations can never be better than the assumptions that guide them, but if numbers are fallible, then what exactly is their point? One answer is that they give us the ability to compare one thing against another using a common scale. It’s a whole lot easier to compare 1,000 to 100 than it is to compare “canoe” to “pineapple.” Another is specificity: we know how much larger 1,000 is than 100, but it’s not necessarily clear what we mean when we call something simply “large” or “small.” Indeed, two recent scientific debates both hinged on the question “How small is small”?

The first is about the risks of eating red meat. Last year a group of researchers argued that Americans should not bother to reduce their consumption of red and processed meats, because any health benefits would be “small.” The second is about the impact of fossil-fuel subsidies. A paper in Nature examined whether eliminating them would have much of an effect on carbon dioxide emissions. The authors concluded that the effect would be “limited.” (Full disclosure: I am an author on a paper challenging that finding.)

How are we to judge these claims? Critics have a point when they suggest that we should not expect people to change their lives—or governments to change their policies—if the benefits of those changes have not been demonstrated or if they have been demonstrated but are insubstantial. But how do we judge what is substantial and what is not? In the case of red meat, the authors wrote: “Dose-response meta-analysis results from 17 cohorts with 2.2 million participants provided low-certainty evidence that decreasing unprocessed red meat intake may result in a very small reduction of overall lifetime cancer mortality (7 fewer events per 1,000 persons with a decrease of 3 servings/wk).”

Is seven fewer cancer deaths per 1,000 people “very small”? In a population of 331 million Americans, that’s 2.3 million people! In comparison, as of late March this year, the Centers for Disease Control and Prevention estimated flu deaths in the 2019–2020 season (which are calculated separately from those caused by COVID-19) at 23,000 to 59,000. It would be accurate—and perhaps more informative—to say that eating three fewer servings of red meat a week could save 100 times more lives than eliminating a year’s worth of deaths from seasonal flu.

In the case of fossil fuels, the authors wrote: “Subsidy removal would lead to a small decrease in global CO₂ emissions: 0.5–2 gigatons … This is much less than the Nationally Determined Contributions (NDCs) from the Paris Agreement, which add up to a decrease of between 4–8 Gt from fossil fuels and industry.” Yes, a range of 0.5 to two is less than four to eight, but is it “much less”? Maybe, but the authors would have been equally correct if they had said that subsidy removal would amount to at least 6 percent of the Paris commitments and possibly as much as 50 percent.

Admittedly, scientists are often admonished to speak in plain English, and “small” and “very small” are plain English. But when the numbers they refer to are not small, they become misleading.