A team of epidemiologists from across the country, led by researchers from Emory University, have launched an interesting defense of their profession in the June issue of the journal Science Advances with a critique of a story written by journalist Gary Taubes and published in Science magazine 27 years ago.
Whether this is a good thing or a bad thing is hard to say at a time when epidemiology is in the pandemic spotlight where it has become as deeply intertwined with politics as with public policy.
Taubes in 1995 took the profession to task for too often treating correlations as causations and in that way reaching sketchy conclusions as to what does and doesn’t threaten human health at the population level.
“He highlighted numerous examples of research topics he viewed as having questionable merit,” the epidemiologists now write in summarizing their critique of Taubes’ reporting. “Studies have since accumulated for these associations. We systematically evaluated current evidence of 53 example associations discussed in the article. Approximately one-quarter of those presented as doubtful are now widely viewed as causal based on current evaluations of the public health consensus.”
“Widely viewed” is an interesting modifier in the context of the study given that the standard falls short of “universally accepted.” But, accepting “widely viewed” as the standard, the summary of the paper slightly undersells the success of epidemiologists in successfully making educated guesses based on limited data.
The paper itself concludes that the now nearly three decades of data gathered on some of those old predictions show the epidemiologists of three decades ago batted about .270.
Good or bad?
Many Americans, of course, like to believe in foolproof science. Those screaming “listen to the scientists” during the height of the still ongoing pandemic certainly did. No one was chanting, “Listen to the scientists; they’re right a quarter of the time!”
Some of the scientists in power likewise didn’t want to accept the possibility that their directives might be less than perfect.
There were those, top American officials among them, who tried to maintain a public belief in some sort of infallible, high priesthood of science by silencing scientists who did what scientists are supposed to do and questioned sketchy scientific conclusions leading to major public policy decisions.
When Freedom of Information Act disclosures finally revealed the extent to which Francis Collins, the director of the National Institute of Health in 2020; Anthony Fauci, the director of the National Institute of Allergy and Infectious Disease in 2020, went to discredit scientists with differing views, Dr. Vinay Prasad, an outspoken associate professor of epidemiology and biostatistics at the University of California San Francisco observed that “Americans would have benefited from a broad debate among scientists about the available policy options for controlling the Covid-19 pandemic, and perhaps a bit of compromise. The emails tell us why that isn’t what we got.”
Prasad understood that a lot of science is more grey than black and white.
Yes, there are blacks and whites. Newtonian physics hasn’t changed in 350 years. The apple still drops down from the tree, and not the other way around, because of the force of gravity. Objects at rest still remain at rest unless an outside force is exerted upon them.
Much science, however, is not black or white. It is gray because it is constantly evolving. We long ago learned smoking greatly increases the risks of people getting lung cancer, but we still don’t know for sure why most smokers don’t get lung cancer.
Accumulating evidence is pointing toward genetic mechanisms that appear to protect some people, but as has long been known from the study of the pandemic of obesity, genetic connections get incredibly complicated.
“Since 2006, genome-wide association studies have found more than 50 genes associated with obesity, most with very small effects,” according to the U.S. Centers for Disease Control. “Most obesity seems to be multifactorial, that is, the result of complex interactions among many genes and environmental factors.”
Further complicating this situation is the fact that not all people genetically inclined toward obesity are fat, and not all fat people are genetically inclined toward obesity. As researchers studying obesity at Harvard’s T.H. Chan School of Public Health have observed, “genes are not destiny,” though they might be more so for smokers than for over-eaters.
Whatever the case, this is the messy scrum in which the reality of science exists. It is not governed by an all-knowing, all-seeing God but by ever-evolving research.
So is batting .270 good or bad?
Well, as with so many things in life, it depends on the standard used to measure.
The epidemiological average is better than the .244 overall batting average for Major League Baseball players who earned an average annual salary of over $4.4 million last year, but well below the league-leading .328 average posted by Los Angeles Dodgers shortstop Trea Turner. And nowhere near the .406 posted by the late Ted Williams in 1941.
The player nicknamed the “Splendid Splinter” was the last baseball player to hit above .400, although more than a half dozen did that in the early years of the professional sport.
Batting .270 would be a great average for a good-fielding catcher in the MLB today given that catchers overall are only hitting at .209 so far this year.
On the other hand, if you were a quarterback hoping to make the roster of an NFL football team, you wouldn’t have a chance with a passing completion percentage of 27 percent. Even Heisman Trophy winner Tim Tebow, the high-profile quarterback for one season for the Denver Broncos, completed 47,9 percent of his passes before the entire league decided he wasn’t meant to be a quarterback.
Zach Wilson of the New York Jets was at the bottom of the passer rankings last year with a completion percentage of 55.6, a percentage of success better than twice as good as that of the epidemiologists.
Every other quarterback who played regularly was at least 10 percentage points better than Tebow, and 29 out of the 33 completed passes at the rate of more than 60 percent. Joe Burrow of the Cincinnatti Bengals topped the chart with a completion percentage of 69.9 percent.
All of this is offered mainly to underline just how relative to the whole the issue of percentages as a measure of success.
A messy world
Getting things right one out of every four times might appear a pretty bad performance, and I have to confess that if I’d been a member of the team involved in this study, it would have been hard not to argue in favor of just tossing the results in the circular file and moving onto to study something else.
The optics just don’t look good. But then maybe these epidemiologists were all golfers.
A 27 percent win rate for a golfer would be unprecedented. None has ever reached that standard. Tiger Woods got close at 26 percent in 2014 before suffering injuries and other problems. He kept playing, however, and now his percentage is down to 22.9 percent, just ahead of the late Ben Hogan who retired with a winning percentage of 21.3.
Woods and Hogan are golf legends, and they underperformed the epidemiologists still stung by Taubes’ reporting all these years later.
His “article prompted an immediate defense by the epidemiologic community,” they write “but together, these defenses have been cited eightfold less often than the original Taubes paper. Therefore, despite these contemporary objections, the Taubes article has exerted a sustained influence on epidemiologists’ self-impression and the impression of epidemiology in the wider scientific community. It has been cited more than 1000 times with most citations relying on the article to cast doubt on the value of epidemiologic research.”
Taubes could not be reached for immediate comment, and it should be noted in his defense there is a difference between casting doubt on the “value” of some difficult research versus the “predictive accuracy” of such research.
Epidemiology – the “study of the factors determining and influencing the frequency and distribution of diseases, injury and other health-related events and their causes” – is not a simple science. Its value, as with many complicated sciences, is often as much in quantifying what doesn’t work as what does.
And there are simple things it can get right because they’re simple, like social distancing during the pandemic.
Covid-19 is caused by the airborne SARS-CoV-2 virus. To catch the disease, you either have to be close enough to someone to inhale a goodly volume of their exhalations, or you need to spend some amount of time in an enclosed space where a lot of people have been shedding SARS-CoV-2 viruses with every breathe.
This has pretty much been known since the arrival of Covid-19, although there was a brief period there when the thinking was it could be transmitted by touch. Some will remember the great hand-washing frenzy.
That’s now been declared largely unnecessary. There are plenty of good reasons to regularly wash your hands, and maintain cleanliness in general, but Covid-19 is no longer the big concern.
“Quantitative microbial risk assessment studies have been conducted to understand and characterize the relative risk of SARS-CoV-2 fomite transmission and evaluate the need for and effectiveness of prevention measures to reduce risk. Findings of these studies suggest that the risk of SARS-CoV-2 infection via the fomite transmission route is low, and generally less than 1 in 10,000, which means that each contact with a contaminated surface has less than a 1 in 10,000 chance of causing an infection,” according to the CDC.
Handwashing was one of those things epidemiologists thought vital at the start.
“Don’t stop washing your hands!” the Mayo Clinic News Network headlined in June 2020 as the pandemic was just beginning.
“As more communities open up and follow changing pandemic protocols, it’s good to remember one of the most important protections against COVID-19 is washing your hands.”
Or so it seemed. But it turned out to be one of those conclusions Taubes might have tagged “as having questionable merit.”
This is unfortunately how science usually works. There are often more unknowns than knowns, more questions than answers, more hypotheses theorized than proved. Professional opinions abound.
Given the value of accumulated knowledge, those opinions are invariably better than amateur opinions, but they are still opinions. And anyone who declares you should take their opinion as established fact because they are an all-knowing scientist – and there have been a few who have done this during the pandemic – should be disqualified as a scientist for a simple lack of understanding of what science is all about.
Taubes had a valid point to make in 1995, and it remains valid still.