Studying the Studies

From The New York Times:

Last fall, The New England Journal of Medicine published a study concluding that spiral CT screening (a kind of three-dimensional chest X-ray) would make most lung cancers curable. It sounded like wonderful news. For proponents of screening, it was a call to action: the Lung Cancer Alliance is starting an advertising campaign featuring sports celebrities trying to persuade you to make the “right call” and get screened.

But just last week The Journal of the American Medical Association published a study concluding that spiral CT screening is not only ineffective, but may actually be harmful, prompting unnecessary surgery that carries risks of its own.

How could these two studies — in the country’s two most prestigious medical journals — arrive at diametrically opposite conclusions?

Three doctors — Gilbert Welch, Steven Woloshin and Lisa M. Schwartz, all senior research associates at the VA Outcomes Group in White River Junction, Vt. — explain how such an outcome is indeed possible.

But first they explain the important difference between increased survial and mortality. And they note that survival statistics are “subject to powerful distortions”: namely, lead time and overdiagnosis. It’s somewhat of a tricky piece, but well worth reading to gain a clearer understanding of how research results are gathered and presented.

The goal of lung cancer screening is to reduce mortality — to save lives. Because the New England Journal study examines only survival, it cannot tell us whether any lives are saved. Because the JAMA study examines mortality, it is the more valid study. It also corroborates the Mayo trial finding that a significantly increased survival rate can coexist with no difference in mortality.

The JAMA study also highlights the tradeoffs involved in lung cancer screening. The findings show that compared with no screening, if 1,000 people are screened over five years there would be 48 more lung cancer diagnoses, 46 more lung cancer operations (which would be expected to cause 2 deaths) and no lung cancer deaths prevented. The study data are consistent with as many as eight deaths prevented by screening, or eight extra deaths caused by it.

But neither study is definitive, because neither was a randomized trial. And both required assumptions. Given the potential benefit (so many people die from lung cancer) and the potential harms (some die from treatments), no one should have to assume anything.