Junk science redux

Author: John Luik
Article Published: 28/12/2003

It used to be that the only junk that Canadians had to deal with regularly was the kind that dropped into their mail boxes. Now however, there is not a week that goes by without a new piece of junk science appearing in the press.

For instance over the last couple of months the Post and the other national newspaper have averaged two-three junk science stories a week, including stories about herbicides and childhood cancers, alcohol and breast cancer, new nutritional guidelines from the WHO and global warming. Despite last year’s musing by FP Editor Terry Corcoran that the FP’s crusade against junk science might be running out of targets, this year has produced an enormous number of new cases. Clearly the epidemic has not run its course.

But why is this? The immediate causes are easy to identify and include a press that is both addicted to the “risk of the week” story, and too scientifically or deadline challenged to probe beyond the junk science news release, a public that reacts on cue to any story about risk and everyday life and is scientifically illiterate, a personal injury bar who see junk science as a sure road to personal wealth, researchers who  need junk science dollars to further their careers and advocates who use junk science as a way to manipulate the public policy debate.

The deeper cause however is to be found in something far more powerful and subtle- namely, risk-factor epidemiology. In fact, risk-factor epidemiology is behind virtually every instance of junk science today.

Now  epidemiology can be one of the great tools of modern medicine. But, in the hands of the junk scientist, it can also be one of the curses of democratic public policy. Its potential as a curse stems from the way in which it is so easily manipulated to serve the policy purposes of those who find good science an impediment. There at least four ways in which this manipulation occurs.

The first way in which epidemiology is abused is through the failure to be clear about what it is. Risk-factor epidemiology is not science, nor is it medicine it is rather a branch of applied statistics that provides an idea of the probabilities of certain diseases being linked with certain risk factors, such as high diets and heart disease. The key word here is probabilities for that is all that epidemiology offers. It does not offer certainty about anything for the standard that defines modern epidemiology is the 95% standard: that is it is right 19 times out of 20. Many of us would be somewhat skeptical about driving a car whose brakes worked only 19 times out of 20.

This means that the junk scientist succeeds in part because he never comes clean about what epidemiology is and what it limits are. The public who reads about a “scientific study” assumes that they are reading about the work of white-coated researchers in labs, not number-crunching statisticians. They are never told that the newest scary story about a previously unsuspected risk, is only a statistical  probability.

For example, so much of what epidemiology purports to study is not susceptible to its methods because the alleged risk is too small. In other words the risk can never be shown to exist. A study that wanted to show a risk of getting cancer of 1 in 1,000 would require about 5 million subjects, yet the usual epidemiological study has at most a few thousand subjects.

Second, epidemiology supports itself to junk science because it has adopted the everyday language of causation. Instead of talking about statistical associations, risk-factor epidemiology in the hands of the junk scientist always speaks of some X causing some disease, for instance, being fat causing heart disease. While this makes for great headlines it fails to tell the truth for epidemiology does not establish causes only associations. For instance, there is a strong statistical association between not wearing shoes and having malaria, but the failure to wear shoes is not the cause of malaria. There is a strong association between the decline of the European stork population and a lower European birth rate but it does not follow from this that storks bring babies.

Good researchers always talk about the limitations of their study, about how very difficult it is for epidemiology to ever speak about cause and effect. They caution that the while epidemiology may point to a cause, it can also just as easily establish an accidental correlation. They also note that any talk of causes must be supported with evidence of biological plausibility- that is by real laboratory science.

Third, epidemiology serves the interests of junk science because it supports the view that death can be cheated by interventions to alter the risk factor. The exponents of junk science use risk factors to claim that if we will just do this- eat differently or less, exercise more, stop using microwaves, we can reduce death and morbidity. The strongest evidence about this epidemiological fallacy comes from the numerous controlled trials about reducing heart disease. In these trials half of the group  undergo a series of interventions- changing their diet away from high fat, stopping smoking and increasing exercise, while the other half are allowed to live as before.

There have been at least five of these trials, some in the US (MRFIT) and some in Europe run under the auspices of WHO. All of these epidemiologically inspired trials have been spectacular failures. In the intervention group 41 people died out of every 1000, while in the control group ( those that had no intervention) 40 people died out of every 1000. Yet the belief that interventions to reduce fat can reduce the chance of heart disease is taken by most people to be as true as the secondhand smoke causing lung cancer.

Finally, epidemiology serves the interests of junk science through its practice of reporting its risks in percentages absent of any context. If the evening news tells you that being obese increases your risk of premature death by 25 % this sounds alarming. If, on the other hand, risks were always reported by their Relative Risk (RR) one would have a better understanding of whether the risk was real. For instance if the obesity figure were reported as 1.25 and if the public were told that risks under 4 are not considered serious, the junk scientist would be out of business, for instead of a frightening percentage figure which is essentially meaningless I now have a context in which to put the risk and judge its significance. Take for example the alleged risk of lung cancer from secondhand smoke. Typically these risks are reported as 20-25% increased risk. Consequently much of the public believes that a single whiff of tobacco smoke is a death sentence. Reported as a risk of 1.20 the true insignificance of the risk would be apparent.The interests of all of those to whom junk science is so useful- politicians, researchers, advocates and personal injury lawyers- suggest that it is not going to disappear. Despite this, its effectiveness can be drastically reduced if its primary tool- risk factor epidemiology- can be both better understood and reformed. Efforts are underway in both the US and Europe to create codes of epidemiological conduct that would censure and expose the worst instances of statistical misconduct. The best remedy, however, remains a press and public that are informed and healthy skeptics about what epidemiology is, how it works and where it limits are.

FORCES is supported solely by the efforts of the readers. Please become a member or donate what you can.

Contact Info
Forces Contacts
Media Contacts
Ian DunbarIan Dunbar, United Kingdom

Latest Article »»  

Bill Brown, USA

Latest Article »»  

Michael J. McFadden, USA

Latest Article »»  

Joe Jackson, United Kingdom

Latest Article »»  

Virginia Day, USA

Latest Article »»  

Robert Prasker, USA

Latest Article »»
Contact Robert Prasker »»

John Dunn, MD, United States

Latest Article »»
Contact John Dunn, MD »»

Andrew Phillips, Canada

Latest Article »»
Contact Andrew Phillips »»

Pat Nurse, United Kingdom

Latest Article »»
Contact Pat Nurse »»

Elio F. Gagliano, MD, Italy

Latest Article »»  

Edmund Contoski, USA

Latest Article »»
Contact Edmund Contoski »»

John Luik, Canada

Latest Article »»  

Norman Kjono, USA

Latest Article »»
Contact Norman Kjono »»

Gian Turci, Italy

Latest Article »»  

Søren Højbjerg, Denmark

Latest Article »»
Contact Søren Højbjerg »»