|
December 2003 -
It used to be
that the only junk that Canadians had to deal with regularly was the kind
that dropped into their mail boxes. Now however, there is not a week that
goes by without a new piece of junk science appearing in the press.
For instance
over the last couple of months the Post and the other national newspaper
have averaged two-three junk science stories a week, including stories about
herbicides and childhood cancers, alcohol and breast cancer, new nutritional
guidelines from the WHO and global warming. Despite last year’s musing by FP
Editor Terry Corcoran that the FP’s crusade against junk science might be
running out of targets, this year has produced an enormous number of new
cases. Clearly the epidemic has not run its course.
But why is
this? The immediate causes are easy to identify and include a press that is
both addicted to the “risk of the week” story, and too scientifically or
deadline challenged to probe beyond the junk science news release, a public
that reacts on cue to any story about risk and everyday life and is
scientifically illiterate, a personal injury bar who see junk science as a
sure road to personal wealth, researchers who need junk science dollars to
further their careers and advocates who use junk science as a way to
manipulate the public policy debate.
The deeper
cause however is to be found in something far more powerful and subtle-
namely, risk-factor epidemiology. In fact, risk-factor epidemiology is
behind virtually every instance of junk science today.
Now
epidemiology can be one of the great tools of modern medicine. But, in the
hands of the junk scientist, it can also be one of the curses of democratic
public policy. Its potential as a curse stems from the way in which it is so
easily manipulated to serve the policy purposes of those who find good
science an impediment. There at least four ways in which this manipulation
occurs.
The first way
in which epidemiology is abused is through the failure to be clear about
what it is. Risk-factor epidemiology is not science, nor is it medicine it
is rather a branch of applied statistics that provides an idea of the
probabilities of certain diseases being linked with certain risk factors,
such as high diets and heart disease. The key word here is probabilities for
that is all that epidemiology offers. It does not offer certainty about
anything for the standard that defines modern epidemiology is the 95%
standard: that is it is right 19 times out of 20. Many of us would be
somewhat skeptical about driving a car whose brakes worked only 19 times out
of 20.
This means
that the junk scientist succeeds in part because he never comes clean about
what epidemiology is and what it limits are. The public who reads about a
“scientific study” assumes that they are reading about the work of
white-coated researchers in labs, not number-crunching statisticians. They
are never told that the newest scary story about a previously unsuspected
risk, is only a statistical probabliliy.
For example,
so much of what epidemiology purports to study is not susceptible to its
methods because the alleged risk is too
small. In
other words the risk can never be shown to exist. A study that wanted to
show a risk of getting cancer of 1 in 1,000 would require about 5 million
subjects, yet the usual epidemiological study has at most a few thousand
subjects.
Second,
epidemiology supports itself to junk science because it has adopted the
everyday language of causation. Instead of talking about statistical
associations, risk-factor epidemiology in the hands of the junk scientist
always speaks of some X causing some disease, for instance, being fat
causing heart disease. While this makes for great headlines it fails to tell
the truth for epidemiology does not establish causes only associations. For
instance, there is a strong statistical association between not wearing
shoes and having malaria, but the failure to wear shoes is not the cause of
malaria. There is a strong association between the decline of the European
stork population and a lower European birth rate but it does not follow from
this that storks bring babies.
Good
researchers always talk about the limitations of their study, about how very
difficult it is for epidemiology to ever speak about cause and effect. They
caution that the while epidemiology may point to a cause, it can also just
as easily establish an accidental correlation. They also note that any talk
of causes must be supported with evidence of biological plausibility- that
is by real laboratory science.
Third,
epidemiology serves the interests of junk science because it supports the
view that death can be cheated by interventions to alter the risk factor.
The exponents of junk science use risk factors to claim that if we will just
do this- eat differently or less, exercise more, stop using microwaves, we
can reduce death and morbidity. The strongest evidence about this
epidemiological fallacy comes from the numerous controlled trials about
reducing heart disease. In these trials half of the group undergo a series
of interventions- changing their diet away from high fat, stopping smoking
and increasing exercise, while the other half are allowed to live as before.
There have
been at least five of these trials, some in the US (MRFIT) and some in
Europe run under the auspices of WHO. All of these epidemiologically
inspired trials have been spectacular failures. In the intervention group 41
people died out of every 1000, while in the control group ( those that had
no intervention) 40 people died out of every 1000. Yet the belief that
interventions to reduce fat can reduce the chance of heart disease is taken
by most people to be as true as the secondhand smoke causing lung cancer.
Finally,
epidemiology serves the interests of junk science through its practice of
reporting its risks in percentages absent of any context. If the evening
news tells you that being obese increases your risk of premature death by 25
% this sounds alarming. If, on the other hand, risks were always reported by
their Relative Risk (RR) one would have a better understanding of whether
the risk was real. For instance if the obesity figure were reported as 1.25
and if the public were told that risks under 4 are not considered serious,
the junk scientist would be out of business, for instead of a frightening
percentage figure which is essentially meaningless I now have a context in
which to put the risk and judge its significance. Take for example the
alleged risk of lung cancer from secondhand smoke. Typically these risks are
reported as 20-25% increased risk. Consequently much of the public believes
that a single whiff of tobacco smoke is a death sentence. Reported as a risk
of 1.20 the true insignificance of the risk would be apparent.
The interests
of all of those to whom junk science is so useful- politicians, researchers,
advocates and personal injury lawyers- suggest that it is not going to
disappear. Despite this, its effectiveness can be drastically reduced if its
primary tool- risk factor epidemiology- can be both better understood and
reformed. Efforts are underway in both the US and Europe to create codes of
epidemiological conduct that would censure and expose the worst instances of
statistical misconduct. The best remedy, however, remains a press and public
that are informed and healthy skeptics about what epidemiology is, how it
works and where it limits are. |
