John Luik
A Response To

"Towards Healthier Communities In Nova Scotia

A Response to
"Towards Healthier Communities in Nova Scotia:
A Discussion Paper"

By John C. Luik

Return to John Luik's main page

This response is a critical examination of the claims and evidence about the effects of Environmental Tobacco Smoke (hereafter ETS) that are advanced in the Discussion Paper. We propose to structure our response into three areas: an analysis of the claims of the Paper, an examination of the evidence adduced in support of those claims, and an analysis of the public policy recommendations made on the basis of that evidence.

Most tobacco issues are raised and discussed in an atmosphere of emotion and rhetoric rather than careful and rigorous analysis. But the same standards that mark the formulation of public policy in any other controversial area must also apply to tobacco. Public policy with respect to tobacco is not granted an exemption from the process of compelling evidence, clear and cogent argument and workable solutions that are the hallmarks of legitimate democratic public policy. Indeed such public policy, to be deemed legitimate must meet the following minimum conditions:

  1. it is based on objective, substantial and careful evidence;
  2. it is reasonable, coherent and consistent
  3. it is carefully designed to achieve its stated ends and does meet those ends;
  4. it respects and enhances the core values of a democratic society.

We wish to suggest that the Discussion Paper clearly and without exception fails to meet each of these conditions.

I. The Claims

The Discussion paper puts forward a variety of claims:

  1. ETS is "a public health hazard" (p.1)
  2. "The smoke that appears off the end of an idle burning cigarette is even more toxic than inhaled smoke" (p.1)
  3. "When you sit in a smoky office, restaurant or bar, you can't help breathing in some of these toxic gases, particles and chemicals. Some of the tar stays in your lungs..." (p.1)
  4. "Second-hand smoke increases the risk of death from heart disease by 20-30% for nonsmokers married to smokers." (p.2)
  5. Second-hand smoke is the third largest cause of lung cancer, after direct smoking and occupational exposure to other hazardous substances." (p. 2)
  6. Working in a smoke-filled environment has about the same long-term effect on a person's health as smoking ten cigarettes a day." (p.3)
  7. "Restaurant workers have a 50 per cent higher risk of lung cancer than the general population." (p. 3)
  8. "Statistics, scientific evidence, health care costs - these all support smoke-free legislation." (p. 6)
  9. "We have overwhelming evidence that secondhand smoke is a major public health hazard." (p. 6)
  10. "A no-smoking policy for all public spaces is the most cost effective solution and the one that provides for the best protection for both nonsmokers and smokers." (p.7)
  11. "At least 80 nonsmoking Nova Scotians die every year from exposure to secondhand smoke." (P-1)

While the claims are various, what unites them is the central assertion that there is "overwhelming evidence" that ETS is a "major health hazard" that must be addressed. What also unites them is the fact that all of them are false - all of them are contradicted by the relevant scientific evidence. In effect, ETS is not a major health hazard and thus it does not warrant government legislation.

II. The Evidence

It is very difficult to reconcile the document's claim that there is "overwhelming evidence" that ETS is a public health hazard with the almost complete lack of evidence put forward in the discussion paper to substantiate this claim. It is particularly interesting, for instance, that whilst the Paper makes sweeping claims about the alleged effects of ETS an health, it does not produce a single piece of Canadian evidence to support these claims. It would appear that the authors fail to understand the basic difference between claims and evidence, between advancing a particular proposition and providing backing for that proposition. What we are offered as cited evidence are:

  • "The Health Effects of Tobacco Use" from the National Clearinghouse on Tobacco and Health
  • "Environmental Tobacco Smoke: Behind the Smoke Screen", a "fact" sheet from the Heart and Stroke Foundation
  • "Tobacco Smoke in the Workplace" from the CMAJ
  • "Involuntary Smoking in the Restaurant Workplace" from JAMA.

References are also made to the US Surgeon General's 1986 Report and to the 1992 US Environmental Protection Agency Report, though these are unreferenced.

We propose to examine this evidence in the following fashion:

  1. We will not comment on the National Clearinghouse on Tobacco and Health citations as this is not a scientific, peer-reviewed study. In short, it does not constitute evidence in any recognized sense. For similar reasons we will not comment on the Fact Sheet of the Heart and Stroke Foundation.
  2. We will comment on the claim that ETS smoke is more hazardous than inhaled smoke.
  3. Since the only real scientific support for the Paper's claims is to be found in the EPA Report (1992) we will devote substantial attention to this Report.
  4. We will not discuss the CMAJ article since it is now almost 13 years old and superseded by more current data.
  5. We will comment at some length on the JAMA article as it is indicative both of the quality of evidence used in this discussion paper and on the entire process of misrepresenting scientific findings on the ETS issue.
  6. We will also look at the entire range of FTS studies on both lung cancer and heart disease.
  7. We will examine the ETS studies that have focused an the issue of occupational exposure to ETS.
  8. Finally, we will look at the work of the Congressional Research Service and its analysis of the alleged health hazards of ETS.

A. ETS, Mainstream and Sidestream Smoke

The Paper claims that the smoke "that appears off the end of an idle burning cigarette" is "even more toxic than inhaled smoke" (p.1 ). But this claim is completely false for it assumes that ETS is in fact equivalent to sidestream tobacco smoke - the smoke produced by a smoldering cigarette. If ETS is sidestream smoke it is extremely diluted sidestream smoke. Indeed, in real- world settings in which ETS has been measured only a very few of the compounds present in mainstream tobacco smoke have been detected. As G. Gori notes:

"Several thousand components of mainstream smoke can be measured, but because of extreme dilutions fewer than two dozen components of ETS can be detected under field conditions. Even assuming that sidestream smoke remains unchanged as it ages and becomes ETS, the concentrations of its components would be between one thousand and one millions times below the corresponding levels permitted in workplaces by the Occupational Safety and Health Administration..." [1]

Even the EPA, which the Paper quotes elsewhere, notes that the equivalency of sidestream and mainstream tobacco smoke, let alone the enhanced toxicity of sidestream smoke, may not be tenable (EPA, 6.6)

Figure 1

Relative dose estimate of respirable suspended
particulates (RSP) in typical active smokers and
ETS exposed non-smokers

ACTIVE SMOKER 30 cigarettes per day
15 mg RSP inhaled per cigarette
90% lung retention efficiency
DAILY DOSE about 400 mg
ETS EXPOSED
NON-SMOKER
0.05 mg RSP/cubic meter of air
1.5 hours per day exposure (*)
0.7 cubic meters per hour inhaled
10% lung retention efficiency
DAILY DOSE about 0.00525 mg

CRUDE DOSE RATIO 0.00525 : 400 = about 1 : 75,000

(*) USOSHA, 1994; Emmonst et al., 1992

B. The EPA Report

The second evidentiary problem, which again suggests the Paper's proximity to corrupted science is its uncritical use of the US Environmental Protection Agency 1992 Report. The Paper refers to the EPA work as the "most extensive recent review of the evidence of passive smoking and lung cancer" (p. 98). Quite surprisingly, the Paper fails to mention the extensive criticism directed at the EPA's ETS analysis, or the fact that. the EPA's findings are the subject of court action. Given the uncritical attention devoted to the EPA's work it is worth noting precisely how strong the evidence is which suggests that the EPA's science on ETS is corrupt science. The evidence that the EPA science on ETS is corrupt science falls into two categories: evidence about the substance of the science and evidence about the processes involved in creating and using the science.

The Substantive Issue

The EPA report Respiratory Health Effects of Passive Smoking: Lung Cancer and Other Disorders [2], claims that "Based on the weight of the available scientific evidence, the U. S. Environmental Protection Agency has concluded that the widespread exposure to environmental tobacco smoke in the United States presents a serious and substantial public health impact." Is this the case" In order to answer this question one must first know something about the data on which the EPA decision is based. The EPA report refers to the 30 epidemiological studies on spousal smoking and lung cancer that have been published from 1982-1990. It is important to note that while EPA Administrator Reilly in referring to the Report spoke about ETS and cancer in children and in the workplace and though the Report has been used as a basis for demanding smoking bans both in public places and in workplaces, the EPA did not examine those studies that look at workplace ETS exposure. The overwhelming majority of which do not find a statistically significant association between exposure to ETS and lung cancer in non-smokers: a fact that by itself destroys the legitimacy of any harm based demand for public or workplace smoking bans.

Thus, to begin with, the EPA case is based not on workplace or public place ETS exposure, but on the risks of non-smoking spouses contracting lung cancer from their smoking spouse. But what of the 30 studies" The 30 studies come from different countries and vary substantially in size. Some have fewer than 20 subjects, others are based on larger populations, with the largest study involving 189 cancer cases. Of the 30 studies, 24 reported no statistically significant association, while six reported a statistically significant association, that is a positive relative risk for those nonsmoking spouses. Now relative risks are further classified into strong risks or weak risks depending on the magnitude of the risk ratio. Within the 30 studies on ETS and lung cancer NONE reported a strong relative risk. Moreover, whenever the assessment of relative risk is weak there is a substantial possibility that the finding, the assessment, is artificial rather than real. That is to say, there is a strong likelihood that even the weak relative risk is a reflection not of some real world risk, but of problems with confounding variables or interpretative bias. There are, for instance, at least twenty confounding factors ranging from nutrition to socioeconomic status that have been identified as important to the development of lung cancer. Yet none of the 30 studies attempts to control for all of these factors. So in assessing the global scientific evidence about ETS and lung cancer the crucial conclusion is that none of the studies report a strong relative risk for non-smokers married to smokers.

Figure 2

Estimated number of cigarettes required to reach TLV
levels from sidestream smoke emission of selected chemicals
in a sealed and unventilated 100 m3 enclosure
(Gori and Mantel, 1991)
(Please note: 100 m3 are equivalent to a room
22' x 21' x 8' ceiling, or 3,696 cubic feet)

 
SSS ComponentSSS output
mg/cigarette
TLV
mg/m3
Cigarettes
required
Methylchloride0.880.301,170
Acetaldehyde1.26180.001,430
Nitrogen oxides2.8050.001,780
Phenol0.2519.007,600
Benzene0.2432.0013,300
Dimetylamine0.03618.0050,000
Benzo(a)pyrene0.000090.20222,000
Polonium0.4pCi3pCi/l750,000
Toluene0.000035375.001,000,000

Now the EPA report discusses all of these 30 studies, but it limits its statistical analysis to only 11 U.S. studies of spouses of smokers, and what do the 11 studies show" Of the eleven, 10 reported no statistically significant association between ETS exposure and lung cancer while one reported a statistically significant association. The EPA analysis of these 11 studies claims that they show a statistically significant difference in the number of lung cancers occurring in the non-smoking spouses of smokers such that they suffer 119 such cancers compared with 100 such cancers in nonsmoking spouses of nonsmokers. It is this finding of statistical significance, a finding based on only the 11 U. S. studies,10 of which found no significant association, that is the basis for the EPA decision to classify ETS as a "Group A" carcinogen. [3]

In order to arrive at its "conclusion" the EPA combined the data from the 11 studies into a more comprehensive data assessment called a meta-analysis. Meta-analysis is governed by its own rules, as not every study is a candidate for such combined analysis. In general, meta-analysis is appropriate only when the studies being analyzed together have the same structure. The difficulty with the EPA's use of meta-analysis of the 11 ETS studies is that it has failed to provide the requisite information about the structure of the 11 studies, information crucial for an independent assessment of whether the studies are indeed candidates for meta-analysis. Thus, the EPA conclusion is based on a meta-analysis that is difficult, if not impossible, to verify.

But even more crucial to the question of assessing the quality of the EPA's ETS science is the issue of confidence intervals, for even by limiting its analysis to only 11 studies, and even by lumping these studies together through a meta-analysis, the EPA could not have achieved the "right" result if it had not engaged in a creative use of what epidemiologists call confidence intervals. Essentially, confidence intervals express the likelihood that a reported association could have occurred by chance. The generally accepted confidence interval is 95%, which means that there is a 95% confidence that the association did not occur by chance. In as much as most epidemiologists use the 95% confidence interval, the EPA itself, until the ETS report, always used this interval. Indeed, all of the individual ETS studies reviewed by the EPA used a 95% confidence interval. Curiously, the EPA decided that in this instance it would use a 90% confidence interval, something that effectively doubles the chance of being wrong. Even more curious is the fact that when asked to justify this departure from accepted scientific procedure, EPA Administrator Reilly simply replied that the 90% confidence interval "was recommended to us by the scientific community as appropriate to this data." What Mr. Reilly really means by appropriate to this data is that without using this 90% standard, the EPA could not have found that the 11 U.S. studies were "statistically significant." Without employing a novel standard, without in effect changing the accepted rules of epidemiological reporting, the EPA result, already painfully coaxed into existence, would not have existed, ETS could not have been labelled a "Class A" carcinogen.

Thus, despite all of its careful selection of the right data, its meta-analysis and finally its relaxed confidence intervals, the conclusive point remains, as Huber, Brockie, and Mahajan note in Consumers Research in the United States (1991), that "No matter how the data from all of theepidemiological studies are manipulated, recalculated, 'cooked', or 'massaged', the risk from exposure to spousal smoking and lung cancer remains weak .... No matter how these data are analyzed, no one has reported a strong risk relationship for exposure to spousal smoking and lung cancer." [4]

Figure 3

Concentrations of respirable suspended particles from
ETS and other sources in various environmental setting.
(Gori and Mantel, 1991)

 
ReferenceSiteRSP
Concentration
(�g/m3)
RSP
Concentration
(�g/m3)
  No smokingSmoking
Coultas et al. 1990HomesNA17
Sheldon et al. 1989Homes22 (**)65 (**)
Spengler et al. 1981HomesNA20
Spengler et al. 1985Offices39 (**)72 (**)
Proctor et al. 1989bOffices8 (*)23 (*)
Oldaker et al. 1990OfficesNA27(*)
Miesner et al. 1988Offices15 (**)36 (**)
Sterling et al. 1983Offices15 (**)29 (**)
Coultas et al. 1990bWorkplacesNA64 (**)
Oldaker et al. 1990RestaurantsNA36 (*)
Crouse 1988RestaurantsNA34 (*)
Proctor 1990Public Transit14 (*)36 (*)

(*) Based on UV-RSP portion of total RSP
(**)Based on total RSP & NA - Data not available or not applicable

The Process Issue

While a careful look at the substance of the EPA's ETS claims clearly shows why this science can be called nothing other than corrupt science in that it uses highly selected data, data that is then further manipulated in breach of accepted scientific norms, all without cogent explanation, to reach the "right" conclusion, an examination of the process underlying this science demonstrates even more clearly its wholly corrupted character. There are at least nine specific process issues worth noting, each of which highlights a slightly different dimension of the corrupted character of the EPA's ETS science.

First, EPA science issues from a health promotion perspective that finds its conceptual home in the Lalonde Doctrine propounded by former Canadian Minister of National Health and Welfare, Marc Lalonde. Lalonde argued that health messages must be vigorously promoted even if the scientific evidence was incomplete, ambiguous, and divided. Health messages must be "loud, clear and unequivocal" even if the evidence did not support such clarity and definition. What we have in the EPA is simply the Lalonde Doctrine as an institutionalized process. Clearly the substance of the ETS data does not support its "Group A" status, nor does it support public and workplace smoking bans on the grounds that ETS threatens the health of non-smokers. But the substance of the ETS data is to be ignored because the Lalonde Doctrine places the process of using such substance ahead of the substance itself; indeed, it requires that the substance be portrayed as something that it is not in order to further the health agenda. What this inevitably does is to build into the heart of the scientific enterprise a institutionalized motivation and justification for allowing ends extrinsic to science to determine the findings of science, for allowing science to be subject to an agenda not its own, for allowing science to lie with a clear conscience. Once one has come to see science as something that of necessity happens within the context of health promotion then the process corruptions of the EPA follow quite "naturally".

This explains why at one level those involved with the EPA decision on ETB are quite frank about their process. For instance, an EPA official responsible for the revised ETS risk assessment was quoted in Science (July 31,1992) as admitting that "she and her colleagues engaged in some fancy statistical footwork" to come up with an "indictment" of ETS. [5] (The footwork to which she refers is the novel 90% confidence interval.) Or to take another process example, the Science Advisory Board which reviewed the initial draft risk assessment on ETS, and found the case against ETS based on its association with lung cancer unconvincing, actually urged the EPA staff to attempt to "make the case" against ETS on the basis of the similarities between ETS and mainstream smoke. [6] To be fair, the consequences of the Lalonde Doctrine are not confined to the EPA's anti-smoking agenda. For instance, a 1989 JAMA article reported a study that claimed to show a link between ETS exposure and increased risk of cervical cancer. In response to critics who noted that such a link was biologically implausible and that the study had ignored confounding factors, the authors replied that the study was justified simply on the grounds that it might reinforce the dangers of smoking message. "While we do not know of a biologic mechanism for either active .... smoking or ETS to be related to cervical cancer, we do know that cigarette smoking is harmful to health. The message to the public, as a result of this study, is one that reinforces the message that smoking is detrimental to health." [7] It would be difficult to find a more succinct example of the Lalonde Doctrine at work. There is no compelling evidence to support our claim, the authors all but admit, but it is important, in the interests of health promotion that the public be made to think that there is scientific evidence of harm.

Figure 4

Environmental tobacco smoke is not equivalent to
mainstream or sidestream smokes

 
CharacteristicsMSSSSSETS
Measure components>4000  
Temperature900 C600 CAmbient
Oxygen content15%2%20%
Pyrolysis productsHighLowLow
pH~6.5~7.0Ambient
Photochemical
transformations
LowLowHigh
Dilution20102105-6
HumidityHighLowAmbient
Age3-5 sec3-5 secHours
Particle diameter~1� ~0.1 �
Particle volume~0.5 �3 ~0.005 �3
Volatiles and water
in particles
HighMediumVery Low
Particle lung
retention efficiency
90% 10%

But second, while those involved in the EPA process are at one level open about the process, at another level they are profoundly dissembling. For instance, the EPA fails to mention that the "Group A" status for ETS was arrived at using a process that violates its own Guidelines For Carcinogenic Risk Assessment. Rather than acknowledging that this suggested that both the substance of its findings and the process were corrupt, the Science Advisory Board reviewing the ETS issue argued that this suggested a need not for concluding that ETS posed no threat to the health of non-smokers, but rather a need for changing the Guidelines For Carcinogenic Risk Assessment. Given that the right conclusion must be reached and the data do not support that conclusion, one must manipulate the data and revise the guidelines governing the process and the conclusion.

Third, the ETS risk assessment process has been corrupted from the outset by the fact that it has repeatedly violated the standards of objectivity required by legitimate science by utilizing individuals with anti-smoking biases. One member of the group working on the ETS issue at the EPA is an active member of U.S. anti-smoking organizations, while the Science Advisory Board that examined the EPA's ETS work included not only a leading anti-smoking activist, but several others strongly opposed to tobacco use. Finally, the EPA contracted some of the work on certain documents related to the ETS risk assessment to one of the founds of a leading anti- smoking group.

Fourth, the EPA changed the accepted scientific standard with respect to confidence intervals, without offering any compelling justification, in order to make its substantive findings statistically significant.

Fifth, the EPA's Workplace Policy Guide which as a policy document would, in the course of normal scientific process, be developed only after the scientific evidence was in, was actually written before the scientific risk assessment was even completed, let alone reviewed and finalized. [8] Quite obviously, science was to be made to fit with policy, rather than policy with science.

Sixth, the EPA fails to note that if the two most recent U.S. ETS studies were to be included along with its eleven other studies it would have resulted in a risk assessment that was not statistically significant, even using the 90% confidence interval. With its entire "conclusion" at risk, there are exceedingly compelling process reasons for the EPA to have excluded these two studies from their analysis.

Seventh, exclusion, however, was apparently insufficient, for the EPA does more than simply not use the studies, it actually refers to them in an appendix and misrepresents the one through claiming that it supports the EPA's ETS conclusions. The study, by Brownson, et al , which appeared in November,1992 in the American Journal of Public Health reported no statistically significant increase in risk between lung cancer and ETS exposure. In order to get round this politically unacceptable conclusion the EPA quotes Brownson as concluding that "Ours and other recent studies suggest a small but consistent increased risk of lung cancer from passive smoking." (ADD - l) But this is not the issue, as the EPA well knows. The question is not whether there is a small increased risk, but whether there is a statistically significant risk, which Brownson concludes that there is not. In effect, the EPA misrepresents a scientific finding through changing the terms of reference from statistical significance to just plain risk.

This penchant for misrepresentation is not, however, just confined to recent studies. For instance, the EPA analysis consistently makes reference to the Garfinkel, et al study. At 5-48 the EPA claims that the Garfinkel study presents "at least suggestive evidence of an association between ET and lung cancer..." [11] But a careful reading of Garfinkel does not confirm this at all.

Garfinkel actually says that "we found an elevated risk of lung cancer, ranging from 13-31 per cent, in women exposed to smoke of others, although the increase was not statistically significant." [12] The entire question of suggestive evidence is bogus: the relevant question is whether Garfinkel found a risk that was statistically significant. He did not, and the EPA misrepresents his finding.

Eighth, the EPA represents its process as a comprehensive and objective analysis of the ETS data. In the usual course of things this would imply a careful examination of the criticisms that have been levelled at the studies used to reach its conclusions. However, a careful examination of the Bibliography accompanying the Report suggests that this is not the case. Although the note with the Bibliography indicates that it is not a "comprehensive list of all references available on the topic" it is still a list of all references cited and reviewed for the Report. Yet, to take but one example, one would never know from the Report or its Bibliography that the work of Trichopolous had been subjected to significant criticism by both Burch and Heller, since neither is mentioned in the Bibliography. Nor indeed, despite its lengthy examination of the Hirayama studies, would one learn from either the discussion or the Bibliography that Hirayama was devastatingly criticized by Rutsch who noted that one could infer from Hirayama's data that lung cancer was more common in unmarried non-smoking women than in the non-smoking wives of smokers, a somewhat curious result.

Now the possible explanations for such selectivity are that:

  1. the authors of the study are not familiar with such criticisms, which would suggest incompetence, or
  2. they are familiar with the criticisms but have misunderstood them, ignored them or discounted them.

But even if one were to discount or ignore them it is still odd, if one is committed to objectivity and openness, to not cite them. To not cite them suggests that one wishes to act as if they didn't exist and to do this is to give rise to more than the suspicion that the EPA's ETS work is really an instance of a closed-loop process abuse. In a closed loop the circle is never opened up to divergent, dissenting views, views that challenge the orthodox conclusion. It is not simply that such divergent views are discounted, it is rather that, as the EPA discussion and Bibliography indicate, they simply never are heard, indeed judging from the Bibliography they don't exist. When one considers this closed loop process in the context not merely of what the EPA excluded in terms of dissenting voices, but in the context of what is sought in to include in terms of determining voices the anti-smoking movement - then it is hard to assign any degree of objectivity to the process.

Ninth, despite the significant difficulties that have been raised about the quality of EPA science, particularly by the Expert Panel in its report Safeguarding The Future: Credible Science, Credible Decisions, a report that noted that:

  1. EPA "science is of uneven quality"
  2. the "EPA has not clearly conveyed to those outside or even inside the Agency its desire and commitment to make high-quality science a priority"
  3. "the science advice function -- that is the process of ensuring that policy decisions are informed by clear understanding of relevant science -- is not well defined or coherently organized within EPA"
  4. the "Agency does not have a uniform process to ensure a minimum level of quality assurance and peer review for all the science developed in support of Agency decision making"
  5. the "Agency lacks the critical mass of externally recognized scientists needed to make EPA science generally credible to the wider scientific community"
  6. "science should never be adjusted to fit policy". [13]

Despite this the EPA process is incapable of correcting itself.

This is perhaps the most significant process corruption of all, namely a process that is quite conscious of its problems but is unwilling and unable to address them. Of course even this characterization is perhaps too kind given that what the Expert Panel describes as problems are really, for the anti-smoking movement, just the normal way that science must proceed if it is to make the anti-smoking case. If this is the case then there is no conscious sense of process problems. What the Expert Panel's Report actually provides, of course, is another description of corrupted science corrupted in its substance and its process, science driven by a pre-determined policy agenda, science based on inadequate data, science of uneven quality and inadequately peer-reviewed, science lacking critical validation by outside scientists representative of "wider scientific community", and science, finally, fully aware of its corruption, but unable to heal itself.

C. M. Siegel "Involuntary Smoking in the Restaurant Workplace", JAMA, Vol 270 No 4.

The major evidence offered for the claim about the hazards of ETS for restaurant workers is the article by Siegel. A close examination of this article, however, shows that it provides absolutely no evidence for the claims of the Paper. Siegel, for instance, notes that his epidemiological evidence "suggested that there may be a 50 percent increase in lung cancer risk among food- service workers that is in part attributable to tobacco smoke exposure in the workplace." Notice the key words - suggested, may be, and in part - these are not the words of scientific evidence or conclusions. Despite this, Siegel made even stronger claims for his findings in USA Today (7/28/93) where he said that ETS had a "devastating effect" on restaurant workers. It is not "a life and death issue for restaurant workers;' he claimed. But is this the case" Can we legitimately draw these conclusions and support Siegel's public policy recommendation to ban smoking, in bars and restaurants based on his "scientific evidence"" Siegel's evidence comes from six studies conducted by others - he did none of the work from which he draws his conclusions. The six studies were about the lung cancer risks associated with various occupations. But none of the six studies was about ETS exposure for any of the occupational groups. In other words, the studies were not ETS studies at all, let alone studies about ETS and restaurant workers. Yet Siegel has both drawn conclusions and made recommendations about ETS based on these studies.

Equally important, even if the figures for increased risk of lung cancer were correct (and remember those words "suggested", "may be") and even if the increased risk were due entirely to ETS (which none of the studies showed), the 50% increase is not statistically significant. It means that the RR (relative risk) is 1.5 and RR's below 2.0 are a product of chance or confounders.

D. The Totality of Domestic and Occupational ETS Studies: Lung Cancer and Heart Disease

But let us leave aside both the EPA and Siegel and examine the complete picture of ETS studies. The paper contends that there is "overwhelming evidence" that ETS is a public health hazard. Does the scientific record actually support such a claim" The answer is most emphatically no. Table 1 provides a summary of the 40 worldwide studies of lung cancer in nonsmoking females married to smokers, with the largest and best studies at the top. It is immediately apparent that most of the studies indicate that there is no statistically significant increase in the risk of contracting lung cancer. In short, the Paper's claim is simply false. The same is true for Table 2 which summarizes the studies of US nonsmoking females married to smokers-there is no statistically significant increase in risk.

Table 3 reports the studies of lung cancers in non-smokers who have been exposed to ETS since childhood. Because of their long term exposure this one group should show an increased risk if one is to be found. Yet again, together these studies indicate no statistically significant increase in risk. Table 4 summarizes the studies which examine workplace exposure to ETS. Taken together the studies show no statistically significant increase in risk.

Finally, Tables 5-7 report the data with respect to heart disease and the exposure to ETS in non- smokers. Once again, the combined evidence does not suggest that ETS poses a statistically significant risk for heart disease in nonsmokers.

E. The report of the Congressional Research Service "Environmental Tobacco Smoke and Lung Cancer Risk"

In response to,a request from the US Congress, the Congressional Research Service undertook an analysis of the potential health effects of ETS. In particular, the analysis looked at the work on ETS which has appeared since the publication of the EPA's findings. Four studies in particular were examined -- Kabat [14], Fontham [15], Brownson [16], Stockwell [17]. Of these studies, two show no increased average risk (Kabat and Brownson), one shows a barely statistically significant risk (Fontham), and one shows an increased average risk which is nonetheless not statistically significant at 95% confidence level (Stockwell [18]). Moreover, if we have to take the Fontham study, which alone of these four shows a statistically significant average increased risk, and as the relevant question about the degree of risk suggested, then in the words of the Congressional Research Service the chance of dying of lung cancer over one's lifetime "for a person exposed only to background ETS, the number drops to about 7/100 of one percent [19]. Moreover, using data from the Brownson study, "there are no annual lung cancer deaths from ETS." [20].

What makes this data even more problematic is the problem of misclassification, where smokers are wrongly classified as nonsmokers. As the CRS noted: "It is clear that misclassification and recall bias plague ETS epidemiology studies. It is also clear from the simulations that modest, possible misclassification and recall bias rates can change the measured relative risk results, possibly in dramatic ways." [21]

"...[P]possible combinations of small rates -- below 10 percent -- could drive ETS relative risks in the highest exposure groups to values no longer distinct from 1.0. While these results are obtained from the Fontham study, similar results are likely from the Brownson study. Even smaller values of these rates -- below 3 percent -- could be combined to reduce the lower bounds of the 95 percent confidence intervals well below 1.0 for these studies." [22] (p.40)

III. The Policy Recommendations

The policy recommendations that flow from the Paper's "evidence" are, not surprisingly, incoherent. If ETS does not constitute a health hazard to non-smokers and we have seen that it does not then there are simply no health related reasons for making all public places smoke free. The claim that "the only way to eliminate the risk of exposure to second-hand smoke is to provide contained, separately ventilated smoking rooms." (p. 7, Paper) is untrue because there is no statistically significant risk from ETS.

The real justifications for the policy, however, have nothing to do with the alleged health effects of ETS. As the Paper's authors note on page 7 smokers faced with a complete ban on public smoking "tend to respond by smoking less, or even quitting." In reality, the claim that ETS threatens the lives on non-smokers is only a crude attempt to use a harm-to-others argument to conceal an offensive and unjustified paternalism. It is not the ETS is harmful, rather it is that smoking is harmful to smokers.

IV. The Consequences of Corrupted Science for Society and for Democratic Public Policy

In the first section of this comment we argued that there was substantial evidence that the Paper adduces corrupted science to support its claims. In the second section of this comment we examine the consequences of using corrupted ETS science in order to frame public policy.

At the outset it is important to note that apart from the direct implications for public policy there is a wider social issue about the Paper and its claims, namely the moral questions which it raises.

The first sort of question is obviously the question of the legitimacy of misrepresentation, for corrupted science is at bottom science that misrepresents the state of reality. And what a careful analysis of the scientific claims of the Paper reveals is a profound and systematic disregard for the truth about the dangers from ETS. Not only is data manipulated to produce the desired results and suppressed or dismissed when it does not fit with the standards of political correctness, but the fact that accepted standards about confidence intervals are changed without justification goes unnoted. In effect, one has an ethic that legitimises misrepresentation in the service of a good cause - "a smoke-free society". But is a smoke-free society a sufficient justification for a public health movement founded on unreliable science and blatant misrepresentation" The frightening thing about deceit - whether in the allegedly righteous cause of eliminating smoking or in the service of any number of other worthy ends - is both that it is so easy to justify and so difficult to restrict its use to the ends that originally justified its employment.

But there is a second moral question here that goes beyond the morality of misrepresentation into what might be called the morality of suppressing dissent. Both the process of producing corrupted science and of utilising it as the basis for public policy demand a fundamental intolerance of dissent, both scientific and otherwise. The imperatives of health promotion are such that the ambiguities and uncertainties that form a legitimate part of science and most importantly the questions about the quality of the evidence and whether it justifies the proposed public policy measures cannot be tolerated. This means that scientific and public policy dissent must be suppressed through portraying dissenters as either in the pay of the tobacco industry he-margins of the scientific establishment, a strategy that raises a host of subsidiary moral questions. Whatever the cost, "science" must be seen to provide a conclusive and united answer to the question of tobacco and harms to the innocent. Thus, despite the vital role of questions, argument and dissent in science as well as in democratic life, the process of corrupted science seeks to silence dissent in the interests of protecting not the truth, but is misrepresentation of the truth.

By far the most morally objectionable aspect of the Paper is its readiness to use corrupted science to deprive smokers not only of their right to pursue their pleasure in public, but quite possibly to gain or retain their employment, or advance their prospects. Put more bluntly, it is the question of whether it is morally justifiable to use bad science to hurt people" What should never be lost sight of in this debate is that without the alleged scientific justification of harm to innocent parties, there is no compelling public policy rationale for banning or restricting smoking in public places or workplaces. Once the corrupted science is stripped away, there simply are no harms, and without those harms, smoking becomes a self -regarding behaviour, interventions against which can only be advanced on patently paternalistic grounds. The Paper might still argue that public and workplace smoking should be banned in order to discourage smokers from smoking, but this argument loses its compelling harm-to-others character and become instead nothing more than an argument about the state intervening in the private lives of competent adults.

What is so morally offensive here is that truly morally blameless people - not the alleged victims of smokers - but smokers themselves, are to be harmed in significant ways on the basis of bogus science and for no good reason. What makes the morality of the Paper as corrupt as its science is that it is prepared to exploit for its own ends our readiness to deprive individuals of certain rights if the exercise of those rights appears to harm others by explicitly manufacturing harms to others. In doing so, the Paper simultaneously violates perhaps the two most fundamental moral principles, first by treating persons, in this case smokers and their alleged harms to others, as merely means to the end of a smoke-free society and not as ends in their own right, and second by inflicting substantial pain on an entire class of people without their consent and for no compelling reason.

But the question of the moral justifiability of using corrupted science to hurt people goes beyond the question of depriving individuals of their right to a significant pleasure, or even of a job to something far more crucial, namely the justifiability of depriving individuals of their moral standing through stigmatising them as moral outcasts. In the end, this is, of course, the logical outcome of ETS science, to make smokers a class of moral miscreants who see themselves and are seen by others as so ruthlessly intent on pursuing their own interests that they are blind to the harm they inflict on others. It is indeed but a short way from the claim "Smoking kills" to the conclusion that "Smokers kill". But then, such a conclusion is the public policy justification for bans on public smoking.

Surely there is nothing more morally loathsome than to attempt to manipulate public policy to create a class of citizens who, on the basis of bad science, come to despise themselves for what they mistakenly believe they do to others and who in turn are despised by their fellow citizens for allegedly threatening their well-being. The ultimate moral corruption of the Paper lies in its considered use of corrupted science to erode the normal standards of respect and tolerance for individual difference and diversity through creating a society that portrays smokers as indifferent menaces to the health of others who by their indifference place themselves on the periphery of, if not outside the moral community.

At the end of the day, the Paper's foundation of corrupted science poses a threat to that most fragile aspect of social capital - trust. To the extent that its evidence is shown to be contrived or disconnected from its conclusions, it lessens every thoughtful citizen's trust and respect for both science and government. To the extent that it misrepresents reality through creating a risk it makes smokers distrustful of themselves and their actions while at the same time creating a distrust of smokers by non-smokers.

For all of the moral problems that the use of corrupted science raises, the equally, if not far more disturbing, issue that it introduces are the implications of bad science in the public policy process. What then are the consequences of introducing corrupted science into the democratic public policy process" In a word, nothing less than disastrous.

The operative word here, is of course, democratic. We are not concerned with the uses of bad science in a non-democratic society. In fact, with the appropriate degree of space we might wish to argue that non-democratic societies, or more specifically totalitarian societies might be peculiarly receptive to the use of corrupted science. Our concern here, however, is with the effects of using corrupted science as a mechanism for framing and justifying democratic public policy and our claim is that the effects of using such science are fundamentally at odds with the character of a democratic society.

The easiest way to understand the threat that corrupted science poses to democratic public policy and to democratic life as a whole is to understand what it is that democratic public policy tries to do. The goal of democratic public policy is to minimise public harms insofar as this is possible within the context of such foundational democratic values as diversity, autonomy, respect, rationality and fairness.

The Value of Diversity

This is the recognition that the persons who make up democratic society bring a diversity of beliefs and values to the community, a diversity whose rich complexity is more often than not, not captured by the social science theories and data that are the tools of choice in the public policy process. This diversity, moreover, is not simply reflected in conflicting notions of what direction society should take, but more basically in differing pictures of what makes a good personal life. By accepting diversity as a foundational value, the democratic society and democratic public policy process accepts it not just as a fact but as something to be encouraged, enhanced and celebrated as a strength.

The Value of Autonomy

This is the recognition that subject to the acceptance of certain minimal core values necessary for any society to exist, the individuals who make up democratic society are the best judges of the shape that they wish their lives to take and they should be accorded the maximum liberty, compatible with similar liberty for everyone else, to think, believe, and live as they choose. This means that the state will resist the impulse, however well-intentioned, through the misuse of the public policy process, to undermine and intrude upon its citizens' capacities and inclinations for self-governance, to engineer the lives of its citizens through collectivising the conscience into one communal vision of good life.

The Value of Respect

This is the recognition of the equality of the human and moral standing with the State that the citizens of a democratic society possess. It is the recognition that the democratic state sees its citizens as persons of intrinsic worth, equivalent in dignity and standing with itself, with lives not to be managed or saved, but to be allowed to develop in ways of their own choosing. It is a recognition that the State's role should be to encourage its citizens to define themselves and their life-projects in widely varying ways, to foster the development of self-respect through deferring, to the greatest extent possible, from moral judgments about these self-definitions and life-projects and to create the conditions which allow its citizens' lives the greatest possible chance of fulfilment.

The Value of Rationality

This is the recognition that the public policy process must be grounded on democracy's respect for the rational, that is adherence to the standards of rationality with respect to the assumptions that it employs, the evidence that it considers and the analysis that it brings to bear on that evidence. None of its elements must be based on irrational considerations: all must meet the minimal test of reasonableness through being clear, coherent, and compelling. The evidence supporting public policy measures must be substantial. The measures must be coherent and consistent with what the evidence shows and the measures must have a significant promise of being effective. At the same time the recognition of the value of rationality is also the recognition that the truth is frequently complex, something that places its own limits on the scope of rationality. Reality will often be richer and denser than evidence and theories, some problems will not have easily identifiable cases and solutions, but this does not justify public policy formulated in the absence of reason and on the basis of surmises, hunches or appeals to emotion or intuition.

The Value of Fairness

This is the recognition that democracy entails a foundational commitment to elicit, examine and consider, within an objective, open and non-arbitrary framework, the views of all of those whose legitimate interests are likely to be affected by collective decision making. It is the recognition that a democratic society will not unfairly discriminate through structuring the power process so as to preclude the statement and examination of certain perspectives.

What this suggests is that both the agenda of legitimate public policy in a democracy and the process used to consider that agenda are constrained by certain non-negotiable values. What marks certain policy options and certain policy processes out as illegitimate and non-democratic is their conflict with these core non-negotiable values.

To take an example, a public policy that significantly undermined the autonomy routinely accorded to citizens in a democratic society, or one that tended to eliminate the diversity and tolerance that are marks of democratic life, or a public policy process that failed to consider fairly divergent points of view, or used poor argument or flawed evidence to advance policy options would fail to qualify as legitimate democratic pubic policy inasmuch as it conflicted with one or more of the key values on which not only democratic public policy but democratic society is founded.

Placed within this context it is clear that the Paper's use of corrupted science is a threat, not at some peripheral point, but at the very centre, to democratic values and to democratic public policy. Corrupted science and the use of corrupted science in the creation of policy threatens each of the process and substance values-diversity, autonomy, respect, rationality and fairness - that characterize democratic public policy. And this is something that should be of concern to everyone, whether non-smoker or smoker.

First, the use of bogus ETS science in an attempt to determine the policy agenda on smoking imperils the distinguishing characteristic of science - its objectivity - and threatens to render science worthless for public policy purposes. Though science is never completely objective, if indeed complete objectivity is possible, it at least, in distinction from much of the political process, professes a fundamental interest in reason, evidence and bias-free judgment. In fact, much of science's standing in contemporary society derives from its objective character, as does much of its usefulness in the public policy process. In effect, we have a high degree of confidence in the scientific process as providing a careful, evidenced and to some degree, value- free, assessment of certain questions relating to public policy, And it is precisely this utility that the use of corrupted science threatens. If science ceases to work outside of the political and policy process, if it ceases to be a tool available to all sides of an issue, if it becomes politicised and ideologically sensitive, then it ceases to be valuable in the policy process for it becomes nothing more than another special pleading rather than the voice of reason.

In this sense to use corrupted science, for however allegedly worthy an end, is to inevitably and irretrievably corrupt science itself. No one who genuinely cares about good public policy, policy crafted on the basis of careful argument, cogent reasoning, and compelling data, policy that can stand the test of careful probing and consistent dissent with countenance the corruption of science.

But the use of bogus ETS science to attempt to manipulate the public policy debate on smoking threatens not just science, but also the standards of rationality that distinguish legitimate public policy. Adherence to the norms of rationality require that the identification of problems, causes and solutions be based on empirical evidence of the most rigorous sort, evidence that is specific, strong, consistent, and coherent and on rational arguments that are clear and logically compelling. Problems and solutions that cannot meet this standard of argument are not allowed a place in the public policy process since to do so is to abandon commitment to reason as a foundational democratic value. Yet the use of corrupted ETS science as a basis of public policy is nothing less than a abandonment of rationality as a measure of legitimate pubic policy. As we noted above, the Paper's ETS "science" cannot meet any of the tests of rationality that determine legitimate public policy problems and solutions. The ETS "evidence" is not specific, strong, consistent, or coherent, and if it fails these tests, it cannot provide compelling rational reasons - as opposed to rhetorical and emotional reasons - for its public policy recommendations. The use of corrupted ETS science, is however, more than simply an abandonment of reason in the public policy process, it is also something for frightening, the attempt to institutionalise a particular irrational view of the world as the only legitimate perspective: to replace rationality with dogma as the legitimate basis of pubic policy. And only those who ultimately fear, if not loathe reason are comfortable with dogma as the basis of public policy.

There is, however, a third peril that the use of corrupted ETS science poses to democratic public policy and that is through its treatment of the question of risk. The question of risk is central to any modern discussion of harm and public policy. If everything depends on science, then everything depends on the science of risk assessment. And, this places a special moral burden on those who use the notion of risk and risk assessment in public policy debates to be certain that the concept is used with integrity and not simply as a lever to frighten. In one sense the misuse of the notion of risk is simply another instance of a fundamental contempt for reason in pubic life for it is an attempt to gain through irrational means something that careful argument denies one.

To use the notion of risk with integrity in public policy discussions would minimally involve:

  1. stating risk assessments in a way that does not exaggerate harms and allows for individuals to make their own decisions about balancing the risks and rewards of various courses of action;
  2. placing particular risk assessment within a general risk context in a way that allows one to answer the question of how significant is this risk compared to other risks associated with everyday living (the question thus becomes not simply is this activity risky, but does it carry a risk level that in other circumstances we would consider worrisome"); and
  3. conveying the full sense of both the inexactness of risk assessment and the complexity of risk assessment - even in the face of the popular preference for simplicity


(As C.P. Snow observed about complexity and ambiguity, "Even at the highest level of decision, men do not really relish the complexity of brute reality, and they will hare after a simple concept whenever one shows its head.")

Given the imperatives of the Lalonde Doctrine and the general aversion of the anti-smoking movement to individual autonomy, it is obvious that these standards of responsible risk discussion will be ignored in the ETS debate. Indeed, given the flimsy nature of the ETS "science", the last thing that the Paper wishes to have is a careful public policy discussion of the real risks that ETS exposure poses. Everything hinges on the public thinking that, for example, the non-smoking wives of smoking men have a 30 percent increase in their risk of getting lung cancer without stopping to think what this figure, even if it were true, might mean. As Peter Finch explains, what the figure really means is something quite different, once the risk involved is explained and contextualised, from what the individuals using the figure want the public to think that it means.

"The annual death rate from lung cancer among non-smoking wives of non-smoking men is of the order of six per 100,000. Among non-smoking wives of smoking men the corresponding figure is eight per 100,000. Thus, two in every 99,994 non-smoking wives of smoking husbands die of lung cancer that, it is claimed, should be attributed to the effects of passive smoking. This is an exposure risk of almost one in 50,000, about the chance of tossing 16 heads in a row. To put such a small exposure risk into perspective note that, in Australia, the death rate from injuries and poisonings for males aged 15-24 years, at 101 per 100,000 is about 50 times as great... by emphasising a 30 percent increase in lung cancer due to passive smoking, health activists have led people to think that this is a high risk situation when that is not the case. (P. D. Finch, "Creative Statistics", Health, Lifestyle and Environment,1991)

Simple statements of risk, even assuming that the risk assessment itself is reliable, can be significantly misleading. Suppose, for instance, that I wish to promote the use of a certain heart drug, I claim that the drug reduces the death rate among patients from 4 out of every 100 patients to 3 out of every 100, something that means that the drug has a risk reduction factor of 25%. What I fail to tell you is that the absolute risk difference is only one percentage point and that I would have to treat 100 patients to save just one extra life. Although there is some truth to my original claim of a 25% risk reduction, my reliance on this figure alone and my failure to place this reduction in context actually significantly distorts reality. In effect, the Paper 's use of corrupted ETS science corrupts the entire discussion of risk in public policy through failing to note the inexactness of the assessment (and indeed exacerbating the inexactness through changing the confidence interval) through failing to contextualise the risk and compare it to other accepted forms of risk in everyday life and through failing to acknowledge the complexity of the entire issue.

The fourth peril that the use of corrupted ETS science presents for democratic public policy is that it undermines the value of fairness that is so central to democratic life. Fairness is undermined in at least two crucial senses. First, the debate about the nature of the evidence, its complexity and its contentiousness is never fairly acknowledged. The existence of significant dissent is rarely admitted and truth is make to appear simple, easy and unambiguously pointing in one policy direction, when in reality, as we have observed above, none of this is the case. Second, the fundamental requirements of fairness - the commitment to elicit, examine and consider, within an objective, open and non-arbitrary framework all views without using the power of authority to suppress and exclude - these requirements are flouted both in letter and in spirit by the corrupted ETS science's substance and process. Indeed, perhaps the defining character of corrupted ETS science is its fundamental non-objectivity, its process unfairness.

The fifth peril that corrupted ETS science presents for democratic public policy is the existence of an official, state-sanctioned scientific ideology that is used to morally stigmatise, morally degrade, morally exclude certain citizens from the civil community. For that is the ultimate public policy purpose of ETS science, to provide a compelling public policy justification for marking out certain behaviour as morally unacceptable. The quite horrible consequences of state-sanctioned science that prescribed certain acceptable ways of living and thinking and singled out certain individuals as moral reprobates in our own century should provide warning enough against the dangers of giving public policy legitimacy to a "scientific" dogma that morally degrades and excludes.

Finally, and perhaps most significantly, the use of corrupted ETS science in the public policy process threatens the central democratic values of autonomy, respect and diversity. The key to these values is the belief that individuals are equal in moral standing with the State, that they are the best judges of the shape of their own lives and should be encouraged to develop genuine diversity, and finally that they are also capable of understanding and participating in the life of the community, including choosing those who are to exercise authority. But inasmuch as ETS science is based on a selective and ultimately untrue reading of reality, an given that it is fundamentally designed to manipulate both individuals and the policy process into believing and doing certain things. It is impossible for such science to value autonomy, respect or diversity. The very nature of corrupted science is such as to deny that individuals can or at least can be trusted to make important and informed decisions about themselves, and most especially about their health. The agenda of corrupted science is at its core based on paternalistic assumption that only a few can think and act correctly, that only a few know the "truth" a truth that must be carefully nuanced for the many, that only a few must ultimately chart one moral/healthy/rational way to live, a way that the State will in turn enforce. In the end, perhaps paradoxically, corrupted ETS science goes wrong at the very start, for when it decides that the interests of health promotion and the interests of a smoke-free society take precedence over the truth, it sets itself on a course of manipulation, fabrication and misrepresentation that cannot but collide with the values of autonomy, diversity and respect.

References

1 "Policy Against Science: The Case of Environmental Tobacco Smoke", Risk Analysis, Vol.13,1993, P.16

2 Office of Health and Environmental Assessment, Office of Research and Development, US Environmental Protection Agency, Respiratory Health Effects of Passive Smoking: Lung Cancer and other Disorders, Washington, DC, December 1992, page 1-1.

3 See, for example, ibid., pages 1-4 and 1-8.

4 G. Huber, R. Brockie and V. Mahajan, Consumers Research in the United States,1991.

5 Science, 31 July 1992, page 607.

6 The Science Advisory Board within the EPA involves outside experts who provide peer review for Agency research. The less-than-objective nature of the SAB with respect to the EPA's ETS report is outlined by M. Perske, 'The Politicized Science of Tobacco Policy', Regulation 3, 1995, pages 11-15. Perske's account reproduces verbatim discussions from the SAB about the relationship between ETS and active smoking.

7 Journal of the American Medical Association, 29 July 1989, page 499.

8 R. Kluger, Ashes to Ashes: America's Hundred-Year Cigarette War, the Public Health and the Unabashed triumph of Philip Morris, Knopf, New York, 1996, page 692.

9 See EPA, op. cit., Appendix A, Addendum.

10 R. Brownson, M. Alavanja, et al., 'Passive smoking and lung cancer in nonsmoking women", American Journal of Public Health, 82, 1992, pages 1525-1530.

11 EPA, op. cit., page 5-48.

12 L. Garfinkel, et. al., "Involuntary Smoking and Lung Cancer: A Case-Control Study," Journal of the National Cancer Institute, 75, 1985.

13 Safeguarding the Future: Credible Science, Credible Decisions, EPA, Washington, DC,1991, page 17.

14 G. Kabat, et al., American Journal of Epidemiology, 142, No.2, 1995, pages 141-148.

15 E. Fontham, et al., Journal of the American Medical Association, 271, No. 22, 1992, pages 1752-1759.

16 Brownson, op. cit.

17 H. Stockwell, et al., Journal of the National Cancer Institute, 84, No. 18, 1992, pages 1417-1422.

18 See C. Redhead and R. Rowberg, Environmental Tobacco Smoke and Lung Cancer Risk, Congressional Research Service, 1995, page 2.

19 Loc. cit.

20 Loc. cit.

21 Redhead and Rowberg, op. cit., page 45.

22 Redhead and Rowberg, op. cit., page 40.


FORCES is supported solely by the efforts of the readers. Please become a member or donate what you can.



Contact Info
Forces Contacts
Media Contacts
Advertisers
Links To Archived Categories

The Evidence
Inside Forces
About Forces
Research
Writers
Book case