SUBJECT: Requested review of OHEA document on Passive Smoking Health Risk Assessment
FROM: Patricia A. Murphy, Epidemiologist - MEDS
TO: Lynn Papa, Acting Chief, MEDS

Although we were asked to review specific proportions if this document, it is our collective opinion that this is a scientifically inadvisable means of approaching the issue.

We found that we were repeatedly forced to either accept certain statements and assumptions at face value or go back to the previous chapters to seek out the information needed for a thorough assessment of the validity of some of the analytical approaches which were taken.

Our office has had no involvement on the development of this important document and as such we are not aware of the rationale for selecting the chosen means of data analysis and presentation, nor are we aware of any scientific deliberations that have taken place in relation to previous reviews of this document.

It is our belief that if a document was not written in such a way that each chapter can stand on its own, then it cannot and should not be reviewed in that way. If the calculation which are used to generate estimates of risk are well known and straight forward then it is easier to review them for accuracy, clarity, and appropriateness of application. If, however, commonly used formulae have been adjusted or manipulated in any way, it becomes impossible to evaluate their validity if they are presented without supporting documentation and references for the various assumptions implicit in their use.

ECAO-CIN was asked to provide statistical reviews of Chapters 6 & 8 and Appendix B, and to provide epidemiologic reviews of Chapter 5 and Appendix A. It was evidently not recognized that most of Chapter 5 revolves around the statistical summarization and pooling of the results of approximately 30 epidemiologic studies of varying design and quality (often referred to as "statistical meta-analysis"). The severe time constraints of this requested review did not allow either Rick Hertzberg or me to review all these sections in a comprehensive manner. Rick's comments on Chapter 8 and Appendix B are included as an attachment to my comments here. I also briefly looked through Chapter 8 and Appendix B and concur with Rick's comments. In addition, we noted an error in Table B-15: Col. labelled "former smokers", line 2 now reads 38%. This should be 21%.

The primary focus of my comments is on Chapter 5. I also had to spend time reading though Chapters 1, 2, and 3 in order to familiarize myself with the document's overall thrust and to assess the validity of some of the measurements which are used to assess exposure to passive smoking. I have made a lot of comments in the document itself, not all of which will be summarized here, so the authors should look at the sections which are being returned for a full commentary.

As mentioned above, the time constraints of the allowed review time necessitate a less thorough commentary that would otherwise be warranted. In many instances, I have raised a lot of questions but have not being able to fully evaluate the potential impact of some of my concerns regarding data presentation, analysis and interpretation. I have tried to make my comments as constructive as possible, although brevity may make many of them quite terse. I will now address the major concerns I have regarding the Chapters I was able to read and assess.

Chapters 1 & 2: see minimal comments noted on document.

Chapter 3:

1. It is never really stated as to whether there is any concordance regarding levels of cotinine in blood, urine, and saliva although in some cases these different measurements are compared across studies. These measurements are used as the basis of several assumptions for the adjustment of some of the summary statistics presented and a more complete discussion of the sensitivity and specificity of the measures should be included as a justification for their use.

2. In several places, references are listed numerically, which is inconsistent with the rest of the document (see chapter for specifics).

3. Some parts of this chapter bear a rather striking resemblance to an article by Brian Leaderer (1990). Risk Analysis, Vol. 10 pp. 19-26. In some cases, the wording is identical. It may be that this person also authored this chapter. If not, this could be a source of embarrassment, as there are surely many people familiar with this article who will also be reading this EPA document in some capacity. This should be looked into by the authors.

Chapter 4:

Did not read, only looked at some of the tables for general information. No specific comments.

Appendix A: Reviews of Epidemiologic Studies on ETS and Lung Cancer. This appendix reviewed 32 epidemiologic studies and was 163 pages long.. As such, I could not review it in any detail on a study by study basis; I also did not have the original studies to refer to if so necessary. I do have several specific recommendations to improve this section. Right now, the introduction does not really say too much of interest. What is lacking is a clear statement of why these particular studies were selected for review. Was any attempt made to include non- published studies (which are likely to have non-positive findings) in the review? Chapter 5 uses these studies as the basis of a meta-analysis, but it is not really clear to me how the process of data abstraction took place, i.e., the specific pieces of information which were being sought from each study to form the data set for re-analysis. This could be presented in tabular form at the end of each studies' review and would be a very useful information source. It would also be helpful to present some type of summary of the histological classification of the lung cancers included in these studied stratified by smoking and ETS classification whenever possible. I tried to reconstruct this information for a couple of the studies but I was unable to with the information presented. I do recognize that this information was not always adequately presented in the original references.

Chapter 5:

1. This might be better titled REinterpretation of studies based on REanalysis of published data. Meta-analysis is repeatedly alluded to but it is never clearly stated what the purpose and form of this data analysis really is. For example, Sender Greenland has authored a magnificent explication of meta-analysis and meta-analytic techniques for specific application to epidemiology but this is not referred anywhere [Greenland S., 1987 (Quantitative methods in the Review of Epidemiologic Literature. Epidemiologic Reviews 9:1-30)]. In fact there is no reference other than the "methods of Mantel & Haenszel" provided for any of the statistical methods employed in this chapter. It is unclear what the objective of the analysis is because it is never stated.

2. An introductory section must be included which fully explains the purpose of this chapter, the data to be used, the statistical methods employed and ALL the assumptions and limitations inherent to their use. As presented, it appears that pooling of summary data (which have been adjusted with extraneous data) across epidemiologic studies is a daily exercise for most people. I do not think that this is the case. I recommend the following procedure outlined by Greenland to be sure that all necessary items have been included. For example, there is no discussion or mention of the "file drawer problem", i.e., the existence of unpublished studies showing no positive effects, which may bias the results of the data pooling in the direction of finding a positive effect.

3. I suggest that formulae be given in explicit terms for all the summary statistics presented. For example, right now we only know the relative weight of this each study included but do not know its true weight (inverse variance). There is also no mention of the potential effect of the variance (albeit it likely small) when using an odds ratio that has been extraneously adjusted but the variance formula for the (apparently) crude measure. It would be helpful to summarize some of these statistics in a more complete manner (see tables 1 & 2 in Greenland) so that they can be used by a reviewer/reader of this document to look at the data in other ways than those presented.

4. No where could I find a thorough discussion of the validity of the summary odds ratios, particularly regarding the assumption of homogeneity of the effect measure across the studies. It is alluded to indirectly but never stated. This is a crucial assumption in data pooling and should be explicitly discussed.

5. It seems that several studies were excluded a priori from meta-analysis for evidently different reasons. It is mentioned that the study of Sandles et al. was excluded to a small number of lung cancer cases -- I thought the purpose of pooling data was to include studies such as these. If it was totally useless, this should be more clearly stated.

6. I feel that the entire section of Statistical Interference (5.3) lacks clarity and assumes too much is known by the reader. This document will be read by a wide variety of people with different backgrounds and could become very confusing very quickly as presented. For example: a) the discussion of power issues in inadequate and there is no reference given for methods of calculation. It should probably be explicitly stated that "power = 1 - (beta-error)" as the text discusses and presents beta-error, but the Tables present power; b) in the section Test for Association (5.333.2.1) there is suddenly a reference to standardized odds ratios but we are never told how these were calculated -- I feel the data should be presented in a manner which is amenable to reanalysis by another party; Figures 5-1 though 5-4 which accompany this section are unclear to me -- there is no label on the x-axis for the log(OR) values but they are referred to in the text. It is also not clear whether the line on the graphs really represent the five equal areas under the normal curve which again are referred to in the text. How were confidence intervals calculated in section 5.3.2.2?; in Test for Trend (5.3.2.3) I feel the importance of the trend test and its associated p-value is overstated -- misclassification and measurement error can mask a dose-response trend but it can also sometimes create one. In some cases, there is a significant p-value but examination of the data shows that there is not a consistent upward trend in the odds ratios [(See M. Maclure and S. Greenland, 1992). Tests for trend and dose response: misinterpretation and alternatives. Am. J. Epidemiol 135:96-104)]. Also, it is not clear whether the p-values for trend tests are one-sided or two-sided and whether they all represent the authors' original calculations or are recalculations for this document -- were any of these estimates adjusted for misclassifications of smoking status? See the text for any further comments on these issues.

For section 5.4, the passages quotes from the Breslow and Day textbooks should be properly cited with the specific page references. The discussion of bias and confounding in the individual studies is pretty thorough, but there is no mention of the potential for residual confounding in or misclassification of erroneously measured variables used for the purpose of confounding adjustment. It is known (see Greenland, 1987 for citations) that this type of error can result in a variety of types and directions of misclassification bias of unknown magnitude. This might seem to be an overly picky comment, but I believe it should be discussed because I'm sure some outside critic will cite this source of potential error as possibility for the positive findings in these studies.

8. In the last part of Chapter 5, I feel that the case for a clear causal relationship between lung cancer and ETS is somewhat overstated. Specifically, there is what seems to be a misinterpretation of the term "specificity". The only studies which were looked at in this review involved lung cancer so this is not "specific" to the ETS issue. Specificity would involve, e.g., the finding of the same histologic type of lung cancer in all the studies. It is alluded to that adenocarcinoma seems the most strongly related to ETS exposure -- is it not curious that among active smokers squamous cell carcinoma is usually found in abundance and the relative risks for adenocarcinoma among active smokers are dwarfed in comparison with those for squamous cell? This has not been thoroughly discussed and I feel it detracts from the presumed causal relationship of lung cancer and ETS. I recall 7 or 8 years ago when it was first noted that adenocarcinoma seemed to occur with greater frequency in women compared to men. At that time, the theory was that these adenocarcinomas were likely due to domestic radon exposure; now they are being attributed to ETS. This issue should be better addressed before a causal relationship is "confirmed".

Chapter 6: I did not get to read this in any detail. I did wonder why no calculations were made for the individual studies regarding attributable fraction (AF) estimates using the calculated odds ratios. A true confidence interval could have been calculated for AF as they could have been compared across studies this way. Just another way of looking at this data, I guess.

Chapter 7: I did not look at it at all.

REFERENCES: Many of the references mentioned in the text are not included here. If the references are not up-to-date, this should be stated up front to reviewers to save them the time of searching for things which do not exist. I did note many typographical errors which I mark as such in the document.

Appendix C: Looked over, no review required (Summary Form to aid in classifying epidemiologic studies).

Appendix D: Incomplete as received. Sections D.2 and D.3 are missing, which makes it impossible to review the methods for calculation of the attributable risk estimates which form the basis of Chapter 6. It should be noted that the concepts of attributable risk (both population and among the exposed) and attributable fraction are never formally defined nor their calculations and limitations formally discussed nor properly referenced (see Rick's comments). This must be done somewhere.

This concludes my comments on this document. I hope that they will help to improve the quality of the final product.

Signed: Patricia Murphy