Validity study of self-reported pesticide exposure among orchardists

Abstract

Self-reported work histories are often the only means of estimating occupational exposures in epidemiologic research. The objective of this study was to examine the accuracy of recall of historical pesticide use among orchardists. All 185 orchardists in this study had participated previously in a cohort study of men occupationally exposed to pesticides. In that study (1972 to 1976), subjects were interviewed annually and asked to list pesticides used since the last interview. In 1997, 265 of the 440 presumed-living orchardists from the original cohort were successfully recontacted and asked to complete a detailed questionnaire concerning their lifetime use of pesticides; 185 (69.8% of farmers successfully contacted) agreed. Considering the 1972–1976 data as the standard, sensitivity and specificity of recall were calculated for certain pesticides and pesticide categories. Sensitivity of recall was good to excellent (0.6–0.9) for the broad categories of insecticides, herbicides, and fungicides, for heavily used chemical classes, such as organophosphates and organochlorines, and for commonly used pesticides; it was lower and more variable (0.1–0.6) for specific pesticides. Recall specificity was greatest (0.7–0.9) for the least used pesticides and chemical classes, such as dithiocarbamates and manganese-containing pesticides, and was generally modest for the rest (0.5–0.6). There was no evidence of selection bias between study participants and nonparticipants. In conclusion, recall accuracy was good for commonly used pesticides and pesticide categories. This level of recall accuracy is probably adequate for epidemiologic analyses of broad categories of pesticides, but is a limitation for detecting more specific associations.

Introduction

Self-report is often the only means of assessing historical exposure in epidemiologic studies. However, concerns about the validity of self-reported exposures arise when the exposures vary qualitatively and quantitatively over time, are not particularly memorable, or when the time lag between exposure and reporting is great (Armstrong et al., 1992). Reporting error can result in either over- or underestimation of the health effects of a given exposure, depending on whether the reporting error is differential or nondifferential.

Pesticides have been associated with a number of adverse health outcomes, including several types of cancer and deleterious effects on the nervous, reproductive, and respiratory systems (Hayes and Laws, 1991; Blair and Zahm, 1995; Dich et al., 1997). The epidemiologic evidence for these associations, particularly for chronic diseases, has typically come from case–control studies using self-reported retrospective exposure data. However, we are aware of only one study that investigated the accuracy of self-reported historical pesticide use among farmers. Blair and Zahm (1993), in a study of mostly grain farmers, found generally good agreement between farmers' and suppliers' accounts of pesticide use. They also found that these farmers reported lifetime use of very few pesticides.

Several other studies have examined the quality of self-reported work histories (Baumgarten et al., 1983; Rosenberg et al., 1987; Stewart et al., 1987; Bond et al., 1988; Bourbonnais et al., 1988; Brisson et al., 1991; Rosenberg, 1993) and occupational exposure histories (Bond et al., 1988; Holmes and Garshick, 1991; Joffe, 1992; van der Gulden et al., 1993) in nonagricultural industries. These studies have found that self-reported work histories are reasonably accurate when compared with company or government records. However, recall of specific jobs and occupational exposures is highly variable. Throughout these studies, researchers find that recall decreases as the number of jobs or assignments held by subjects increases (Baumgarten et al., 1983; Rosenberg et al., 1987; Bond et al., 1988; Bourbonnais et al., 1988; Brisson et al., 1991; Rosenberg, 1993). In addition, validity or reliability of recall is greater for the fact of employment than for the dates of that employment (Stewart et al., 1987; Bond et al., 1988; Bourbonnais et al., 1988).

The present study examined recall of pesticide use among a cohort of farmers, mostly orchardists, in central Washington state (Washington State Epidemiologic Study Project, 1976). All subjects were participants in an earlier cohort study of men occupationally exposed to pesticides, which was begun in the early 1970s by the Washington State Department of Health. Subjects in the present study completed a detailed questionnaire eliciting data on demographics, lifestyle factors, and lifetime farming and pesticide practices. This information was used to examine agreement between the pesticides reported originally by these orchardists in the early to mid 1970s with those reported in 1997 for the same earlier time period. This study is part of a larger investigation of possible neurological effects from long-term pesticide exposure (Engel et al., in press).

Methods

Subjects

All subjects had previously participated in a cohort study of 1300 men carried out by the Washington State Department of Health from 1972 through 1976 (Washington State Epidemiologic Study Project, 1976). Of these, 739 (56.8%) reported their primary occupation as orchardist (i.e., farmer) in any of the original interviews and are the focus of the present analyses. Subjects were mostly non-Hispanic Caucasian (95.0%) and aged 18 to 88 at the time they entered the initial study.

Of the farmers in the original cohort, 299 (40.5%) were determined to be deceased, leaving 440 (59.5%) presumed-living subjects (Figure 1). Using information from various sources, including contact information provided by subjects in the original study, we located and attempted to contact 396 subjects (53.6% of all farmers, 90.0% of presumed-living farmers). Five subjects (0.7% of all farmers) were determined to be residing outside of Washington State and were, therefore, not included in the present study, leaving 391 potentially eligible subjects (52.9% of all farmers). We successfully contacted 265 of these potentially eligible subjects (67.8% of potentially eligible farmers) and invited them in the summer of 1997 to participate in the current study; 185 (69.8% of farmers successfully contacted) accepted. The 175 farmers who lived outside the study area or were presumed living but could not be located or reached for follow-up were similar to participants in the present study in age (mean=72.9 vs. 71.2 years, respectively), number of interviews in which they had participated in the original study (mean=4.0 vs. 4.1, respectively), history of disease (including various cancers and illnesses of the hematologic, nervous, respiratory, cardiovascular, gastrointestinal, endocrine, musculoskeletal, and urogenital systems) as reported in the original study, and pattern of pesticide use reported in the original study, including cumulative lifetime years of pesticide use (mean=22.1 vs. 21.8, respectively) and days per year pesticides were applied during that study (mean=10.0 vs. 12.0, respectively). The 80 farmers who refused to participate in the present study were also similar in age (mean=73.9 years), number of interviews (mean=4.0), history of disease, and pattern of pesticide use, including cumulative lifetime years of pesticide use (mean=22.3) and days per year pesticides were applied during that study (mean=10.3). Study participants underwent a 3- to 4-h assessment at a centrally located testing center. The study protocol was approved by the University of Washington and Oregon Health Sciences University Human Subjects Committees, and all participants provided written, informed consent.

Figure 1

Subject selection in the present study.

Pesticide Use Ascertainment

In the original study, subjects were interviewed approximately once each year. Subjects completed an average of 4.0 interviews, although 7.2% completed only 2 to 3. During the annual interview, each subject listed up to five agricultural chemicals to which he was “substantially” exposed since the previous interview.

In the 1997 follow-up study, each subject was given a self-administered questionnaire. This questionnaire asked detailed questions about the subject's use of pesticides throughout his farming/work career. The subject was asked to provide information on years of farming or occupational pesticide-related employment. For discrete time periods, subjects reported crops grown, number of acres of each crop, and activity (i.e., mixing, loading, or applying) when using specific insecticides, herbicides, and fungicides. Subjects were provided with a comprehensive list of pesticides used in the region to facilitate recall. Pesticide use information was solicited for 5-year increments from 1960; reports preceding 1960 were open-ended. The subject was asked to provide information on any pesticides used but not included in the list provided.

Data Analysis

Since the 1972–1976 pesticide use information was collected within the year of use, we assumed it was reasonably accurate and we treated it as the “gold standard” in all analyses. Although this study had no true gold standard for pesticide use in this population, the data available from the original interviews offered a rare and practical opportunity to assess accuracy of retrospective pesticide use reporting. Limitations in the original information or in comparisons between the original and follow-up information were addressed by performing subanalyses or complementary analyses when possible, as described below. Sensitivity was calculated as the proportion of those subjects reporting use of a particular agent in the original study who reported use of the same agent in the follow-up study (during the comparable time period). Specificity was calculated as the proportion of those subjects who had not reported using a particular agent in the original study who did not report using it in the follow-up study.

Sensitivity and specificity were calculated for use of (1) pesticides generally; (2) the functional classes: insecticides, herbicides, and fungicides; (3) the chemical classes: organophosphates, organochlorines, dithiocarbamates, and manganese-containing pesticides (a subset of dithiocarbamates); and (4) specific pesticides such as azinphos methyl, DDT, ferbam, lead arsenate, mancozeb, maneb, methyl parathion, paraquat, tetraethyl pyrophosphate (TEPP), thiram, zineb, and ziram that were selected because of their potential neurotoxicity. The categories (1, 2, and 3) were created by grouping the appropriate individually reported pesticides. Analyses were limited to chemicals with reported use by 10 or more subjects in the original study. Carbamates were not analyzed for this reason.

Because the original study solicited pesticide use information annually during the years 1972–1976, whereas the follow-up study solicited this information in 5-year blocks over a lifetime, one set of analyses compared data from 1972 to 1974 in the original study to data from 1970 to 1974 in the follow-up study. To examine the accuracy of exposure recall while allowing for some error in the timing of exposure, another set of analyses compared 1972–1976 in the original study to 1970–1974±5 years (i.e., 1965–1979 — the smallest incremental increase possible with our data) in the follow-up study. Lastly, to assess accuracy of recall of ever exposure, sensitivity analyses were conducted comparing 1972–1976 in the original study to lifetime use in the follow-up study.

All analyses were restricted to subjects in the current study. Comparisons of stratified sensitivities and specificities were done using a chi-square test with one degree of freedom. In all statistical tests, a 5% two-sided level of significance was used.

Results

The 185 subjects in this study had a mean age of 71.2 years in 1997, ranging from 49 to 96 (Table 1). All subjects were male; almost all were non-Hispanic Caucasian (95.1%). They had farmed an average of 47.8 years (range: 8–80). Most (56.5%) were still farming at the time of the follow-up interview; 21.5% had stopped farming within the preceding 9 years. Each subject had been interviewed an average of 4.1 times in the original study (range: 2–5).

Table 1 Selected demographics of subjects.

During the original study, subjects reported use of an average of 5.4 different pesticides during the 3-year period 1972–1974, consisting of 4.9 insecticides, 0.2 herbicides, and 0.4 fungicides (Table 2). As expected, these numbers were greater for the 5-year period 1972–1976, with a total reported use of 6.3 pesticides, including 5.7 insecticides, 0.3 herbicides, and 0.4 fungicides. In the follow-up study, subjects appeared to overreport pesticide use during this approximate time period, reporting an average of 8.8 pesticides used during the 5-year period 1970–1974, consisting of 5.5 insecticides, 1.5 herbicides, and 1.8 fungicides. The number of pesticides used in each 5-year period steadily increased between 1965 and 1979.

Table 2 Number of pesticides reported in original and follow-up studies by time period

Temporal and point patterns of pesticide use by these subjects as reported in both the original and follow-up studies are shown in Table 3. In the original study, insecticide use was reported by 97.3% of subjects in at least one interview; insecticides accounted for 91.1% of all pesticides reported. Herbicide use was much lower, reported by only 23.1% of subjects, and accounting for only 2.5% of pesticides reported. Fungicide use was also much lower, reported by 47.3% of subjects, and accounting for only 5.7% of all pesticides. Organophosphates and organochlorines were widely used, reported by 96.2% and 80.1% of subjects, respectively. Organophosphates accounted for 51.6% of pesticides reported, whereas organochlorines accounted for only 18.5%. Dithiocarbamates and manganese-containing pesticides were used by less than one quarter of subjects and accounted for only about 3% of all pesticides reported. Individual pesticide use ranged from a high of 84.4% of subjects and 16.9% of pesticides reported for parathion to a low of 0% of subjects for DDT.

Table 3 Selected pesticide usage patterns reported in original and follow-up studies

More subjects reported using insecticides during the original study than reported using them around that general time period in the follow-up study (Table 3). In contrast, fewer subjects in the original study reported using herbicides and fungicides than in the follow-up study. Reporting was also higher in the original study for use of organophosphates, organochlorines, and manganese-containing pesticides, but was somewhat lower for dithiocarbamates. Reporting of specific pesticides in the original study was also much higher than in the follow-up study for the most commonly reported pesticides in the original study. Reporting in the original study was frequently lower than in the follow-up study for the least commonly reported pesticides in the original study. This is illustrated most dramatically for DDT, a pesticide not reported by any subjects in the original study, even during interviews covering the year before DDT was banned. DDT was reported as used during the period by almost one third of subjects in the follow-up study.

During the follow-up study, all subjects reported using one or more pesticides on their crops at some time (Table 4). All had used insecticides, 89.7% had used herbicides, and 85.9% had used fungicides. Ninety-three percent reported using five or more insecticides, 35.7% used five or more herbicides, and 43.2% used five or more fungicides. The median lifetime number of pesticides used was 18.

Table 4 Lifetime pesticide use reported in follow-up study

In comparisons between the 1972–1974 period in the original study and the 1970–1974 period in the follow-up study, sensitivity was good to very good for use of pesticides (0.94), insecticides (0.90), herbicides (0.66), and fungicides (0.62) (Table 5). Specificity was much lower, ranging from 0.25 for insecticides to 0.49 for fungicides. Sensitivity was high for use of organophosphates (0.90), but lower for organochlorines (0.60) and dithiocarbamates (0.42), and very low for manganese-containing pesticides (0.09). Specificities ranged from 0.27 for organophosphates to 0.90 for manganese-containing pesticides. Sensitivities for particular pesticides varied greatly from a high of 0.72 for parathion to a low of 0.22 for phosmet; specificities ranged from 0.84 for ethylan to 0.48 for diazinon (Figure 2). In general, sensitivity was directly related and specificity inversely related to how widely and frequently a chemical or chemical class was used (Table 3).

Table 5 Sensitivity and specificity of pesticide use recall for different time periods in both the original and follow-up studies

Figure 2

Sensitivities and specificities of pesticide use recall, comparing follow-up study 1970–1974 to original study 1972–1974, ordered by increasing sensitivity within groups of increasing generality. (DDT is not included in the figure because no subjects reported its use in the original study.)

When we widened the matching intervals to 1965–1979 in the follow-up study and 1972–1976 in the original study, all the sensitivities increased or remained the same, but all the specificities decreased (Table 5). The greatest increase in sensitivity was observed for use of herbicides, organochlorines, azinphos methyl, diazinon, and phosmet. Specificity decreased most for organochlorines, DDT, diazinon, paraquat, and parathion. Most of these changes had a modest effect on interpretation of sensitivity and specificity estimates.

When we considered ever use of these chemicals reported in the follow-up study compared to 1972–1976 in the original study, all sensitivities increased appreciably (Table 5). Sensitivities became very high for pesticide functional classes (range: 0.87–1.00) and for organophosphates (0.99) and organochlorines (0.97). Sensitivities were good for dithiocarbamates (0.67) and fair for manganese-containing pesticides (0.41), and were good to excellent for most individual pesticides. This general pattern persisted in all subsequent stratified analyses.

In general, sensitivity was similar or moderately higher among orchardists 70 years of age or younger compared to those over 70, although few differences were statistically significant (Table 6). Specificity was generally similar. However, although sensitivity in the younger group was good to very good for many general pesticide categories, it was lower and quite variable for individual pesticides.

Table 6 Sensitivity and specificity of pesticide use recall by age, comparing follow-up study 1970–1974 to original study 1972–1974

Because in the initial study, subjects could report no more than five pesticides per interview, low specificities may partially reflect limitations of the original interviews rather than poor recall on the part of the orchardists (i.e., some “false positives” could represent correct retrospective reporting of pesticides that the subject was unable to report in the original interviews because of the five pesticide reporting limit in those interviews). In fact, 76% of orchardists reported five pesticides in at least one interview. Therefore, sensitivity analyses were performed comparing those orchardists reporting five pesticides in any interview to those reporting fewer than five in all interviews (Table 7). The number of subjects in these two categories, particularly in the group consistently reporting fewer than five pesticides per interview, was small for several chemicals; some estimates were consequently unstable. As expected, there was a general pattern of higher specificity among those orchardists consistently reporting fewer than five pesticides per interview. On the other hand, sensitivity tended to be lower in this group.

Table 7 Sensitivity and specificity of pesticide use recall by maximum number of pesticides reported in any interview, comparing follow-up study 1970–1974 to original study 1972–1974

Sensitivity analyses stratified by lifetime number of pesticides used revealed that for orchardists reporting use of more than 20 pesticides, sensitivity was almost always appreciably higher (range: 0.17–0.98 vs. 0.05–0.91) and specificity similarly lower (0.28–0.80 vs. 0.50–0.99) compared to those reporting 20 or less (data not shown). However, there was a strong inverse association between age and lifetime number of pesticides used; when number of pesticides was further stratified by age, this difference disappeared. Additional analyses comparing subjects with 50 or fewer years of farming to those with more produced estimates that were generally similar in both groups. Furthermore, results from analyses restricted to subjects still engaged in farming were similar to those calculated for all subjects.

Discussion

The purpose of this study was to assess the validity of self-reported retrospective pesticide use among orchardists. We found that recall of pesticide use 21–25 years previously was highly variable. Sensitivity was good for general pesticide categories, such as insecticides, herbicides, and fungicides. It was more variable for chemical classes (e.g., organophosphates, organochlorines) and for specific pesticides. Recall tended to be better in subjects 70 years of age or younger.

Orchardists in this study reported more pesticide use than farmers elsewhere. Blair and Zahm (1993), in a study of primarily grain farmers, found that 83% of farmers reported ever using insecticides, 54% herbicides, and 9% fungicides during their lifetimes. Few subjects reported use of five or more from any one of those groups. In contrast, the majority of orchardists in the present study reported using chemicals from all of those groups. The median number of pesticides used by orchardists in the present study was 18, and one quarter reported using 28 or more. This discrepancy between studies is most likely due to differences in study populations, since subjects in the present study were predominantly apple orchardists, and apples are one of the most pesticide-intensive crops in the United States (Ferguson, 1985). Participation of subjects in the original study may also have improved their recall in the follow-up study. However, the original study ended over 20 years prior to the follow-up study, and any appreciable memory booster effect is unlikely to have persisted.

The fact that overreporting was pronounced for herbicides and fungicides, but not for insecticides, suggests that this bias may be of concern primarily in certain situations. At the time of the original study, insecticide use in this population was already well established, but herbicides and fungicides were only beginning to become prominent. U.S. Department of Agriculture agricultural pesticide use trend data show that fungicide and herbicide use were increasing rapidly while overall insecticide use was changing relatively slowly between 1964 and 1976 (Osteen and Szmedra, 1989). Widening the comparison time period in the follow-up study produced more marked proportional increases in the sensitivities for herbicides and fungicides than for insecticides. The overreporting of DDT is likely due to the opposite phenomenon, since DDT was banned shortly after the beginning of the original study. Subjects may correctly recall using these chemicals around that general time period, but may be unsure of the specific years. Thus, the overreporting we observed may have resulted from examining a narrow time interval in a period of rapid change. Unfortunately, this study lacked the necessary data to test this hypothesis.

The low to moderate recall sensitivities for specific pesticides suggest the limitations of long-term recall of pesticide use among farmers using large numbers of pesticides. In general, the highest sensitivities were observed for those pesticides that were most widely and frequently used. A farmer is probably less likely to err in reporting the fact of use of a pesticide than in the temporal pattern of that use. In fact, we observed a general increase in sensitivity as the window was widened around the time period of interest (i.e., from 1970–1974 to 1965–1979). Given the appreciable number of pesticides used by these subjects and the frequently changing regulations, effectiveness, and economics of pesticides, it is not surprising that these orchardists would have difficulty recalling exactly when they started and stopped using particular ones.

The true recall specificities are probably higher than our estimates suggest. There are two reasons for this. The first is the approximation of time frames between the two studies. The most appropriate comparison was between 1972–1974 of the original study and 1970–1974 of the follow-up study. However, pesticides, which subjects reported using during 1970–1974 in the follow-up study but which were not reported during 1972–1974 in the original study (apparent “false positives”) may simply be ones that were used during the first, but not latter, part of the 1970–1974 time period. In fact, most subjects who reported such false positives also indicated use of these pesticides in 1965–1969. However, in many cases, with the notable exception of DDT (which was banned in the United States in 1972), these subjects also reported use of the pesticides in 1975–1979.

In addition, the original study solicited and recorded at each interview a maximum of five pesticides used by each subject. There was a substantial proportion of subjects (76%) who listed five pesticides in at least one interview and a high proportion who did this multiple times. These subjects may, in fact, have used more than those five pesticides during the relevant time period, but reported only five because of design limitations of the original interviews. Therefore, there may be some pesticide use correctly reported by subjects in the follow-up study that was not reported/recorded in the original study for this reason. These “false positives” would also reduce the estimated recall specificity, as well as contribute to the apparent overreporting in the follow-up study.

Some subjects may have forgotten to report use of some pesticides in one or more of the original interviews. There is no way to determine the extent to which this might have occurred. If these nonreported pesticides were used intermittently or were consistently not reported in the original study, but were reported in the follow-up study, then estimated specificities could be reduced. We believe that this situation is more likely than the converse, in which a subject were to report in the original study the use of a pesticide that he had not used, but not report this in the follow-up study; this would tend to reduce sensitivity.

Confusion around pesticide names can lead to a decrease in both sensitivity and specificity. Most pesticides have multiple names. We attempted to list several of the most common names for each pesticide; however, a subject might have used a particular chemical under a brand name not listed or he might have missed a name he knew while scanning through a list of unfamiliar names. Such errors would result in underreporting and a consequent reduction in sensitivity. Some subjects may have confused similar names of different pesticides and mistakenly reported pesticides that they had not used. This error would result in overreporting, with a resultant reduction in specificity.

One limitation of this study was the narrow time frame in which pesticide use data were originally collected. Having only a 5-year period from 1972 to 1976 prevented us from matching the comparison time frames more closely, thus artificially reducing agreement between data from the two studies. In addition, it limits the generalizability of our results since recall accuracy tends to be related to time elapsed since the event being recalled. One would expect recall to be better for more recent time periods. However, a caveat to this is that the number of pesticides used in any given period has increased over time, thus creating greater complexity that might adversely affect recall.

There was no evidence of volunteer bias in this study. Subjects who were lost to follow-up or who refused to participate were similar to participants in the present study in age, number of interviews, health status, and patterns of pesticide use at the time of the original study. Nonparticipation was probably due primarily to inconvenience. Sixty-three percent of refusers indicated only that they were not interested, but refusers were more likely than participants to live at least 1 h away from the testing center (39% vs. 28%, respectively). Orchardists are generally tied to the land they cultivate. Because they tend to be more directly involved than hired agricultural workers in ordering and paying for pesticides, they may have better memory of pesticide use. By restricting these analyses to subjects working as orchardists in the original study, we attempted to reduce the risk of selection bias resulting from differential loss to follow-up among different categories of agricultural workers. This may limit the generalizability of our findings to similarly involved persons.

Our validity results are comparable to those of Blair and Zahm (1993), whose study comparing farmers' recall of pesticide use to information provided by pesticide suppliers observed agreement of about 60% for use of insecticides and herbicides. We also observed similar patterns to those found in studies examining the quality of self-reported work or occupational exposure histories among nonfarmers. Joffe (1992) observed that sensitivity improved while specificity worsened among workers in the printing and plastics industries when exposures were described in more general terms. Other researchers have found accuracy of recall to be higher for the fact of employment than for the dates of that employment (Stewart et al., 1987; Bond et al., 1988; Bourbonnais et al., 1988). Our data suggest that farmers are better able to recall the fact of use of a given pesticide than its specific years of use.

In conclusion, we found pesticide use to be high in this cohort of elderly orchardists, with many reporting use of a large number of different pesticides over their lifetimes. There appeared to be a modest overreporting bias for herbicides and fungicides during the comparison period. Sensitivity of pesticide recall was good to excellent for broad categories such as insecticides, herbicides, and fungicides, for certain heavily used chemical classes such as organophosphates and organochlorines, and for commonly used pesticides. It tended to be fair to poor for less frequently used pesticides and chemical classes. Sensitivity increased as the time frame was widened, becoming generally good to excellent for ever use. Recall specificity was highest for the least used pesticides and chemical classes, such as dithiocarbamates and manganese-containing pesticides, and was generally modest for the rest. This level of recall accuracy is probably adequate for epidemiologic analyses of broad categories of pesticides, but is a limitation for detecting more specific associations.