Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Original Article
  • Published:

Reliability and validity of expert assessment based on airborne and urinary measures of nickel and chromium exposure in the electroplating industry

Journal of Exposure Science & Environmental Epidemiology volume 24, pages 622–628 (2014)Cite this article

Subjects

Abstract

The reliability and validity of six experts’ exposure ratings were evaluated for 64 nickel-exposed and 72 chromium-exposed workers from six Shanghai electroplating plants based on airborne and urinary nickel and chromium measurements. Three industrial hygienists and three occupational physicians independently ranked the exposure intensity of each metal on an ordinal scale (1–4) for each worker’s job in two rounds: the first round was based on responses to an occupational history questionnaire and the second round also included responses to an electroplating industry-specific questionnaire. The Spearman correlation (rs) was used to compare each rating’s validity to its corresponding subject-specific arithmetic mean of four airborne or four urinary measurements. Reliability was moderately high (weighted kappa range=0.60–0.64). Validity was poor to moderate (rs=−0.37–0.46) for both airborne and urinary concentrations of both metals. For airborne nickel concentrations, validity differed by plant. For dichotomized metrics, sensitivity and specificity were higher based on urinary measurements (47–78%) than airborne measurements (16–50%). Few patterns were observed by metal, assessment round, or expert type. These results suggest that, for electroplating exposures, experts can achieve moderately high agreement and (reasonably) distinguish between low and high exposures when reviewing responses to in-depth questionnaires used in population-based case-control studies.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1

Similar content being viewed by others

References

  1. Siemiatycki J, Day N, Fabry J, Cooper JA . Discovering carcinogens in the occupational environment: a novel epidemiologic approach. J Natl Cancer Inst 1981; 66: 217–225.

    CAS  PubMed  Google Scholar 

  2. Stewart PA, Stewart WF, Siemiatycki J, Heineman EF, Dosemeci M . Questionnaires for collecting detailed occupational information for community-based case control studies. Am Ind Hyg Assoc J 1998; 59: 39–44.

    Article  CAS  Google Scholar 

  3. Rybicki BA, Peterson EL, Johnson CC, Kortsha GX, Cleary WM, Gorell JM . Intra- and inter-rater agreement in the assessment of occupational exposure to metals. Int J Epidemiol 1998; 27: 269–273.

    Article  CAS  Google Scholar 

  4. Benke G, Sim M, Forbes A, Salzberg M . Retrospective assessment of occupational exposure to chemicals in community-based studies: validity and repeatability of industrial hygiene panel ratings. Int J Epidemiol 1997; 26: 635–642.

    Article  CAS  Google Scholar 

  5. Siemiatycki J, Fritschi L, Nadon L, Gérin M . Reliability of an expert rating procedure for retrospective assessment of occupational exposures in community-based case-control studies. Am J Ind Med 1997; 31: 280–286.

    Article  CAS  Google Scholar 

  6. McGuire V, Longstreth WT, Nelson LM, Koepsell TD, Checkoway H, Morgan MS et al. Occupational exposures and amyotrophic lateral sclerosis. A population-based case-control study. Am J Epidemiol 1997; 145: 1076–1088.

    Article  CAS  Google Scholar 

  7. Goldberg MS, Siemiatycki J, Gerin M . Inter-rater agreement in assessing occupational exposure in a case-control study. Br J Ind Med 1986; 43: 667–676.

    CAS  PubMed  PubMed Central  Google Scholar 

  8. Fritschi L, Nadon L, Benke G, Lakhani R, Latreille B, Parent ME et al. Validation of expert assessment of occupational exposures. Am J Ind Med 2003; 43: 519–522.

    Article  Google Scholar 

  9. Friesen MC, Coble JB, Katki HA, Ji BT, Xue S, Lu W et al. Validity and reliability of exposure assessors’ ratings of exposure intensity by type of occupational questionnaire and type of rater. Ann Occup Hyg 2011; 55: 601–611.

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Teschke K, Olshan AF, Daniels JL, De Roos AJ, Parks CG, Schulz M et al. Occupational exposure assessment in case–control studies: opportunities for improvement. Occup Environ Med 2002; 59: 575–594.

    Article  CAS  Google Scholar 

  11. Tielemans E, Heederik D, Burdorf A, Vermeulen R, Veulemans H, Kromhout H et al. Assessment of occupational exposures in a general population: comparison of different methods. Occup Environ Med 1999; 56: 145–151.

    Article  CAS  Google Scholar 

  12. Steinsvåg K, Bråtveit M, Moen BE, Kromhout H . Inter-rater agreement in the assessment of exposure to carcinogens in the offshore petroleum industry. Occup Environ Med 2007; 64: 582–588.

    Article  Google Scholar 

  13. Rocheleau CM, Lawson CC, Waters MA, Hein MJ, Stewart PA, Correa A et al. Inter-rater reliability of assessed prenatal maternal occupational exposures to solvents, polycyclic aromatic hydrocarbons, and heavy metals. J Occup Environ Hyg 2011; 8: 718–728.

    Article  CAS  Google Scholar 

  14. Mannetje A, Fevotte J, Fletcher T, Brennan P, Legoza J, Szeremi M et al. Assessing exposure misclassification by expert assessment in multicenter occupational studies. Epidemiology 2003; 14: 585–592.

    Article  Google Scholar 

  15. Post V, Kromhout H, Heederik D, Noy D, Duilzentkunst RS . Semiquantitative estimates of exposure to methylene chloride and styrene: the influence of quantitative exposure data. Appl Occup Environ Hyg 1991; 6: 197–204.

    Article  CAS  Google Scholar 

  16. Teschke K, Hertzman C, Dimich-Ward H, Ostry A, Blair J, Hershler R . A comparison of exposure estimates by worker raters and industrial hygienists. Scand J Work Environ Health 1989; 15: 424–429.

    Article  CAS  Google Scholar 

  17. Hertzman C, Teschke K, Dimich-Ward H, Ostry A . Validity and reliability of a method for retrospective evaluation of chlorophenate exposure in the lumber industry. Am J Ind Med 1988; 14: 703–713.

    Article  CAS  Google Scholar 

  18. Guillemin MP, Berode M . A study of the difference in chromium exposure in workers in two types of electroplating process. Ann Occup Hyg 1978; 21: 105–112.

    CAS  PubMed  Google Scholar 

  19. Kiilunen M, Utela J, Rantanen T, Norppa H, Tossavainen A, Koponen M et al. Exposure to soluble nickel in electrolytic nickel refining. Ann Occup Hyg 1997; 41: 167–173.

    Article  CAS  Google Scholar 

  20. Pierre F, Diebold F, Baruthio F . Biomonitoring of two types of chromium exposure in an electroplating shop. Int Arch Occup Environ Health 2008; 81: 321–329.

    Article  CAS  Google Scholar 

  21. Lumens ME, Ulenbelt P, Géron HM, Herber RF . Hygienic behaviour in chromium plating industries. Int Arch Occup Environ Health 1993; 64: 509–514.

    Article  CAS  Google Scholar 

  22. Tola S, Kilpio J, Virtamo M . Urinary and plasma concentrations of nickel as indicators of exposure to nickel in an electroplating shop. J Occup Med 1979; 21: 184–188.

    Article  CAS  Google Scholar 

  23. Liu CS, Kuo HW, Lai JS, Lin TI . Urinary N-acetyl-beta-glucosaminidase as an indicator of renal dysfunction in electroplating workers. Int Arch Occup Environ Health 1998; 71: 348–352.

    Article  CAS  Google Scholar 

  24. Caglieri A, Goldoni M, Acampa O, Andreoli R, Vettori MV, Corradi M et al. The effect of inhaled chromium on different exhaled breath condensate biomarkers among chrome-plating workers. Environ Health Perspect 2006; 114: 542–546.

    Article  CAS  Google Scholar 

  25. Bavazzano P, Bolognesi R, Cassinelli C, Gori R, Li Donni V, Martellini F et al. Skin contamination and low airborne nickel exposure of electroplaters. Sci Total Environ 1994; 155: 83–86.

    Article  CAS  Google Scholar 

  26. Makinen M, Linnainmaa M . Dermal exposure to chromium in electroplating. Ann Occup Hyg 2004; 48: 277–283.

    PubMed  Google Scholar 

  27. NIOSH. Manual of Analytical Methods (NMAM): Method 7300 4th edn US Department of Health and Human Services: Cincinnati, OH, USA. 1994.

  28. NIOSH. Manual of Analytical Methods (NMAM): Method 8310 4th edn US Department of Health and Human Services: Cincinnati, OH, USA. 1994.

  29. Smith TJ, Kriebel D . A Biologic Approach to Environmental Assessment and Epidemiology. Oxford University Press: New York, NY, USA. 2010 pp 77–79.

    Book  Google Scholar 

  30. Landis JR, Koch GG . The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174.

    Article  CAS  Google Scholar 

  31. Ciccone G, Vineis P . Inter-rater agreement in the assessment of occupational exposure to herbicides. Med Lav 1988; 79: 363–367.

    CAS  PubMed  Google Scholar 

  32. Correa A, Min YI, Stewart PA, Lees PS, Breysse P, Dosemeci M et al. Inter-rater agreement of assessed prenatal maternal occupational exposures to lead. Birth Defects Res A Clin Mol Teratol 2006; 76: 811–824.

    Article  CAS  Google Scholar 

  33. Cherrie JW, Schneider T . Validation of a new method for structured subjective assessment of past concentrations. Ann Occup Hyg 1999; 43: 235–245.

    Article  CAS  Google Scholar 

  34. de Cock J, Kromhout H, Heederik D, Burema J . Experts’ subjective assessment of pesticide exposure in fruit growing. Scand J Work Environ Health 1996; 22: 425–432.

    Article  CAS  Google Scholar 

  35. Semple SE, Proud LA, Tannahill SN, Tindall ME, Cherrie JW . A training exercise in subjectively estimating inhalation exposures. Scand J Work Environ Health 2001; 27: 395–401.

    Article  CAS  Google Scholar 

  36. Vermeulen R, Stewart P, Kromhout H . Dermal exposure assessment in occupational epidemiologic research. Scand J Work Environ Health 2002; 28: 371–385.

    Article  Google Scholar 

  37. Bakke B, Stewart PA, Waters MA . Uses of and exposure to trichloroethylene in U.S. industry: a systematic literature review. J Occup Environ Hyg 2007; 4: 375–390.

    Article  CAS  Google Scholar 

  38. Park D, Stewart PA, Coble JB . A comprehensive review of the literature on exposure to metalworking fluids. J Occup Environ Hyg 2009; 6: 530–541.

    Article  CAS  Google Scholar 

  39. Pronk A, Coble J, Stewart PA . Occupational exposure to diesel engine exhaust: a literature review. J Expo Sci Environ Epidemiol 2009; 19: 443–457.

    Article  CAS  Google Scholar 

  40. Lavoue J, Friesen MC, Burstyn I . Workplace measurements by the US Occupational Safety and Health Administration since 1979: descriptive analysis and potential uses for exposure assessment. Ann Occup Hyg 2013; 57: 77–97.

    CAS  PubMed  Google Scholar 

  41. Froines JR, Baron S, Wegman DH, O’Rourke S . Characterization of the airborne concentrations of lead in U.S. industry. Am J Ind Med 1990; 18: 1–17.

    Article  CAS  Google Scholar 

  42. Friesen MC, Coble JB, Lu W, Shu XO, Ji BT, Portengen L et al. Combining a job-exposure matrix with exposure measurements to assess occupational exposure to benzene in a population cohort in Shanghai, China. Ann Occup Hyg 2012; 56: 80–91.

    CAS  Google Scholar 

  43. Burstyn I, Jonasi L, Wild TC . Obtaining compliance with occupational health and safety regulations: a multilevel study using self-determination theory. Int J Environ Health Res 2010; 20: 271–287.

    Article  CAS  Google Scholar 

  44. Koh DH, Bhatti P, Coble JB, Stewart PA, Lu W, Shu XO et al. Calibrating a population-based job-exposure matrix using inspection measurements to estimate historical occupational exposure to lead for a population-based cohort in Shanghai, China. J Expo Sci Environ Epidemiol 2014; 24: 9–16.

    Article  Google Scholar 

  45. Peters S, Vermeulen R, Olsson A, Van Gelder R, Kendzia B, Vincent R et al. Development of an exposure measurement database on five lung carcinogens (ExpoSYN) for quantitative retrospective occupational exposure assessment. Ann Occup Hyg 2012; 56: 70–79.

    CAS  PubMed  Google Scholar 

  46. Stewart PA, Carel R, Schairer C, Blair A . Comparison of industrial hygienists’ exposure evaluations for an epidemiologic study. Scand J Work Environ Health 2000; 26: 44–51.

    Article  CAS  Google Scholar 

  47. Pronk A, Stewart PA, Coble JB, Katki HA, Wheeler DC, Colt JS et al. Comparison of two expert-based assessments of diesel exhaust exposure in a case–control study: programmable decision rules versus expert review of individual jobs. Occup Environ Med 2012; 69: 752–758.

    Article  Google Scholar 

  48. Kromhout H, Oostendorp Y, Heederik D, Boleij JS . Agreement between qualitative exposure estimates and quantitative exposure measurements. Am J Ind Med 1987; 12: 551–562.

    Article  CAS  Google Scholar 

  49. Logan P, Ramachandran G, Mulhausen J, Hewett P . Occupational exposure decisions: can limited data interpretation training help improve accuracy? Ann Occup Hyg 2009; 53: 311–324.

    CAS  PubMed  Google Scholar 

Download references

Acknowledgements

This study was funded by the Intramural Research Program of the Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health.

Author information

Author notes

  1. Yu-Cheng Chen

    Present address: 5Current address: National Environmental Health Research Center, National Health Research Institutes, 35 Keyan Road, Zhunan Township, Miaoli County 35053, Taiwan.,

Authors and Affiliations

  1. Division of Cancer Epidemiology and Genetics, Occupational and Environmental Epidemiology Branch, National Cancer Institute, Bethesda, Maryland, USA

    Yu-Cheng Chen, Joseph B Coble, Nicole C Deziel, Bu-Tian Ji, Shouzheng Xue & Melissa C Friesen

  2. Formerly of the National Cancer Institute,

    Joseph B Coble

  3. currently 1412 Harmony Lane, Annapolis, Maryland, USA

    Joseph B Coble

  4. Shanghai Municipal Center for Disease Control, 1380 Zhongshan Road, Shanghai, China

    Wei Lu

  5. Formerly of the National Cancer Institute,

    Patricia A Stewart

  6. Stewart Exposure Assessments, LLC, Arlington, Virginia, USA

    Patricia A Stewart

Authors
  1. Yu-Cheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joseph B Coble
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nicole C Deziel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bu-Tian Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shouzheng Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Patricia A Stewart
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Melissa C Friesen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Melissa C Friesen.

Ethics declarations

Competing interests

The authors declare no conflict of interest.

Additional information

Supplementary Information accompanies the paper on the Journal of Exposure Science and Environmental Epidemiology website

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, YC., Coble, J., Deziel, N. et al. Reliability and validity of expert assessment based on airborne and urinary measures of nickel and chromium exposure in the electroplating industry. J Expo Sci Environ Epidemiol 24, 622–628 (2014). https://doi.org/10.1038/jes.2014.22

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/jes.2014.22

Keywords

This article is cited by

Search

Advanced search

Quick links