Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Original Article
  • Published:

Developing a job-exposure matrix with exposure uncertainty from expert elicitation and data modeling

Journal of Exposure Science & Environmental Epidemiology volume 27, pages 7–15 (2017)Cite this article

Subjects

Abstract

Job exposure matrices (JEMs) are tools used to classify exposures for job titles based on general job tasks in the absence of individual level data. However, exposure uncertainty due to variations in worker practices, job conditions, and the quality of data has never been quantified systematically in a JEM. We describe a methodology for creating a JEM which defines occupational exposures on a continuous scale and utilizes elicitation methods to quantify exposure uncertainty by assigning exposures probability distributions with parameters determined through expert involvement. Experts use their knowledge to develop mathematical models using related exposure surrogate data in the absence of available occupational level data and to adjust model output against other similar occupations. Formal expert elicitation methods provided a consistent, efficient process to incorporate expert judgment into a large, consensus-based JEM. A population-based electric shock JEM was created using these methods, allowing for transparent estimates of exposure.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1
Figure 2
Figure 3
Figure 4

Similar content being viewed by others

References

  1. Goldberg M, Kromhout H, Guenel P, Fletcher AC, Gerin M, Glass DC et al. Job exposure matrices in industry. Int J Epidemiol 1993; 22 (Suppl 2): S10–S15.

    Article  PubMed  Google Scholar 

  2. Peters S, Vermeulen R, Portengen L, Olsson A, Kendzia B, Vincent R et al. Modelling of occupational respirable crystalline silica exposure for quantitative exposure assessment in community-based case-control studies. J Environ Monit 2011; 13: 3262–3268.

    Article  CAS  PubMed  Google Scholar 

  3. Friesen MC, Coble JB, Lu W, Shu XO, Ji BT, Xue S et al. Combining a job-exposure matrix with exposure measurements to assess occupational exposure to benzene in a population cohort in shanghai, china. Ann Occup Hyg 2012; 56: 80–91.

    CAS  PubMed  Google Scholar 

  4. Lavoue J, Pintos J, Van Tongeren M, Kincl L, Richardson L, Kauppinen T et al. Comparison of exposure estimates in the Finnish job-exposure matrix FINJEM with a JEM derived from expert assessments performed in Montreal. Occup Environ Med 2012; 69: 465–471.

    Article  CAS  PubMed  Google Scholar 

  5. Van Tongeren M . A job-exposure matrix for potential endocrine-disrupting chemicals developed for a study into the association between maternal occupational exposure and hypospadias. Ann Occup Hyg 2002; 46: 465–477.

    CAS  PubMed  Google Scholar 

  6. Wernli KJ, Astrakianakis G, Camp JE, Ray RM, Chang CK, Li GD et al. Development of a job exposure matrix (JEM) for the textile industry in Shanghai, China. J Occup Environ Hyg 2006; 3: 521–529.

    Article  CAS  PubMed  Google Scholar 

  7. Bracken TD, Kavet R, Patterson R, Fordyce T . An integrated job exposure matrix for electrical exposures of utility workers. J Occup Env Hyg 2009; 6: 499–509.

    Article  Google Scholar 

  8. Wheeler DC, Burstyn I, Vermeulen R, Yu K, Shortreed SM, Pronk A et al. Inside the black box: starting to uncover the underlying decision rules used in a one-by-one expert assessment of occupational exposure in case-control studies. Occup Environ Med 2013; 70: 203–210.

    Article  PubMed  Google Scholar 

  9. Ditchen DM, Ellegast RP, Gawliczek T, Hartmann B, Rieger MA . Occupational kneeling and squatting: development and validation of an assessment method combining measurements and diaries. Int Arch Occup Environ Health 2015; 88: 153–165.

    Article  PubMed  Google Scholar 

  10. Morgan MG, Henrion M . Recent Milestones Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. Cambridge University Press: New York, NY. 1990.

    Book  Google Scholar 

  11. Walker KD, Evans JS, MacIntosh D . Use of expert judgment in exposure assessment. Part I. Characterization of personal exposure to benzene. J Expo Anal Environ Epidemiol 2001; 11: 308–322.

    Article  CAS  PubMed  Google Scholar 

  12. United State Environmental Protection Agency (US EPA) US EPA: Expert Elicitation Task Force White Paper. Environmental Protection Agency (US EPA): U.S.. 2009, p 1–143.

  13. O'Hagan A, Buck C, Daneshkhah A, Eiser JR, Garthwaite P, Jenkinson DJ et al Uncertain Judgements: Eliciting Experts' Probabilities. John Wiley & Sons Ltd: Chichester, England. 2006.

    Book  Google Scholar 

  14. Ramachandran G, Banerjee S, Vincent J . Expert judgment and occupational hygiene: application to aerosol speciation in the nickel primary production industry. Ann Occup Hyg 2003; 47: 461–475.

    CAS  PubMed  Google Scholar 

  15. Hastings W . Monte Carlo Sampling Methods using Markov chains and their applications. Biometrika 1970; 57: 97–109.

    Article  Google Scholar 

  16. Greenland S . Multiple-bias modelling for analysis of observational data. JR Statist Soc A 2005; 168: 267–306.

    Article  Google Scholar 

  17. Greenland S Introduction to Bayesian Statistics. In: Rothman KJ, Greenland S, Lash T (eds) Modern Epidemiology, 3rd edn Lippincott Williams & Wilkins: Philadelphia, PA. 2008.

    Google Scholar 

  18. Lash T, Fox MP, Fink AK . Applying Quantitative Bias Analysis to Epidemiologic Data. Springer Publishing: Boston, MA. 2009.

    Book  Google Scholar 

  19. Greenland S, Fischer H, Kheifets L . Methods to account for bias introduced by job exposure matrices. Risk Anal 2015, Submitted.

  20. Vergara XP, Kheifets L, Silva M, Bracken TD, Yost M . New electric-shock job exposure matrix. Am J Ind Med 2012; 55: 232–240.

    Article  PubMed  Google Scholar 

  21. Morgan MG, Henrion M . The Nature and Sources of Uncertainty Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. Cambridge University Press: New York, NY. 1990.

    Book  Google Scholar 

  22. O'Hagan A . Eliciting expert beliefs in substantial practical applications [Read before The Royal Statistical Society at ameeting on 'Elicitation' on Wednesday, April 16th, 1997, the President, Professor A. F. M. Smithin the Chair]. J Royal Stat Soc Series D(The Statistician) 1998; 47: 21–35.

    Article  Google Scholar 

  23. U.S. Census Bureau. DataFerrett. Census Population Survey 1992–1999 2009.

  24. U.S. Department of Labor, Electrical Safety-Related Work Practices, 29 CFR Part 1910; US DOL, Washington, DC 1990.

  25. Mccann M, Hunting K, Murawski J, Chowdhury R, Welch L . Causes of electrical deaths and injuries among construction workers. Am J Ind Med 2003; 43: 398–406.

    Article  PubMed  Google Scholar 

  26. Hintz H . Protection against electric shock. Telecommunications Conference, 2005 INTELEC '05 Twenty-Seventh International 2005. IEEE: Berlin, 647–654.

    Chapter  Google Scholar 

  27. Lombardi DA, Matz S, Brennan MJ, Smith GS, Courtney TK . Etiology of work-related electrical injuries: a narrative analysis of workers' compensation claims. J Occup Environ Hyg 2009; 6: 612–623.

    Article  PubMed  Google Scholar 

  28. Chi CF, Lin YY, Ikhwan M . Flow diagram analysis of electrical fatalities in construction industry. Work 2012; 41 (Suppl 1): 3757–3764.

    PubMed  Google Scholar 

  29. O'Hagan A, Buck C, Daneshkhah A, Eiser JR, Garthwaite P, Jenkinson DJ et al The Elicitation of ProbabilitiesUncertain Judgements: Eliciting Experts' Probabilities, Statistics in Practice. John Wiley & Sons Ltd: Chichester, England. 2006.

    Book  Google Scholar 

  30. Belgorodski N, M. G, Tolksdorf K, Schueller K . rriskDistributions: Fitting distributions to given data or known quantiles. Available from http://CRAN.R-project.org/package=rriskDistributions2014.

  31. Vergara X, Fischer H, Yost M, Silva M, Lombardi D, Kheifets LI . Job exposure matrix for electric shock risks with their uncertainties. Int J Env Res Public Health 2015; 12 (4): 3889–3902 doi:10.3390/ijerph120403889.

    Article  Google Scholar 

  32. Cooke RM . Building Rational Consensus. Experts in Uncertainty: Opinion and Subjective Probability in Science. Oxford University: New York, NY. 1991.

    Google Scholar 

  33. White E, Armstrong BK, Saracci R . Principles of Exposure Measurement in Epidemiology; Collecting, evaluating and improving measures of disease risk factors. Second ed. Oxford University Press: New York. 2008.

    Book  Google Scholar 

  34. Burstyn I, de Vocht F, Gustafson P . What do measures of agreement (kappa) tell us about quality of exposure assessment? Theoretical analysis and numerical simulation. BMJ Open 2013; 3: e003952.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Moye A . Elementary Bayesian Biostatistics. CRC Press: New York, NY. 2007.

    Google Scholar 

  36. Mester B, Schmeisser N, Lunzmann H, Pohlabeln H, Langner I, Behrens T et al. Development and evaluation of a tool for retrospective exposure assessment of selected endocrine disrupting chemicals and EMF in the car manufacturing industry. Ann Occup Hyg 2011; 55: 736–751.

    CAS  PubMed  Google Scholar 

  37. Hopf NB, Waters MA, Ruder AM . Cumulative exposure estimates for polychlorinated biphenyls using a job-exposure matrix. Chemosphere 2009; 76: 185–193.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

This publication was supported by Grant Number: 5R21OH009901 from CDC-NIOSH. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of CDC-NIOSH.

Author information

Authors and Affiliations

  1. Department of Biostatistics, UCLA Fielding School of Public Health, Los Angeles, California, USA

    Heidi J Fischer

  2. EPRI, EMF/RF, Palo Alto, California, USA

    Ximena P Vergara

  3. Department of Env. and Occ. Health Sciences, University of Washington, Seattle, Washington, USA

    Michael Yost

  4. PE, Enertech Consultants, Cambell, California, USA

    Michael Silva

  5. Liberty Mutual Research Institute for Safety, Hopkinton, Massachusetts, USA

    David A Lombardi

  6. Department of Epidemiology, University of California School of Public Health, Los Angeles, California, USA

    Leeka Kheifets

Authors
  1. Heidi J Fischer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ximena P Vergara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Yost
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David A Lombardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Leeka Kheifets
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Heidi J Fischer.

Ethics declarations

Competing interests

Regarding potential conflict of interest, we present the detailed information for each author. Dr. Vergara is currently employed by the Electric Power Research Institute (EPRI), a non-profit research organization geared toward research on generation, transmission and distribution of electricity. Dr. Kheifets has received funding from EPRI for other studies. For this work, Dr. Kheifets was funded, from a Center for Disease Control and Prevention-National Institute for Occupational Safety and Health (NIOSH) grant (5R21OH009901). Dr. Yost is a subcontractor on the NIOSH grant. Mr. Silva has received funding from EPRI and other sources. Mr. Silva, Drs. Lombardi and Yost received NIOSH support for participation on the expert panel. Dr. Fischer also received NIOSH support for work on this project.

Appendix

Appendix

Appendix Expert Model Descriptions

Expert 1

Expert 1’s strategy was to initially estimate the median exposure estimate using a weighted average of the provided proportion estimates for non-fatal and fatal electrical injuries. His model gave greater weight to the non-fatal estimates as the expert considered the injury outcome a broader surrogate for the electrical exposure of interest. The weighted proportions were then projected across a 30-year time period (that is, working lifetime in that occupation). Next, this expert’s knowledge of each occupation and the “face validity” of their likely exposure was used to modify the quantitative estimate. Estimates for the 25th and 75th quantiles were initially obtained by assigning a distribution to the final weighted proportion, and setting the 25th and 75th quantiles of that distribution equal to their respective quantiles being estimated. Specifcally, to estimate the 25th and 75th percentiles, he used the normal approximation to binomial distribution and applied a Z-score such that P(z<z0)=0.25 and P(z>z0)=0.75.

These estimates were then reviewed to ensure they were consistent with that expert’s knowledge of the risks present in each occupation. This expert emphasized the use of occupational knowledge in his methods over models utilizing the data as he felt that provided a more accurate measure of exposure distributions.

Expert 2

To estimate the likelihood of electric shocks in the JEM, Expert 2 adopted a probability model for the risk of electric shocks in each job category. The underlying framework for estimating the risk of shocks was a 2-stage “random box model” which simulates the process of drawing tickets representing a shock or no shock event at random with replacement from a box. The 2-stage model implies there are two boxes. Stage one of the model simulates the probability of shock for the individual worker, who has a fixed probability of getting a shock in each year of his working lifetime (Pshock). Stage two of the model simulates the experience of 100 workers who each had this fixed probability of getting shocked during their working lifetime. The box model for stage two is characterized by Pshock |30 Yrs, which represents the probability that one of the 100 workers in the room had a painful shock in their working career. This is derived from a binomial model calculating the probability of one worker being shocked at least once in a year given Pshock which is itself derived by taking into account both fatal and non-fatal accident rates.

His estimate for Pshock was created by assuming it is directly proportional to the probability of non-fatal shocks, PNFshock. On the basis of this proportional relationship, then the annual probability of injury, PNF, adjusted to an annual rate from the 8-year proportion QUOTE, is used to estimate the annual risk of a painful shock with a fixed constant: PNFshock=SNF * PNF where SNF is a constant that represents the number of painful shock events per non-fatal injury event. Conceptually, he imagined an event pyramid where there are many shock events occurring, and these events occasionally lead to a non-fatal injury that gets reported.

Next, he generated estimates for PFshock from the fatal injury data. Accident data from fatal injuries generates a different distribution of risk estimates. To correct this, so that the two distributions were comparable, he applied a power law adjustment to the distribution of PFshock from the fatal accident data to maximize the overlap in the risk estimates.

The final step was to create an average of the two estimates of Pshock from the data in order to find a final estimate for each job category. In cases where there was only one type of injury data (e.g., non-fatal or fatal) available for a particular occupation, he used this estimate directly. In cases where both non-fatal and fatal data were present, upon reviewing the data and based on the expert group discussion, he adopted a weighted average to estimate Pshock which would favor the non-fatal data. After the in-person meetings this weight was equal to 0.87.

He obtained Pshock |30 Yrs, as the probability of at least one of 30 independent Bernoulli trials, each with a probability of success Pshock, resulting in shock. He considered this the median estimate for the working definition. The 25th and 75th percentiles can be evaluated from the inverse cumulative binomial function, which returns the smallest value for which the cumulative binomial distribution is greater than or equal to a specified criterion value, here either the 25th or 75th percentiles, give P= Pshock |30 Yrs and N=100, for the 100 people in the room.

This expert placed the most weight on using data models to estimate risk, however, he still adjusted estimates based on his occupational knowledge if the model results seemed unaligned.

Expert 3

The initial assumption is that the non-fatal electrical injury rate is a crude proxy for electric shock potential. Median estimates were initially made for the top five occupations with injury rates that expert experience indicates as clearly having a high median rate (e.g., 95–98%) of exposure to electric shocks. These median accident rates were used to scale in a simple linear manner to other occupations with lower accident rates for an initial estimate for each occupation's median. These median estimates where then carefully reviewed and revised based on expert knowledge of occupations and review of the Classified Index of Industries and Occupations (Bureau of the Census 1990) to understand each job description. In addition, information on electrical injury sources in the workplace (Lombardi 2010) were used to inform the evaluation of the median values. To create rough starting points for 25% and 75% bounds, a normal approximation to the binomial distribution was assumed about the median. These preliminary bounding estimates were then appropriately revised by applying direct knowledge or experience. The 25%/75% estimates were initially thought to be too tight and were expanded to accommodate the 30-year working career criteria used in the process. In addition, the bounds were adjusted to be asymmetric as appropriate using expert judgment for each occupation. To derive the risk estimates for those occupations where expert experience was lacking, a comparison was made with better understood occupations having similar non-fatal injury rates and similar potential exposure sources. Expert judgment was then applied to develop the appropriate estimates.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fischer, H., Vergara, X., Yost, M. et al. Developing a job-exposure matrix with exposure uncertainty from expert elicitation and data modeling. J Expo Sci Environ Epidemiol 27, 7–15 (2017). https://doi.org/10.1038/jes.2015.37

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/jes.2015.37

Keywords

This article is cited by

Search

Advanced search

Quick links