Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Original Article
  • Published:

Choosing entrustable professional activities for neonatology: a Delphi study

Journal of Perinatology volume 37, pages 1335–1340 (2017)Cite this article

Subjects

Abstract

Objective:

To assess candidate neonatology EPAs taken from separate but overlapping sets from two organizations.

Study design:

Using a Delphi process, we asked that neonatology fellowship directors (1) assess importance and scope of 19 candidate EPAs, and (2) propose additional EPAs if necessary. In round 2, we sought clarification of first round responses and evaluated proposed additional EPAs.

Results:

Twenty program directors participated. In round 1, all EPAs were scored as important, but four were overly broad. In round 2, respondents rejected proposed subdivisions of one overly broad EPA, retaining it as originally proposed. Specification of entrustment criteria improved the scope of the three other broad EPAs. However, after specification, they were re-rated as insufficiently important and therefore rejected. Neither newly proposed EPA from round 1 was rated as sufficiently important.

Conclusion:

The Delphi process yielded 13 EPAs with which to assess capability to practice clinical neonatology.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1
Figure 2

Similar content being viewed by others

References

  1. Carraccio C, Wolfstahl SD, Englander R, Ferentz K, Martin C . Shifting paradigms: from Flexner to competencies. Acad Med 2002; 77: 361–367.

    Article  Google Scholar 

  2. Hawkins RE, Welcher CM, Holmboe ES, Kirk LM, Norcini JJ, Simons KB et al. Implementation of competency-based medical education: are we addressing the concerns and challenges? Med Educ 2015; 49: 1086–1102.

    Article  Google Scholar 

  3. Englander R, Carraccio C . From theory to practice: making entrustable professional activities come to life in the context of milestones. Acad Med 2014; 89: 1321–1323.

    Article  Google Scholar 

  4. Huddle TS, Heudebert GR . Taking apart the art: the risk of anatomizing clinical competence. Acad Med 2007; 82: 536–541.

    Article  Google Scholar 

  5. Lurie SJ . History and practice of competency-based assessment. Med Educ 2012; 46: 49–57.

    Article  Google Scholar 

  6. ten Cate O . Entrustability of professional activities and competency-based training. Med Educ 2005; 39: 1176–1177.

    Article  Google Scholar 

  7. ten Cate O, Scheele F . Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med 2007; 82: 542–547.

    Article  Google Scholar 

  8. ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M . Curriculum development for the workplace using entrustable professional activities (EPAs): AAME guide no. 99. Med Teach 2015; 37 (11): 983–1002.

    Article  Google Scholar 

  9. Warm EJ, Mathis BR, Held JD, Pai S, Tolentino J, Ashbrook L et al. Entrustment and mapping of observable practice activities for resident assessment. J Gen Intern Med 2014; 29 (8): 1177–1182.

    Article  Google Scholar 

  10. Jones MD Jr, Rosenberg AA, Gilhooly JT, Carraccio CA . Competencies, outcomes, and controversy—linking professional activities to competencies to improve resident education and practice. Acad Med 2011; 86: 161–165.

    Article  Google Scholar 

  11. American Board of Pediatrics. Entrustable professional activities for subspecialties: neonatology. Available at https://www.abp.org/subspecialty-epas#Neonatology(accessed 6 July 2016.)

  12. American Academy of Pediatrics. United States Neonatal-Perinatal Training Program Centers, 2012. Available at https://www2.aap.org/sections/perinatal/pdf/ONTPDDirectory12.pdf(accessed 12 March 2017.)

  13. de Villiers MR, de Villiers PJT, Kent AP . The Delphi technique in health sciences education research. Med Teach 2005; 27 (7): 239–643.

    Article  Google Scholar 

  14. Rowe G, Wright G, Bolger F . Delphi—a reevaluation of research and theory. Technol Forecast Soc Change 1991; 39: 235–251.

    Article  Google Scholar 

  15. Rowe G, Wright G . The Delphi technique as a forecasting tool: issues and analysis. Int J Forecast 1999; 15: 353–375.

    Article  Google Scholar 

  16. Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 2013; 67: 401–409.

    Article  Google Scholar 

  17. Boyce P, Spratt C, Davies M, McEvoy P . Using entrustable professional activities to guide curriculum development in psychiatry training. BMC Med Educ 2011; 11: 96.

    Article  Google Scholar 

  18. Shaughnessy AF, Sparks J, Cohen-Osher M, Goodell KH, Sawin GL, Gravel J Jr . Entrustable professional activities in family medicine. J Grad Med Ed 2013; 5: 112–118.

    Article  Google Scholar 

  19. Hauer KE, Kohlwes J, Cornett P, Hollander H, ten Cate O, Ranji SR et al. Identifying entrustable professional activities in internal medicine. JGME 2013; 5: 54–59.

    Article  Google Scholar 

  20. Rose S, Fix OK, Shah BJ, Jones TN, Szyjkowski RD . Entrustable professional activities for gastroenterology fellowship training. Gastroenterology 2014; 147: 233–242.

    Article  Google Scholar 

  21. Leipzig RM, Sauvigne K, Granville LJ, Harper GM, Kirk LM, Levine SA et al. What is a geriatrician? Ameican Geriatric Society and Association of Directors of Geriatrics Academic Programs end-of-training entrustable professional activities for geriatric medicine. JGAS 2014; 62: 924–929.

    Article  Google Scholar 

  22. Caverzagie KJ, Cooney TG, Hemmer PA, Berkowitz L . The development of entrustable professional activities for internal medicine residency training: a report from the education redesign committee of the Alliance for Academic Internal Medicine. Acad Med 2015; 90: 479–484.

    Article  Google Scholar 

  23. Schultz K, Griffiths J, Lacasse M . The application of entrustable professional activities to inform competency decisions in a family medicine residency program. Acad Med 2015; 90 (7): 888–897.

    Article  Google Scholar 

  24. Jonker G, Hoff RG, ten Cate O . A case for competency-based anaesthesiology training with entrustable professional activities. An agenda for development and research. Eur J Anaesthesiol 2015; 32: 71–76.

    Article  Google Scholar 

  25. ten Cate O . Nuts and bolts of entrustable professional activities. J Grad Med Educ 2013; 5: 157–158.

    Article  Google Scholar 

  26. Fessler HE, Addrizzo-Harris D, Beck JM, Buckley JD, Pastores SM, Piquette CA et al. Entrustable professional activities and curricular milestones for fellowship training in pulmonary and critical care medicine. Report of a multisociety working group. Chest 2014; 146 (3): 813–834.

    Article  Google Scholar 

  27. Messick S . The interplay of evidence and consequences in the validation of performance assessments. Educ Res 1994; 23 (2): 13–23.

    Article  Google Scholar 

  28. Downing SM . Validity: on meaningful interpretation of assessment data. Med Educ 2003; 37: 830–837.

    Article  Google Scholar 

  29. Downing SM, Haladyna TM . Validity threats: overcoming interference with proposed interpretations of assessment data. Med Educ 2004; 38: 327–333.

    Article  Google Scholar 

  30. Cook DA, Beckman TJ . Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006; 119: 166.e7–16.

    Article  Google Scholar 

  31. Newton PE . Clarifying the consensus definition of validity. Measurement 2012; 10 (1-2): 1–29.

    Google Scholar 

  32. Van Loon KA, Driessen EW, Teunissen PW, Scheele F . Experiences with EPAs, potential benefits and pitfalls. Med Teach 2014; 36: 698–702.

    Article  Google Scholar 

  33. Carraccio C, Englander R, Holmboe ES, Kogan JR . Driving care quality: aligning trainee assessment and supervision through practical application of entrustable professional activities, competencies, and milestones. Acad Med 2016; 14 (2 Suppl): S38–S54.

    Google Scholar 

  34. ten Cate O . Entrustment decisions: bringing the patient into the assessment equation. Acad Med 2017; 92 (6): 736–738.

    Article  Google Scholar 

  35. Okoli C, Pawlowski SD . The Delphi method as a research tool: an example, design considerations and applications. Inf Manag 2004; 42: 15–29.

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank the members of ONTPD for participation in this study and Jennifer Gong for her help in the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Department of Pediatrics, University of Colorado School of Medicine, Aurora, CO, USA

    T A Parker & M D Jones Jr

  2. Department of Medicine, University of Colorado School of Medicine, Aurora, CO, USA

    G Guiton

Authors
  1. T A Parker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G Guiton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M D Jones Jr
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T A Parker.

Ethics declarations

Competing interests

The authors declare no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Parker, T., Guiton, G. & Jones, M. Choosing entrustable professional activities for neonatology: a Delphi study. J Perinatol 37, 1335–1340 (2017). https://doi.org/10.1038/jp.2017.144

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/jp.2017.144

This article is cited by

Search

Advanced search

Quick links