Introduction

Learning outcomes have been prominent in the medical education literature for over twenty years.1 Their role in dentistry is also now widespread throughout the continuum of dental education and evident within regulatory documentation. Arguably there is considerable variety in terms of their quality and therefore their utility in the learning arena. We feel an improved understanding of the purpose of learning outcomes and strategies for their optimal development may change this situation. The aim of this paper is therefore to provide an overview of the function and structure of learning outcomes as they may be applied in dental education.

Background

Regulation in both dentistry and medicine is long standing but continues to develop. In particular, demonstration of ongoing competence and continuing professional development (CPD) have come to the fore with declarations of CPD and, in medicine, revalidation. In many professions, including dentistry, declaration of specific undergraduate and postgraduate curricula has also became more widespread and nationally adopted. In the United Kingdom, this has taken the form of the regulator, the General Dental Council (GDC), publishing documents which outline the standards and requirements for providers of accredited training programmes.2,3,4 Documents for primary registration, maintenance of registration, and for speciality education refer to the programme assessment being appropriate to demonstrate attainment of the relevant learning outcomes to be monitored by the GDC.5,6

Who is using learning outcomes and what is driving their use?

The use of learning outcomes in professional education has been adopted by multiple stakeholders for differing purposes and may reflect an increasing engagement with regulation, fiscal probity and public accountability. The move to outcome-based education has evolved to a point where learning outcomes are very visible in policy documentation and integral to the design of a curricula as evidenced in learning and assessment strategy.7,8 Regulatory bodies adopting this approach can now describe and audit key attainments within programmes. From a regulatory perspective, demonstration of attainment of learning outcomes offers a welcome assurance of quality. Ultimately this may result in the alignment of curricula and a greater standardisation of the endpoint which results in a professional qualification, thus giving access to regulated clinical practice despite programmes being delivered by a diverse group of education providers.

Today, undergraduate dental students have access to two main sets of learning outcomes: those developed and used within their own institution, and those published by the regulator. In addition to this, at a European level, competence documents have been produced for the new dental graduate.9 In the UK, the GDC document Preparing for practice2 contains the outcomes that education providers must demonstrate attainment of to satisfy UK registration requirements for dental professionals. The general principle of a document produced by a national regulatory body, outlining the requirements of a new graduate entering the profession, is mirrored by regulatory boards in other countries.10,11

Once qualified, all GDC registrants are required to undertake and record CPD to maintain registration. In 'Continuing professional development for dental professionals'12 the regulator outlines the requirements for verifiable CPD, this includes the need for course providers to have 'clear anticipated outcomes'. When determining the value of a particular course before enrolment, learners may review these outcomes to determine whether the course fulfils their needs. In addition, learning outcomes are used in course evaluation questionnaires to feed back to the provider that they have been achieved. We consider this process to encompass the participants' self-evaluation of their achievement of the stated outcomes.

Universities providing postgraduate courses in dental education will also publish their aims and learning outcomes to allow potential students to determine whether their future needs and aspirations will be addressed by a particular learning episode.

For speciality training, there are curricula which have been approved by the GDC for each of the specialist groups, for example, Speciality Training Curriculum: Paediatric Dentistry and the Curriculum for Speciality Training In Restorative Dentistry.4,13 These can be found on the GDC website,14 and are currently under review. The template on which these curricula are produced have outcomes recorded by knowledge, skills, attitudes and behaviours in key areas. With the GDC recently initiating this process of revision through the Special Advisory Committees of the Royal Colleges in all speciality curricula, the importance of optimising the outcomes they develop is both timely and critical.

With so much emphasis on learning outcomes it seems appropriate that as a profession we stop for a moment to understand what they are and how to write them.

What is a learning outcome?

Within the realm of learning and teaching there are a range of differing terms used to express educational goals including aims, objectives, standards and outcomes. These terms all feature in dental education and are differentiated and explained in Table 1.

Table 1 Learning terminology used within dental education with a description and an example
Full size table

To focus specifically on learning outcomes, these are statements outlining what an individual who has successfully completed a period of study will be able to do at the end of that programme. As an educational goal they are not aspirational (as an aim would be) but focus on the 'bottom line' and need to be seen to be achieved by assessment. They must be concise, applicable and accessible. When written well, they give clarity to the learner, to those developing and delivering a programme, and those scrutinising outputs.

What underpins a 'good' learning outcome?

There are multiple taxonomies which have been developed in relation to learning outcomes, including Bloom's taxonomy, structure of observed learning outcomes (SOLO), and Finks.15,16,17,18 Of these, Bloom's taxonomy tends to be most widely used, probably because it is the most simplistic and easy to apply. Hierarchical taxonomies can be useful if a programme uses a scaffolding or spiral19 approach to a curriculum with outcomes intended to assess declarative knowledge and expressed using active verbs. While outcomes developed to express learning outputs within cognitive and psychomotor domains tend to be easier to express and have obvious methods for assessment, outcomes addressing the affective domain can pose greater challenges, especially when it comes to assessing attainment.

To have utility, outcomes must be practical descriptors and give those using them clarity and support when designing the learning episode as well as direction on how to assess. When an explicit statement of achievement is articulated the outcome becomes measurable and a tangible link to assessment outputs. Further, learning outcomes are not static and will require ongoing maintenance with review/re-evaluating over time to consider whether they remain useful and fit for purpose (ie their validity).

What is the purpose of a learning outcome?

To understand the purpose of learning outcomes it is helpful to consider them from the perspective and function of various stakeholders; the learner, those involved with programme delivery, and those with roles in quality assurance.

Role for the learner

In terms of course selection and self-management during study, outcomes explicitly state what the learners need to achieve and should effectively signpost preparation for assessment.20 In many supporting documents outcomes have a tendency to be 'grouped' by underlying themes of commonality. Different approaches have been adopted for these 'groupings', the GDC have grouped their 'Preparing for practice' outcomes into four domains: clinical; communication; professionalism; leadership and management.2 Others have grouped them longitudinally by themes, eg caries, or included other themes of competences, eg critical thinking and health promotion.10,11

Role for programme development and delivery

Learning outcomes direct the planning, design, delivery and assessment of a curriculum and there should be constructive alignment between learning opportunities, content, methods of teaching, assessment and the outcomes.1,21,22 Outcomes give direction for learning and teaching, and while they can assist in developing appropriate delivery methods, less prescriptive outcomes are beneficial to avoid constraining those delivering learning to a specific mode of delivery. Outcomes should permit flexibility to adapt if warranted by individual learning styles, and allow scope for reflexive change at both the course and session level. Effectively, it is the destination which is the focus, without being too prescriptive about the means of travel (delivery of learning) to get there. Outcomes also suggest the most appropriate method for assessment in order to demonstrate attainment; ie, can a particular outcome be assessed in a written format, or does the content of the outcome necessitate practical activity better suited to an OSCE or Case-Based Discussion format? Outcomes are also integral to the blueprinting process of assessment design.

Role in quality assurance

Another key area in which learning outcomes impact is in quality assurance processes, both in terms of internal quality assurance mechanisms and supporting external validation. The GDC's document 'Quality assurance process BDS' which applies to undergraduate education, states that 'The Standards for Education require that providers only allow students to be awarded a qualification if they demonstrate a set of learning outcomes, which have been defined by the GDC...'.23 Therefore the ability of a programme provider to map their programme of study to an external reference set of outcomes from the regulator is essential in the process of demonstrating attainment of nationally accepted requirements.

Constructing learning outcomes

Areas to consider when developing learning outcomes are outlined, with a worked example, in Figure 1. Aspects to address when developing outcomes are the:

  • Level of detail/scope

  • Achievement level, and

  • Means of assessment.

Figure 1
figure 1

Suggested key stages in developing a learning outcome aligned to a worked example

Full size image

Level of detail/scope

There is debate about the level of detail or specificity a learning outcome should be written at. This would be influenced by the purpose that outcome was prepared for. Outcomes can be prepared at the programme level, the module/course level, or the individual session level. Different characteristics of each of these will affect utility: a very small number of (broad) outcomes can lead to insufficient detail for stakeholders, to the point that they become meaningless, (mis)interpreted or partially achieved. Being too broad results in outcomes which are difficult, if not impossible, to fully assess in one episode. Partially assessing outcomes then leads to challenges of quality assurance and may mean some component elements of an outcome are regularly assessed, others only superficially assessed or not at all. An example of a broad, poorly conceptualised outcome which is challenging in terms of assessment is: 'To be able to manage a dental practice'.

Conversely, if outcomes are written very specifically and focus on small elements of the whole, a lot of outcomes are required to cover the area to be addressed. This can be challenging in terms of keeping the number of outcomes manageable and contemporaneous, and also in maintaining acquisition or assessing competence of these outcomes.24 For example: 'To be able to deliver local analgesia to an upper left lateral incisor'.

Pitching outcomes fittingly for the intended audience, by developing outcomes with appropriate scope, gives learners a clear indication of how to measure their current performance and provides direction for further study which may be needed. Maintaining an achievable quantity and range of content in an outcome additionally reduces ambiguity for those mapping to attainment. Examples of this approach to developing an outcome follow:

By the end of this course you should be able to:

  • Plan, critique and safely deliver advanced restorations to patients within a supervised clinical environment and justify your methods and strategy by using relevant clinical and research data.

  • Formulate an Index of Orthodontic Treatment Need (IOTN) following an orthodontic assessment.

Appropriate for the level of achievement required

An outcome must be appropriate for the level of learner it is aimed at. A pre-clinical dental student learning their first clinical skills may benefit from having outcomes at a session level which could include elements like 'assemble a matrix band correctly', whereas students further through the programme would not expect a skill to be so tightly 'compartmentalised' and therefore 'manage and appropriately restore a carious lesion' would be more appropriate. However, even this is arguably too detailed for an end of programme outcome. Therefore, tailoring outcomes appropriately makes them purposeful; failure to do this means they will be disregarded by those for whom they are intended and they risk becoming counterproductive – present on paper, but not fulfilling their purpose or the needs of the learner.

Means of assessment

There is a questionable purpose in setting an outcome which either cannot be assessed, or for which there is currently no assessment tool available. This point is especially valid if a parallel criterion is that tangible demonstration of attainment of the outcome is required.

Assessment, reliability, validity, feasibility and acceptability are important criteria to apply to the design of outcomes. In addition, there should be an element of driving future learning. Examples of learning outcomes providing guidance on how a quantifiable approach of demonstrating achievement/attainment of a skill, knowledge or behaviour are as follows:

  • 1.1.9 'Describe the properties of relevant medicines and therapeutic agents and discuss their application to patient management'2

  • 6.4 'Maintain and protect patients' information'.2

In these examples you can imagine how an assessment would be designed to explore and demonstrate attainment of this outcome. In the first example, this could be in the form of a written essay, and in the second, through a practical record keeping task and demonstration of information governance knowledge and application.

What goes wrong?

Recognising there may be subjectivity in writing outcomes is important, this includes consideration of the agenda of those drafting the outcomes and their insight into the learning episode or programme. Challenges with learning outcomes do arise when they are given a higher status than they 'deserve', or when they are (or are perceived to be) prescriptive and inhibit initiative and, within reason, wider interpretation. Having outcomes is one step in structuring learning, but 1) are they 'workable'? and 2) are they actually 'worked to'? are key questions to consider.

It cannot be ignored that many of the attributes desired in a dental professional are difficult to describe and/or quantify. This raises questions of whether learning outcomes are the most appropriate mechanism in this part of the development arena. Certain decision making and affective qualities, including the so-called 'soft' skills which are of paramount importance to the dental profession, may need alternate methods of articulating desired or required standards of achievement. This may be addressed by concepts such as assessment of competence through Entrustable Professional Activities25 or the application of professional standards as published by a regulator. In the context of this commentary on learning outcomes, if outcomes are considered to be appropriate, then those preparing them need to ensure they are optimally constructed.

For these reasons, involvement in and engagement with learning outcome development needs to be widely considered and reflected upon. Engagement between regulators, programme directors and learners has the potential to enhance their application and thereby ultimately achieve a result beneficial to all stakeholders. Those developing outcomes and designing curricula need to ensure that the delivery and the assessment structure are aligned to the same outcomes and understanding of purpose, and application is key if outcomes are to optimally fulfil their potential.

Conclusion

Learning outcomes are widely applied to many elements of educational activity both in general and more particularly, in dentistry. When well designed and constructed, they have the ability to aid both the learner and those delivering programmes, and they also have a fundamental role in aspects of quality assurance. Arguably the quality of these outcomes will affect their utility, so raising awareness of factors to consider when constructing outcomes is profoundly beneficial. Tangible outcomes, with sufficient specificity to permit fair and reliable assessment which clearly demonstrates attainment of the outcome, are required if we are to be truly convinced that this is the correct way to develop and learn.