Introduction

Significant variation in teaching methods, time allocated to orthodontics, as well as the level of orthodontic clinical experience and laboratory procedures afforded to undergraduates has been exposed in previous research.1,2 While experience of undergraduate orthodontic teaching is generally positive, staff shortages and funding pressures are a recognised barrier to optimal delivery of orthodontic teaching and limited use of information technology has been identified as a problem.1,3,4,5

Traditionally, undergraduate orthodontic teaching has taken the form of lectures, seminars and clinical teaching allied to limited use of problem-based learning.1,5,6 The use of technology has become embedded within medical and dental teaching in recent years.7,8,9,10,11 For example, Walmsley has explored the use of Podcasts in dental teaching and has shown that they are popular among students as they are flexible and can be accessed as and when they please.12 Similarly, an American study by McCann has shown that students feel that e-materials 'extensively' enhance learning.13 However, students often comment that they would like e-learning to supplement lectures rather than replace them.13,14

E-learning offers a range of potential affordances including accessibility, versatility, and reduced costs and teaching commitments once established. Its use, however, may be hindered by limited uptake of mobile technologies; learning may also be undertaken in an isolated environment, and knowledge gain has not been confirmed universally among students.15,16

E-learning and blended learning approaches, which incorporate a combination of e-learning and face-to-face consolidation of knowledge, have generally met with positive feedback among undergraduate students.7,8,11 A recent study has, for example, demonstrated both positive feedback and knowledge gain related to the use of orthodontic e-learning resources.11 The latter study was confined to the analysis of cephalometric radiographs with students assessed simply on their ability to locate hard tissue cephalometric points. The aim of the present study was to assess student learning experiences and knowledge gain among a cohort of undergraduate dental students exposed to e-learning resources covering a broad range of orthodontic topics including a glossary of orthodontic terms, indices and clinical cases. The null hypothesis was that there would be no improvement in student learning, experiences and knowledge when exposed to the orthodontic e-resource for a 6-week period.

Materials and methods

This study received ethical approval and the reference number is QMREC1370b.

Participants

Participants included 63 dental students in the second term of fourth year at Queen Mary, University of London, UK. All students had commenced orthodontic teaching in the previous term being exposed to a combination of fortnightly clinical sessions involving assessment and diagnosis of new patients, fortnightly seminars and a total of six orthodontic lectures covering basic aspects of orthodontic assessment and diagnosis. Regular teaching, particularly clinical teaching and seminars, continued over the study period. The study commenced in January 2015. A sample size calculation was not performed as numbers were pre-determined being dictated by the size of the year using convenience sampling methods.

Randomisation

There were eight student groups comprising of 7-8 students in each group in the academic year assessed. Random allocation was performed at group level with four of the eight student groups exposed to an e-learning resource for a period of six weeks (intervention group), while the other four groups were denied access to the resource over the study period (control group). Both sets of students continued to attend clinical sessions and seminars as normal throughout the study period (Fig. 1).

Figure 1
figure 1

Flow diagram of participant flow through the study

Full size image

E-learning resources

The e-learning resource was developed over a period of six months and comprised of six modules including: Fundamentals of Orthodontics, Types of Malocclusion, Appliances, Interceptive Treatment, Hypodontia and Referrals and Management of Emergencies. The resource has several interactive features such as videos on fixed and removable retainers, multiple choice questions with feedback for incorrect answers, access to definitions via a glossary of terms and also the ability to access further resources such as relevant papers. In this way, formative electronic quizzes were included throughout the e-resource. Two summative quizzes were provided to all students, one as a baseline, before the 6-week period and one to assess knowledge gain, after the 6-week period. Similar clinical cases to these were imbedded throughout the e-resource and presented in a variety of ways; for example, a description of a clinical case would be presented and the user would be requested to fill in the gaps in the diagnostic summary. Alternatively, the user may be asked a multiple choice question, based on interpretation of a radiograph from a clinical case. Its development was prompted by a lack of exposure of undergraduate students to patients in active orthodontic treatment and due to a feeling that students lacked practical knowledge and understanding of orthodontic planning and treatment. The resource, therefore, incorporated a range of material including definitions and fundamentals of orthodontics but also relatively simple diagnostic and treated orthodontic cases. The material was used by students for independent study for knowledge acquisition with blended use encouraged during seminars with orthodontic tutors. Knowledge acquisition was subsequently assessed with imbedded quizzes with real-time responses within each module (Fig. 2). These included both guidance when incorrect responses were given and further detail and explanation in the case of correct answers.

Figure 2
figure 2

(a) Example of multiple response question within the resource; (b) with feedback following incorrect response

Full size image

Initially, the orthodontic e-resource was piloted among undergraduate students; these students were not included in the study. We also obtained feedback from a postgraduate student, orthodontic specialists and non-orthodontic lectures. The resource was then made available to the test group through the student online Virtual Learning Environment (VLE), namely QMPlus.

Data collection

All students were asked to complete electronic quizzes both before (T0) and after (T1) the 6-week period in addition to a general questionnaire concerning familiarity with electronic learning. The test group also completed a user satisfaction questionnaire at T1. Baseline data obtained included demographics, familiarity with electronic and mobile devices and prior use of e-learning resources. Orthodontic knowledge was gauged using assessments comprising of analysis of records of an orthodontic patient. Both theoretical knowledge and diagnostic ability were assessed with similar formats adopted at T1 and T2 (Appendix 1).

Focus groups

Following the completion of the quantitative study, two focus groups involving 5-7 participants from the test group were undertaken to explore learners' experiences and suggestions in relation to the resource, and overall impressions of e-learning and its potential utility within the undergraduate course. A topic guide was developed before the sessions and sessions continued until saturation was achieved. The areas explored are indicated in Figure 3. Audio recording of the sessions were made. Framework analysis was carried out after the focus group sessions, in order to further analyse the qualitative data. Data analysis was undertaken using a qualitative approach by two researchers (SM, FC), which involves five distinct but interconnected stages: familiarisation, identifying a thematic framework, indexing, charting, mapping and interpretation.17

Figure 3
figure 3

Discussion topics during focus group session

Full size image

Results

Baseline characteristics

All students completed a questionnaire on familiarity with electronic learning (Fig. 3; Table 1). The majority of students were female (63.5%) with 93.7% aged between 22-25 years. All students reported having access to the Internet and 97% had a laptop computer. VLE usage was common to all students with 58.7% using online learning resources on multiple occasions weekly. Nevertheless, most students expressed a baseline preference for traditional learning resources such as lecture notes (65.6%) rather than e-learning (20.6%).

Table 1 Demographic and baseline characteristics of the sample (n = 63)
Full size table

Orthodontic knowledge/assessment

All students completed a validated baseline quiz before the intervention and also another validated quiz after the 6-week period. The students were presented with a clinical case and relevant case information such as intra-oral and extra-oral clinical photographs, dental panoramic tomograph radiograph and basic lateral cephalometric values. They were then asked a series of questions based on the case. An example of the baseline quiz can be found in Appendix 1. A different case was used before the study and after the study, but both cases had a similar level of difficulty and the same type of questions was asked. The sample of students that were exposed to the intervention (T1) were of mixed abilities, with 27.3% being the lowest score in the baseline quiz in both control and test groups and highest scores of 90.9% in the control group and 100% in the test group. There was no statistical difference between control and test groups in terms of baseline knowledge (t-test: P = 0.551). A marginal improvement in both control and test groups was observed over the 6-week study period and the overall mean score for all 63 students increased from 64.1% to 66.8%. The mean score for the control group increased by 3.9%, from 6.39 (with a standard deviation of 1.86) to 6.64 (with a standard deviation of 1.68). The mean score for the test group increased by 4.5%, from 6.42 (with a standard deviation of 1.48) to 6.71 (with a standard deviation of 1.9). No statistical difference in knowledge gain was observed between the groups (T-test: P = 0.941).

User satisfaction with the resource was very high with 90.3% feeling it met expectations and 100% agreeing that it fulfilled students' learning objectives (Table 2). Interactive elements were also considered beneficial by the vast majority (93.6%). Overall the resource was rated as 'very good' by 45.2% and 'good' by 51.6%. Students also commented on that the use of practical cases was particularly useful and the inclusion of quizzes and interactive elements with instant feedback was particularly beneficial and useful in consolidating learning.

Table 2 User satisfaction and feedback from the test group
Full size table

Focus groups

Students were keen to be involved in the focus group sessions and had very clear and consistent messages about their experiences and how they would like to see e-learning incorporated into their learning generally. A number of recurring themes emerged from the focus group sessions. Areas of orthodontics reported to be challenging for students were terminology, IOTN, lateral cephalometric tracing, treatment planning as well as designing, fitting and adjusting appliances. All of these topics have been incorporated into the e-resource. When asked to consider what aspects of orthodontic teaching could be improved, both groups requested more clinics and more support from tutors. One group also suggested synchronisation of lectures with clinics. This has been incorporated into the new curriculum and all students will benefit from this change from September 2016.

Both focus groups emphasised that they found the resource 'user-friendly', 'liked the graphics' and 'enjoyed the interactive quizzes.' Figure 4 summarises those elements students found most useful. During the discussion students felt the content 'enhanced knowledge', 'helped basic understanding' and many specifically mentioned the immediate feedback to questions as being particularly helpful. Most students would like the orthodontic e-resource to be incorporated into the curriculum in the form of blended learning and felt it was important to have the resource available on mobile devices.

Figure 4
figure 4

Spider diagram based on focus group discussion

Full size image

Discussion

The successful development of an undergraduate orthodontic e-learning resource has been demonstrated in the present study. The resource met with wide approval among the test group with students almost unanimous in believing it to be a useful adjunct to lectures and keen to explore further use of e-learning within other areas of dental teaching. The use of the resource did not, however, translate into a significant improvement in knowledge over the relatively short study period of six weeks. This response to the adoption of e-learning is typical with a previous randomised study failing to highlight additional knowledge gain during assessment directly after either e-learning, blended learning or face-to-face learning.18 Nevertheless, student feedback, particularly on blended learning, was positive.18

A potential problem with e-learning resources is lack of accountability of learners and difficulty in encouraging diligent use of the resource. During the study period multiple email reminders were sent to students to encourage use of the resource; it was therefore possible to collate data on all students. The e-resource was used as a stand-alone tool so that users could engage with the e-resource independently. They could re-visit the e-resource as often as they wanted and could re-take quizzes as necessary; this allowed students to hone in on personal areas of weakness and build on their knowledge of specific topics. The same e-resource was also used as part of a 'blended learning' approach. Students were requested to study certain modules in the e-resource with follow-up sessions with their orthodontic tutor. This allowed for face-to-face consolidation as the orthodontic tutor would ask the students questions based on the selected topics and also address any queries raised. This approach proved popular with students and is useful in increasing student engagement. Previous assessments related to e-learning within undergraduate groups have alluded to similar issues with student absence and illness allied to lack of motivation cited as potential barriers to enthusiastic adoption and rigorous assessment.18

The focus groups highlighted that e-learning needs to take account of students' expectations as they have very clear ideas of where e-learning can and should be incorporated into the dental curriculum. Moreover, students expressed clear preferences for specific features of the e-resource which will help to inform future e-learning development specifically access via mobile devices, specifically tablets. The present cohort of students had had previous exposure to e-learning resources within other dental speciality areas throughout their training. Increasing familiarity of dental undergraduates with modern technologies has previously been demonstrated in a UK-based survey and is likely to apply internationally.15 The present mixed-methods evaluation was inherently helpful in guiding further development of this learning resource as well as other new e-learning modules, particularly as the emphasis on a working knowledge of inter-disciplinary management of child patients increases within the undergraduate curriculum.19

While knowledge gain was not demonstrated this might relate to the relatively short study period of just six weeks, although the possibility that higher order knowledge was not explored in the assessment of knowledge acquisition. Prolonging the study may have produced more marked differences; however, depriving students of a potentially beneficial resource was not considered appropriate from an ethical viewpoint. Previous analyses have either incorporated assessment immediately following a series of one-off teaching exercises or having allowed a period of open access. Clark et al. also allowed students access to a cephalometric e-learning resource for six weeks, while shorter periods have been used in other research.18,20,21 In keeping with our own findings none of these studies demonstrated a significant improvement in student knowledge with e-learning, although one study focusing on space planning techniques alluded to a short-term improvement in knowledge gain which declined over time.18,20,21,22

The assessment of knowledge used was robust from an educational viewpoint involving assessment and diagnosis of orthodontic cases and was within the remit of 4th year undergraduate knowledge in keeping with General Dental Council requirements in the UK.18 Similar assessments have been used in allied studies both in the UK, continental Europe and in the United States.11,18,20,21,22,23 Students were particularly positive in relation to the clinical aspects of the resource in our study. This is intuitive, as undergraduates often have limited experience of clinical orthodontics with the emphasis at this level being on diagnosis. It would seem both reasonable and educationally stimulating, however, for learning to be placed in a clinical context to permit development of the requisite practical knowledge and understanding. Previous studies with clinical elements have led to similar conclusions with students viewing electronic resources as an adjunct to traditional methods of teaching.22

A potential caveat to the use of e-learning as the chief method of teaching is lack of familiarity with information technology.15,24 However, ownership of laptop computers was very high in the present group and in keeping with allied research of dental students alluding to 98% ownership among undergraduates in New Zealand. Previous research has suggested that most students (85%) have sufficient information technology aptitude to meet academic requirements; similarly this did not appear to be a hindrance within the present group of students. Moreover, mobile technology uptake is ingrained among dental students and may serve to enhance the ease with which teaching resources are accessed among undergraduates.24 Indeed, it is conceivable that enthusiastic use of electronic technologies among younger students may lead to improved uptake of e-learning resources among students in the future and may help to explain students' appetite for further similar resources throughout undergraduate training, despite the lack of a demonstrable improvement in their academic knowledge.

The existing resource will undergo further student-led modification to improve access and further enhance learning experiences. While dental staff members are known to have similar competence levels in information technology as their students, a further barrier to the institution of e-learning within undergraduate curricula may stem from reticence among teachers to adopt new and innovative teaching modalities.25,26,27 However, Handal et al. (2011) highlighted positive responses in a survey of 55 dental educators citing the flexibility, interactivity and accessibility of these resources despite a self-reported lack of competence in information technology.26 It is, therefore, likely that the development of stimulating and pertinent educational material will prompt educators to adapt teaching styles and modes to enhance student learning.

Conclusions

The e-learning modules have proved popular among 4th year undergraduate orthodontic students and, although within the short study period of six weeks did not result in statistically significant change in subject knowledge, were found to enhance the learning experience. Students found that the e-resource was a useful supplement to lectures and interactive features such as immediate feedback to questions were received particularly favourably. Further refinement of the resource with student input and direction will be undertaken.