By Stephen Turner, Joanne Healy, Elizabeth Conner and Lucy Wands

figure 1

©damircudic/E+/Getty Images Plus

Introduction

The profound impact of the COVID-19 pandemic on health services and clinical higher education has resulted in the publication of a number of papers dealing with how services and clinicians may adapt to the crisis.1,2,3 As well as the impact on patients, clinicians and lecturers, we must of course consider students, whose future as clinicians faces a threat like none they could have anticipated. It is clear that the impact on teaching and examinations in universities and colleges worldwide required an immediate and radical response. The challenge was particularly complex where the curriculum involves a mixture of written exams, project work and clinical assessments, and the necessity of gaining approval of any changes from the General Dental Council (GDC). The four-year University of Edinburgh Oral Health Sciences (OHS) BSc Honours course includes a substantial clinical component allowing students to undertake dental treatments for patients from their first year, and to develop skills in subject areas for their final year dissertation. Teaching methods have included lecture/seminars, Top Hat classroom, flipped classroom, reflective practice, peer assisted learning, case based learning and journal clubs.4

This paper describes the steps taken from March to June 2020 to maintain these different strands of teaching and assessment, and includes ratings and comments from students across all four years reflecting their own experience of remote teaching and assessment completed during those four months. By placing the students' experiences centre-stage, it aims to inform future teaching and assessment in this course, and may also be relevant to similar clinical courses.

Materials and methods

The following sources are utilised:

  1. 1.

    COVID-19: teaching and assessment planning and delivery (paper for University of Edinburgh Board of Studies meeting, June 2020)

  2. 2.

    Simulated clinical case assessments report (June 2020)

  3. 3.

    An online survey conducted 26 May to 4 June, using University of Edinburgh supported software, asked OHS undergraduates to rate how useful they found different formats as a way of remote teaching and assessment. Some descriptive statistics are presented together with representative verbatim comments. The student feedback was given on an anonymous and voluntary basis, and on the assurance that examination results would not be influenced by their participation.

The immediate task was to deliver or rearrange the lectures, clinics, tutorials and other sessions scheduled for this crucial period in the academic calendar. Planning for the remote delivery of teaching sessions was conducted using Teams software, which was unfamiliar to both staff and students. Additional Teams channels for staff and students were set up, and training sessions in the use of teaching platforms such as Collaborate and Tophat were undertaken.

From Year 2 onwards, progression and graduation is dependent on achieving credits in clinical practice simulated clinical cases. As a result of COVID-19 these students were required to undertake virtual simulated case examinations to replace the normal face to face format, where two students were examined simultaneously in two separate examination rooms by two sets of two internal and external examiners. The intention was to make the 2020 online simulated case examinations as similar to the face to face ones as possible. Years 3 and 4 were given group mock clinical simulated cases in preparation for their restorative and paediatric cases, and year 2 and 4 students also had a mock periodontal case. Years 2, 3 and 4 students were asked for their ratings and comments on these assessments. All four years were asked about other elements of teaching and assessment delivered remotely, as well as more general aspects of the impact on student life and morale.

Results

Nineteen students completed the online survey out of a possible 29 (response rate: 66%). All four years are represented, although only three of the nine Year 1 students responded.

Methods of delivering teaching remotely

Recorded PowerPoint with voiceover

Over three weeks 15 lectures, including a small number given by outside speakers, were delivered as PowerPoint slides with added audio commentary. Access to this material did not require the students to log in at a particular time, and there was no scope for immediate dialogue or clarification online. Students were asked to rate how useful they found this format using a scale of 1 (not at all) to 10 (extremely). Nineteen responses were received. Responses suggest this format was well received, with a mean rating of 7.8 out of 10 (n = 19). There were no significant differences in mean ratings between years (Yr1: 6.6; Yr2: 8.6; Yr3: 7.4; Yr4: 9.0: F = 2.01; p = 0.15). Several made comments, the following two being representative:

I find listening to recorded lectures really helpful when I'm revising but a few of the lectures from external lecturers didn't have as much content in the actual slides and I missed having the interaction with the lecturer… Yr2: rating:7/10.

I think the recorded lectures are extremely handy especially when revising. One thing that can be difficult is not being able to ask questions there and then, even though we have the option to email… Yr3: rating: 8/10

Collaborate

Where a more interactive approach was required, lectures were delivered via Collaborate, where students logged on remotely at the scheduled time. Thus this software delivered real-time teaching much closer to traditional face to face lectures, while still providing a permanent recording which the students could refer to. Eleven such sessions were completed, involving all four years. These included Journal Club sessions, where comments and questions were integral to the nature of the session, and Statistics lectures, which often generate questions from students unfamiliar with statistical concepts and methods. The third year Collaborate session involved a mock exam.

The ratings for Collaborate classes had a mean of 7.7 (n = 18). Again there were no significant differences in mean ratings between years (Yr1: 6.3; Yr2: 9.4; Yr3: 6.7; Yr4: 7.7: F = 2.39; p = 0.11).

This format is useful because you are able to ask questions at any time. Yr1: rating:7/10

Having a run through of the exam was helpful, as I hadn't used Collaborate before. Splitting into groups made it feel as if we were at a normal revision session, so … the session was able to be kept the same to allow for discussion. Yr 3: rating: 7/10

However, 11 of the 19 students had technical problems, mainly relating to their home Wi-Fi connection and overload by other occupants, including poor sound, delay, and temporary loss of connection.

Collaborate with TopHat

Collaborate sessions with TopHat takes the interactive element one stage further by allowing a quiz format within a live session. A small number of these sessions were delivered, not involving all years. The eight students with experience of this method of teaching who responded gave it a mean usefulness rating of 8.8.

The following comments are from year 1 and year 2 students.

I love using TopHat, it either gives me a boost because I know that I have managed to learn and understand the lecture or it's a good way to know which areas I need to work on. Rating: 9/10

Allows for quick revision to see if what you had learned and remembered what just was taught. Is a good way of jogging your memory for beginning or end of sessions as it isn't very long or difficult to use. Rating:10/10

Preferred format

Students were asked which of the formats for remote teaching they preferred overall (Fig. 1).

Fig. 1
figure 2

Students' preferred remote teaching format (n = 19)

Full size image

There was a clear preference for using Collaborate. Many of the nine comments stressed its interactive quality.

I feel it's easier to interact and understand the content as you can ask questions and immediately have them answered if need be.

… being able to have a discussion with other classmates and the lecturer as we have the lecture is really helpful - questions that I might not have thought of are asked, which help with understanding the content.

Clinic cancellations

After 16 March all clinical sessions were cancelled or postponed because they were patient-based or took place in NHS premises. Students were asked to rate how much of an impact they felt this was on them personally with 1 = no negative impact to 10 =extreme negative impact. Figure 2 presents these ratings.

Fig. 2
figure 3

Cancellation/postponement of clinical sessions: students' assessment of negative impact (n = 18)

Full size image

Years 3 and 4 rated the impact of the cancellations more negatively than did Years 1 and 2 (Yr1: 6.0; Yr2: 4.6; Yr3: 8.3; Yr4: 7.7: f = 4.57; p = 0.02). For brevity, only comments from years 3 and 4 are shown here.

It would have been ideal to have the last few weeks at outreach - especially due to it being final year it obviously means we won't have any more clinical time to perfect doing restorations or any treatment. (7/10)

Unable to finish getting competencies, worried I will struggle when moving onto outreach. I feel I need more practice, very long time until next clinic. (8/10)

I'm personally quite a practical learner and take in a lot of information on clinic from the supervisors so I really missed being on clinic. I haven't had much restorative experience yet and was hoping to complete a couple of my paeds competencies before the end of term… (8/10)

I feel like I have missed out on vital experience and feel I will be less confident now going into placement in final year as we have had so long away from clinics and I have not finished a lot of my competencies and I am worried that this is negatively impact me next year when we are allowed back onto clinic. (10/10)

Remote examinations and assessments

Simulated clinical case assessments

Years 2 to 4 were asked if undertaking simulated clinical case assessments remotely using Collaborate presented any particular challenges: 11 answered yes, four no, and two could not say. Ten gave comments, including five reflecting particular concerns.

The link for the actual exam didn't work and I had to use my phone which made it difficult to see all information for the patient e.g. clinical photos and radiographs. This made me very panicky and stressed. Yr2

I did not enjoy having to interpret the radiography image as when I zoomed in/manipulated the image it blurred, and I feel this is less of an issue if had it in the flesh or on PACS. Yr3

I felt like because we would usually have the paper in front of us, it was quite time consuming flicking back and forth on the PowerPoint document to go back and check what you had written against what was in the information given to us. The clinical photos and radiographs also became blurry when you zoomed into them. I definitely feel like I struggled more than I would have face to face when delivering my answers and in the restorative case I felt I kept looking at my laptop screen and not at my notes… Yr3

I found it extremely difficult not being able to have the patient histories, clinical photographs and radiographs out in front of me. I found treatment planning was more difficult due to this because I was nervous I was not retaining information properly, and flicking from slide to slide just made it a bit more difficult for me. Yr4

I lost connection during the perio case and was unable to move the presentation back and forward. I also couldn't send a message to the examiners on Collaborate due to having no internet. I was very rushed for time so didn't think to try and call the emergency number. I also felt like I wasn't able to say my full answers without being moved on... I feel like extending the questioning time would've helped with this as it all seemed very rushed. Yr4

Poster and patient leaflet assignment

Year 3 students were marked on the team assignment to produce a patient leaflet on a chosen subject. Four of the eight reported problems with working as a team remotely, summarised by the following comment.

I think the process took longer because if we had been together we could have all sat around one computer and designed the leaflet however we had to all create our own designs and then try to merge all of our ideas together, but only one person could do this so it felt a little unfair in regards to trying to delegate so that everyone was contributing the same amount.

Five Year 3 students also reported problems on their individual assignments to produce and present an academic poster. These problems centred on the lack of normal contact with the audience of examiners and fellow students during their presentation, summarised by the following comment.

…it felt strange to present to my computer screen without having anybody visible to present to and I found it much more difficult. I also feel that it was more difficult then to gain marks for 'engaging the audience' (one of the marking points on the mark scheme), when it felt like I was not presenting to an audience at all as they had turned off their microphones and cameras.

Examinations using Questionmark

Students were asked to rate the exams delivered live in a timed format using Questionmark software, with a member of staff available online if technical problems arose. The rating used a scale of 1 (not at all) to 10 (extremely effective). Fifteen responded (year 3 did not use this format). Figure 3 presents these ratings.

figure 4

Fig. 3 Students' rating of the effectiveness of Questionmark exam format (n = 15)

The following two comments are representative of the generally favourable view of this format.

The exams were no different to a usual exam we would normally have so I liked that and knowing there were staff present prevented any panic that would occur if issues were to arise, we were also reassured a lot by staff making this even easier.

The software worked well and we were given all of the necessary information we may have needed in plenty of time before the exam (ie how it all works and what buttons to press etc). The practice exam was also very useful.

Four reported internet problems - three involving loss of connection and one of delay.

Students' view of the worst aspect of the COVID-19 crisis

The survey ended with the following question:

'Finally, what would you say has been the worst aspect of the COVID-19 crisis for you as a student?'

All responding students added comments, many referring to several themes. The most common of these was loss of clinical sessions, mentioned by ten students, and summarised by the three following quotes:

Missing out on patient clinics, worrying that I'm not progressing as I should be at this stage and the unknown of when clinics will be back up and running.

Having time on clinic cut short as I would have liked to have had the remaining few weeks of the semester with supervised clinics.

Missing out on the last few weeks of paediatric and restorative clinics.

Seven mentioned the impact on revision and the need to adapt to different exam formats:

Trying to find motivation to study and focus at home, the fact that some exams have been postponed till further notice.

Not being able to use the library and our common room as an area to revise, worry about online assignments.

Not having a routine and being able to study as I normally would have in Edinburgh in the lead up to exams - not having all documents in front of me during the simulated cases.

Twelve comments referred to uncertainty over the future, the loss of social contact, and stress, summarised by these comments.

Lack of money security and worrying about my accommodation for the next year, and how I will complete this degree as there will have to be new measures to ensure our and patient safety.

Being in complete isolation and not being able to see anybody as I could not get home.

The worry and stress of how it will impact my final year and if my graduation will be postponed.

For me personally … (not) being around the other students and all of the teaching staff. It was initially a really hard adjustment going from being so busy and I think now I've adapted to it...

Missing out on celebrating finishing final year in person with the rest of the year group and lecturing team/supervisors.

Discussion

Although the response to the online survey was not as full as hoped, the students were often preoccupied with other concerns, including accommodation, income, travel and family issues, and the 19 that did respond gave very full and valuable answers.

The majority of non-responses came from Year 1 students.

Both from a student and staff perspective delivery of lectures, tutorials and other sessions scheduled for the March to May period generally went smoothly. The exception to this was the clinic sessions, which required postponement or cancellation to maintain patient and staff safety. The complexities involved in returning to student-led clinical sessions have been detailed here.5

The pandemic has accelerated an already perceptible move toward online teaching in dental education.6,7 However, 11 of the 19 students reported technical problems with Collaborate sessions, mainly relating to their home Wi-Fi connection and overload by other occupants, including poor sound, delay, and temporary loss of connection.

Clearly this added to the stress involved in the final weeks of term leading to examinations. Similar problems were reported regarding live examinations and by Year 3 students presenting marked assignments.

The mock examinations permitted a 'road test' of IT reliability both for staff and students. Investigation of technical issues at this stage revealed that these were mainly due to the amount of people logged on at one time, some students not using the recommended browser, and some not using headphones or earphones. For the actual examination there was a maximum of two internal examiners and one external examiner logged in. The students were also encouraged to use external earphones or headsets and offered support, including an individual practice session to log on to Collaborate and check their connection and audio/visual quality. Students and external examiners were given instructions on what to do if IT problems occurred mid-exam, and an emergency telephone number should they be unable to log in. Every clinical simulated case examination began with an assurance that if students experienced any IT issues during either preparation or questioning time, then the timing would be stopped until the issue was rectified. Nevertheless, the response to the student survey question on this aspect revealed considerable concern, which, along with the loss of clinical training, represent the most challenging areas for the days ahead. It may be useful to investigate any technical issues in individual student accommodation, especially if there are several occupants all making heavy demand on household connectivity.

The OHS course in the 2020/21 academic year has moved to digital/hybrid teaching, based on tools and platforms already used by students and staff. This will better enable ongoing support from the University Digital Education Unit. Such adaptation mirrors that adopted in dental schools across Europe.8,9 A Cochrane Collaboration working group has reviewed how dental health services can be resumed - the first step towards re-establishing clinical teaching for dental and oral health sciences students.10 This resumption is vital not only for teaching purposes but also for the return to previous levels of patient care.11 While reports of the impact of the move to online teaching have begun to appear, these often are in the form of single interviews.12

The University of Edinburgh Oral Health Sciences course is one of four available in Scotland, but is the only one run over four years. It was not possible to broaden the scope of the survey across other institutions and courses, partly due to time pressure, and partly because some centres chose to delay assessments until the 2020-21 academic year.

Nevertheless, it is hoped that the findings of this survey regarding student experience will be relevant to other teaching centres both in Scotland and more widely.

Conclusion

This review of the steps taken to complete teaching and assessment of both academic and clinical performance in the face of the COVID-19 pandemic has shown that despite the loss of facilities and lack of preparation time, students were generally positive in their assessment of the University's response. Nevertheless, many have serious concerns about the completion of their studies and the impact on their future career.

In many areas of education the COVID-19 crisis has provoked both stress and a positive feeling of mutual support, illustrated by this final comment from a Year 3 student.

The uncertainty of everything has been quite difficult to process. However, nobody can change what's happened now and I know the teaching team have tried their best to make this time as smooth as possible for us all, and that big effort is greatly appreciated.

Acknowledgements

The authors would like thank the OHS students for their prompt and full responses to this survey at a time of considerable upheaval and stress for all of them. The following members of Edinburgh Dental Institute were instrumental in the design and delivery of online teaching and assessment: Andreana Austin, Zoe Coyle, Amber Jenkins and Louise O'Dowd.