Introduction

In the UK, the quality of dental education and training is overseen by the General Dental Council (GDC), the regulatory body for UK dental professionals. Dental education programme providers must demonstrate that they meet a set of educational standards determined by the GDC.1 The standards cover three domains: protecting patients; quality evaluation and review of the programme; and student assessment. Within the quality evaluation domain, standard 11 states that 'patient and/or customer feedback must be collected and used to inform programme development', and standard 12 states that this should also relate to student placements.1 This is not unique to dentistry; the use of feedback from patients and service users in the development of UK medical curricula is now included in the most recent requirements of medical education.2

Newcastle University School of Dental Sciences (NUSDS) runs two undergraduate degree programmes; the BDS, and the BSc in oral and dental health sciences (a clinical degree that allows graduates to register as dental hygienist therapists). Within these programmes there is already an established use of a short, 'just one thing', patient feedback card for formative assessment of individual undergraduate students' performance.3 This runs alongside more in-depth patient feedback questionnaires assessing students' communication skills in stage 4 clinics and the student outreach clinics (placements within local salaried service clinics). However, previously, patient feedback has not consistently and purposively been used to inform the development of our undergraduate programmes. While we might be able to assume which areas of the curriculum patients are able to comment upon, there is no evidence to robustly inform the development of a patient feedback instrument with that specific intent.

Aims

The aim of this study was to determine which areas of our dental training programmes patients felt able to comment upon, using qualitative research methods to enable the patient's voice and experience to feed into the production of a relevant, effective and valid feedback questionnaire used to inform future programme development.

Methods

A favourable ethical review was received from the Health Research Authority before commencing (REC reference 17/SC/0062). Information sheets were supplied to all participants and written consent to contact patients was obtained to allow participation to be organised. Written, informed consent for participation was obtained by the researcher (ZF) on the day of the focus groups. The same facilitator (ZF) conducted each focus group, following a topic guide.

Phase 1: Information seeking

Two focus groups of patients (n = 5 and n = 6) were held to discuss participants' experiences of receiving student dental care, being the 'subject' of teaching, and how they prefer to give feedback. Participants were adult patients, or the carers of paediatric patients, who had received dental care from students on the BDS or BSc programmes in either Newcastle Dental Hospital (NDH) or one of the student outreach clinics. Participants could be recruited from the paediatric, prosthodontic, and restorative clinics, or from outreach clinics. Participants were excluded from the study if they were under the age of 18, were being treated by non-student hospital staff, if they were unable to converse and understand complex themes and ideas in English, or if they lacked the capacity to consent to participate.

Phase 2: Questionnaire design

Themes and subthemes were identified from phase one data, and a set of questions was developed for each theme. Three questionnaires were developed, including the same questions on each, but using different response formats throughout. These formats were: dichotomous yes/no; five item Likert-type scale; and a 100 mm visual analogue scale (VAS). Each questionnaire had spaces for free text comments.

Phase 3: Content validity and face validity

Participants from phase one were invited to return to a second group (n = 4) to test the content validity of the questionnaires. They were presented with the three questionnaire formats and given time to read each one to assess whether the questions were an accurate representation of the areas discussed previously. Comments were sought on the clarity and format of the questionnaire and, in particular, preferences for any one of the three answer styles.

Phase 3: Distribution and collection, utility

A staff focus group (n = 4) and student focus group (n = 8) then discussed whose responsibility it should be to distribute the questionnaire, how often it should be distributed, and barriers and facilitators to the process. The utility of the questionnaire was explored, as was the best method for disseminating the results. Staff were eligible to participate if they provided clinical teaching on the BSc and/or BDS programme. The staff who participated were a combination of clinical academics and clinicians from the restorative, paediatric, hygiene and therapy, and outreach clinics. A combination of stage 4 and stage 5 BDS students participated. Clinical students from both degree programmes were eligible to participate.

Data analysis

All focus groups were audio-recorded and transcribed verbatim. Thematic analysis, using a framework analysis approach, was completed manually by ZF, who indexed initial emergent themes.4 Transcripts were independently reviewed by another in the research team and a consensus was reached about emergent themes and subthemes. Data collection and analysis occurred simultaneously, enabling findings from initial groups to guide subsequent discussions. Data were collected until data saturation was reached.

Results

The composition of the focus groups is shown in Table 1.

Table 1 Focus group composition
Full size table

Phase 1

Participants contrasted their experiences between NDH and general dental practice. By doing this they were able to comment upon the quality of the treatment received at NDH and the communication of our students and clinical staff. Participants reported that our students effectively explained processes to their patients:

'They tell you everything and explain everything. It gives you loads of confidence in them' (FG2, male).

However, it emerged that there were occasions when communication could be improved; for example, students needed to avoid talking to patients who were not wearing their dentures, as this made them feel vulnerable. Attendees also thought that the emotional impact of tooth loss needed to be considered more. One participant stated:

'The depression, when you lose your teeth…they don't realise how much it hurts you' (FG2, female).

Some participants reported that they would welcome the opportunity to explain, in greater depth, how their medical conditions affected their oral health because 'no people are the same' (FG2, female). There was a feeling that mental health problems should be spoken about more freely because the 'mental ones are never talked about' (FG2, female).

The experience of being a teaching 'subject' was explored and it emerged that part of the reassurance of receiving treatment at NDH came from knowing that the students were appropriately supervised. Participants welcomed students asking supervisors questions about the treatment they provided, suggesting that this added to the confidence in the care they received:

'I don't mind people asking questions and being given advice. It doesn't bother me at all. It makes you feel that things are being conducted carefully' (FG1, female).

Participants were perceptive of the relationship between supervising staff and students; the majority noted this to be a supportive one, even 'brilliant' (FG1, male). However, one participant recalled an episode where the approach taken by a member of staff affected her student's demeanour:

'One occasion I felt that the clinician was critical of the student in the presence of me, which made me feel uncomfortable…it affected the student and I could tell by her quietness and her facial expressions' (FG1, female).

While only one participant reported a negative experience like this, attendees felt that it was important for the school to monitor whether this behaviour was common. Attendees thought that this interaction could have negatively impacted the quality of the student's treatment and it could be dangerous if the staff member always had this effect. Discussion followed about the feeling of safety at NDH, and this was generally positive:

'I've always felt very safe and comfortable with a student and, again, it's the support the student gets from the clinicians' (FG1, female).

A contrast was noted between the time it takes to receive dental treatment at NDH and a dental practice, with appointments and courses of treatment taking longer at NDH. Participants felt that this was because treatment was being conducted more thoroughly, and also because patients were not 'hounded in and hounded out' (FG1, male) of the hospital. The fact that all treatment was checked by supervisors was reassuring to patients but it added time to appointments. Participants suggested that the staff to student ratio might need to be increased in places. One participant was concerned about how busy some clinicians were:

'When you think of the clinicians and all the people they've got to see and all the students they've got to support it raises a doubt.' (FG1, female).

Participants saw the value of providing feedback to the school, although only if the feedback questions are specific, not 'How can we do this better?' (FG2, female). The participants wanted more focus on the quality of specific aspects of treatment and 'also the way you feel' (FG1, female). One participant was very precise about how they preferred questionnaires to be formatted: a folded A4 sheet, which is easier to hold, and can be completed with it resting on the knee. Participants suggested a combination of tick-box answers and free text comment boxes that allow them to elaborate on answers because 'things don't always fit in little boxes' (FG1, female). All agreed that the questionnaire should be anonymous to enable honest feedback to be given. They suggested that completed questionnaires should be posted into a box in the reception areas. Participants thought that questionnaires should be available as part of a continuous feedback process.

Phase 2

Three versions of the questionnaire were designed by ZF and initially reviewed by the authors. Each questionnaire had the same question stems, phrased differently to accommodate three answer styles: dichotomous (agree/disagree or yes/no), five point Likert-style, or 100 mm visual analogue scale (VAS). Each questionnaire had opportunities for providing free text comments. There were 18 questions aligning with the emergent themes of phase one: quality of care, safety, staff support, and time. The full list of VAS questions is shown in Table 2. Figure 1 provides an example of the 'staff support' questions in the dichotomous answer format. Part of the 'quality of care' section is shown in Figure 2 to demonstrate the Likert-style questions; and the 'safety' section of the VAS questionnaire is shown in Figure 3. The questionnaire was a booklet with an explanatory paragraph on the front page and a small section 'for office use only', collecting information about the student clinic, course, and stage, to which the answers relate.

Table 2 Questions used in the questionnaire with visual analogue scale
Full size table
Fig. 1
figure 1

The 'Staff Support' section demonstrating the dichotomous answer format. The 'please circle' question was the same in all versions of the questionnaire

Full size image
Fig. 2
figure 2

Part of the 'Quality of Care' section demonstrating the 5 point Likert-style answer format

Full size image
Fig. 3
figure 3

The 'Safety' section demonstrating the 100 mm visual analogue scale (VAS) answer format

Full size image

Phase 3

Clarity and user friendliness

Patients were happy with the language used and the questions were easily understood. One patient said that the staff support section was too wordy but, overall, it was not burdensome:

'They're not too much to read and you know when you're reading words and you think "what does that mean?" They're quite, they're okay.' (Patient).

Patients preferred the VAS because they felt that this was providing the most feedback, especially when it was used alongside free-text comments. They valued being able to mark on a scale as it gave the 'most accurate measure of what you were feeling'; rather than yes/no answers, which was the least preferred answer style. All groups thought that the VAS contributed to the user friendliness of the questionnaire:

'That was quite comfortable for me (yeah, yeah)' (Patient).

'That's a nice, quick response, rather than having to look through multiple answers for each one which can take time.' (Student).

'It's quite easy for them…just to mark on quickly as the go down, instead of having to choose a specific word to describe how they feel' (Staff).

Content and face validity

One member of staff was concerned that there were too many questions on the questionnaire, however, patients thought it was an appropriate length, thorough and an accurate representation of the topics they discussed. Staff could not think of any additional questions, or see the need for any to be removed. Students agreed that the questions were relevant as they covered issues patients had raised with them in the past, for example, staffing levels and waiting times. A student suggested that a question should be added about how long they have to wait for a dental nurse to come and assist them, as this can contribute to the appointment length.

Utility

Staff felt that the questionnaire asked useful questions and that the answers would inform the quality assurance programme in the school by flagging areas for improvement and highlighting areas of success. Additionally, they felt that the questions would make supervisors reflect on their teaching practice. It was felt that the questionnaire would benefit the school, showing patients that we care about their opinions and that we 'want them to feel involved in everything that we're doing as a school' (Staff).

A student thought that the booklet format would encourage better completion of the questionnaire, and therefore make it a useful exercise:

'They're more likely to give more honest feedback because it's enclosed rather than just on a piece of card…the layout of it is better like that' (Student).

Students also thought that the anonymity would make it easier for patients to raise concerns but, because patients can get quite 'protective' of students, it was suggested that the introductory text should reassure patients that the supervisory staff and students do not see responses. Students could see that the school may want to make recommendations for change based upon questionnaire findings and they felt that students might be more likely to act upon these recommendations because they resulted from patient feedback.

Students were unsure about the utility of the questionnaire without the date and time of the appointment to which it relates. They were concerned that bullying or safety issues would not be able to be fully investigated without this. However, if they knew how the information was going to be acted upon, they thought the questionnaire would be useful because it would be reassuring for them to know that something would be done to help.

Issue and collection

There was a difference of opinion about questionnaire distribution. Students thought that staff and students should be removed from the process, meaning that reception staff should issue them, or they should be freely available in the waiting rooms. Staff thought that both parties should issue the questionnaires and, to increase the response rate, explain why they were being issued.

All agreed that questionnaire should be completed and posted in the waiting room so that patients do not feel they have to rush its completion. Potential barriers to questionnaire issue and collection included the general busyness of the clinic, making it easy to forget; the high burden of paperwork in the outreach clinics; and a potential clash with 'just one thing' cards.3 Dissemination

Staff and students agreed that both parties should be informed of findings because they relate to the quality of the care and teaching provided. Both wanted the process to be as transparent as possible. The school and hospital trust clinical governance afternoon was suggested as an opportunity for dissemination as all clinics are cancelled and staff and students all attend the session. The annual NUSDS staff 'education development day' was also suggested as a potential route for dissemination. Students suggested that its student-staff committees would be a good forum for discussion of results. Students also suggested that any small changes or findings could be fed back to students in the pre-clinic briefing meetings.

Discussion

This study reports the development of a questionnaire to collect data about patients' experience of receiving student care at NDH. The use of patient focus groups, or interviews, in the development of questionnaires is not novel; this methodology has been used in dentistry to produce questionnaires related to patient expectations of treatment, the impact of dental treatment, and patients' experiences of specific procedures.5,6,7 Wener et al.8 took this approach to design questionnaires assessing the communication skills of dental and hygiene students. Yielding similar findings to those presented above, their patients were able to comment upon their involvement in treatment plan discussions, explanations given by staff and students, and finer points relevant to communication.

Our research investigated patients' views on the overall experience of being a 'subject' of teaching in a dental teaching hospital setting. This meant that patients were able to comment upon a range of topics such as their safety and vulnerability, the staff-student relationship, and discussions about their physical and mental health. The context in which research takes place is important; the subtle differences between our findings and those of Wener et al.8 could be a reflection of cultural differences between Winnipeg, Canada, and Newcastle, UK. This highlights the relevance of using the local patient voice to shape feedback instruments.

Potential limitations of the study are the low number of participants in general, that few men were recruited and there was a narrow age range of patient participants. This may be due to selection bias of those recruiting patients to the study, or it could be that more females were interested in participating. Eight men were interested in participating, however, the researcher had to hold the focus groups when the largest number of people could attend. Participant age was not recorded by the researcher but, during discussion, it emerged that many of the participants were retired, and the researcher estimated that all patients were over 50 years of age. When phoning to arrange attendance at a focus group, younger patients seemed to struggle to get time from work for the research. As a result, the focus groups may not be representative of the patient population of the dental hospital. For future research, it may be worth considering holding focus groups outside normal working hours to increase participation from younger age groups. Such younger age groups may have had concerns that are not addressed by the questionnaire.

Only a small number of patients were able to attend the phase three focus group; particularly as severe weather affected patients' ability to attend and three people had to cancel on the day. However, there were representatives present from both of the phase one groups and the researcher was confident that data saturation was reached. In hindsight, patients could have been asked how they would like NUSDS to demonstrate that they are acting on patient feedback. Results will probably be posted on 'you said, we did' noticeboards in clinic waiting rooms but patients may have been able to suggest alternative methods.

A small number of staff were able to attend the phase three group, but they covered several disciplines: paediatrics, restorative dentistry, outreach clinics, and dental hygiene and therapy. It was felt that data saturation was reached and opinions had been sought from a broad enough range of specialties.

Students approached the research with maturity and professionalism; they cared about patients being able to provide honest feedback, recognising that patients can be quite protective of their students and that they can feel inhibited about saying something negative. Their concerns about what action would be taken if patients highlight bullying, for example, are justified and thought must be given to how negative feedback will be acted upon.

The process of involving patients in the development of the questionnaire was a valuable one; without it, the subtleties such as considering patient vulnerability and mental health, as well as finer aspects of questionnaire design would inevitably have been missing.

Before fully implementing the questionnaire it will be piloted for one week across all student clinics. Qualitative and quantitative analysis of the pilot results will enable the research team to gauge patient response to the questionnaire and the validity, reliability and utility of the feedback given. The student-staff liaison committee will be approached to discuss students' reactions to the questionnaire and their involvement in the process of issuing it to their patients. The results of the pilot will be taken to the central education committee to consider how the questionnaire results can be used within the existing quality enhancement and assurance processes of the school.

Conclusion

Patient feedback needs to inform dental programme delivery. Through focus groups, patients were able to identify areas of the BDS and BSc curricula that they would be able to comment on with validity. These themes were incorporated into a patient feedback questionnaire which has undergone content and validity checks and has been approved by staff and students of NUSDS. A one-week pilot will soon be undertaken across all student clinics. This will be evaluated before the questionnaire forms part of the quality assurance programme of the school.