Introduction

Peninsula is the first new dental school in the UK for a generation. With newness comes risk and a thirst for innovation. It also brings an opportunity to utilise evidence to inform strategy, and it is this opportunity which has not necessarily been available to older schools. For a new school to be 'successful', or judged as 'fit for purpose' it must produce dentists of the highest quality. It is 18 months until Peninsula produces its first graduates. At that point proof-of-concept for the school and its educational philosophy can start to be established but the journey towards producing first-class dental graduates begins with selection of the best students.

While many define 'best' as those achieving the highest academic grades at 'A' Level, or high achievement in some other type of academic test, the Peninsula view is different. We are a graduate entry school, and have chosen to focus on personal attributes as the criteria for entry while assuring aptitude for the academic challenges ahead through the fact that the students have acquired a science degree or reached an appropriate standard in the Graduate Australian Medical Student Aptitude Test. Peninsula's strategy has been to, firstly, define the attributes which typify an 'excellent' dentist. The second task was to identify which of these 'excellent dentist' attributes are innate characteristics of an individual, and which can be taught, developed or learnt through an appropriately delivered programme. Peninsula has therefore developed a selection process which selects students on the desirable characteristics of the 'excellent' dentist which are innate to an individual. Thereafter, it strives to deliver a curriculum which develops the mutable attributes of the 'excellent dentist'.

This paper describes the selection process at Peninsula Dental School. By comparing the results of the selection process with performance after the first year of the dental programme, the validity of the process is examined and the difference between innate characteristics and the learnable characteristics are highlighted. This paper therefore demonstrates the practicality of measuring by interview the attributes revealed by the literature as being important.

The aim of the study was to compare student performance in this evidence-based structured interview, with performance in an aptitude test, test of academic knowledge and test of ability to apply knowledge.

Materials and methods

A comprehensive review of the literature carried out by Allison et al.1 was used to derive a list of desirable attributes for a dental professional (Table 1) and these were the basis of the interview. The interview did not, include attributes which were deemed to be 'learnable' such as clinical, technical skills, competency or knowledge, as development of these is the objective of the educational programme.

Table 1 The mapping of attributes regarded as desirable in the literature, to behaviours measured in the situational interview
Full size table

Interview procedure

A situational interview was conducted as follows. Each prospective dental student was given three scenarios and underwent two non-assessed exercises. The non-assessed exercises were in order to encourage the student to not rely on preparedness. The students were given 30 minutes to study the scenarios and select one of them to discuss in their interview.

There were seven scenarios in total, of which three were offered to each candidate. Having selected their preferred scenario, each student met with three or four interview panellists who had no prior knowledge of the student (ie no information regarding past educational or exam history, and no knowledge of any titles or degrees held by the student).

Nine scripted questions with scripted prompts (for use when a student found a question difficult) were asked and the responses to eight were scored. The following is an illustration scenario and interview:

Catherine – Dental scenario 5

You are a newly qualified dentist. Catherine is a 15-year-old girl who has come to the dental surgery because her teeth are sensitive. She is extremely thin, and when you ask Catherine about her dietary habits, it becomes clear that she may have an eating disorder. Catherine becomes very distressed during the conversation and begs you not to discuss it with her mother. She says that things are bad at home, her younger brother has leukaemia and she doesn't get on with her mother's live-in boyfriend or her elder sister anymore. Her mother then arrives at the surgery - someone has told her that Catherine is there - and she demands to know what is going on and to see her daughter.

Q1 Key issues

Attributes: Communication, empathy, flexibility, honesty

Explain briefly what you think are the key issues raised in this situation. You should consider and identify the issues from the viewpoint of all those involved.

Prompt: Try to concentrate on the issues not the action you would take.

Good response: Identification of viewpoint of the patient, parents and health care team or identification of broad ethical issues such as rights, autonomy, paternalism etc.

Unsatisfactory response: Dogmatic failure to recognise and respect the point of view of one of the parties.

Q2 Decision making

Attributes: Communication, decision making skills, ability to work in a team, insight about self

You are in your first week as a newly qualified dentist in the dental surgery when Catherine attends and then her mother arrives. How would the decision on how to proceed and how to deal with her mother be made?

Prompt: What might you have to consider?

Good response: Recognition that would need to involve others including other health care providers.

Unsatisfactory response: Failure to consult with others entirely.

Q3 Team Work

Attributes: Communication, flexibility, team work, honesty, reflection, insight

In a situation like this you would need to consult the wider healthcare team. Can you give us an example of when you have worked in a team or a group - it may be a school group or a family group - what made it work well or not so well?

Prompt: How do you think your role and input impacted on the team or group's performance?

Good response: Analytical description of why group worked or didn't, recognition that everybody in the group has responsibility for group success. Some recognition that good team players play to others' strengths and cover for others' weaknesses.

Unsatisfactory response: Either that success was solely due to the candidate or failure due to the rest of the group and not the candidate.

Q4 Difficult decisions

Attributes: Communication, empathy and pro-social behaviour, decision making, reflection

We all have to make decisions at some time in our lives. We would like you to tell us about a time when you have had to make a difficult and significant decision and describe it to us,- explaining how you reached that decision?

Prompt: What factors did you consider when deciding on your career choice for example?

Good response: Good description, identification of key factors needed to make decision, discussion with other people.

Unsatisfactory response: Failure to identify key factors and failure to take advice – again dogmatic.

Q5 Impact of illness

Attributes: Communication, empathy and pro-social behaviour, insight about illness and dentistry

Returning to the scenario, Catherine's younger brother has leukaemia. What effect do you think that living with a child with a life threatening illness would have on the family?

Prompt: How do you think it might affect the relationships in the family?

Good response: Description of how the illness would affect other children in the family, and the mother's relationships. [There tend to be different interpretations of this scenario. Some candidates say that relationships may improve as members come together and some say that tension may cause a rift. Either is acceptable if reasoned well in context.]

Unsatisfactory response: Complete failure to recognise that the illness would have a large impact on the emotional balance of the family.

Q6 Insight into self

Attributes: Communication, honesty, reflective, insight about self

Soon after you, the junior dentist, start treating Catherine, her mother rushes in shouting and swearing in a very abusive manner: 'You can't stop me seeing her, she's only 15 – it's my right, I'm her mother'. How do you think this situation would make you feel?

Prompt: Try to concentrate on your feelings not the action you would take.

Good response: Recognition that the dentist would feel confused, upset, hurt, uncertain.

Unsatisfactory response: Failure to recognise or acknowledge the above.

Q7 Uncertainty

Attributes: Communication, honesty, reflective, insight about self

Junior dentists often feel uncertain, confused or upset. Can you tell us about a time when you have felt like that?

Prompt: Can you think of a problem you have encountered and remember how it made you feel?

Good response: Admission/perceptive account.

Unsatisfactory response: Complete failure to recognise that everyone feels out of their depth sometimes.

Q8 Stress response

Attributes: Communication, insight about illness and dentistry, insight into stress management

Dentistry is a stressful profession. People, including dental practitioners, often respond to or deal with stress in negative ways. Can you describe to us what you think these might be?

Prompt: Can you think of anything you might have read or heard about in the media regarding dentists and their levels of stress?

Good response: full range of symptoms including irritability, substance abuse, suicide, divorce.

Unsatisfactory response: inability to accept that dentists show the above.

Q9 Additional reflection

NB NOT to be considered in scoring of interview performance

Can you summarise in about one minute why you want to become a dentist?

Panellists awarded candidates scores by taking into account the response to each question based on the extent to which they judged an individual's response to meet the desirable 'attributes' (Right hand column Table 1). Individual panellists independently scored the responses on a 0-3 scale where 0 = unsatisfactory, 1 = borderline, 2 = satisfactory and 3 = good. In addition, panellists were also asked to give an overall global judgement score to the question, 'Would I like this person as my dentist?', scored as 0 = no, 1 = maybe, 2 = yes and 3 = definitely. This global score was double weighted. The scores for each of the panellists were averaged to give a maximum possible score of 36.

Graduate Australian Medical School admissions test (GAMSAT)

GAMSAT measures an individual's ability to master information. Using reasoning and critical thinking candidates select relevant responses from lists of options. Students who think laterally, creatively and across wide parameters perform well in GAMSAT. To score highly in GAMSAT, a good general knowledge base along with the ability to think conceptually is required. Some scientific knowledge is needed but the central tenet of the test is that it measures mental aptitude for sifting and interpreting information in order to make appropriate judgements. All prospective applicants for the student cohort described in this paper sat the GAMSAT test.

Student progress

Student progress at the end of the first year of the programme was measured in several ways, but for the purposes of this study the two quantitative methods of measuring progress were selected for comparison with the results of the interview. At the end of the first year of study, Peninsula students undertake four assessments. One is a straightforward MCQ test, which measures the students' knowledge of the sciences underpinning dentistry. The other is a progress test.2,3,4 This is set at the level of a graduating dentist and seeks to measure understanding, critical path analysis and ability to apply acquired knowledge. The remaining two assessments used at Peninsula involve qualitative judgements about the students' clinical competence, and analysis of the degree to which they bring a professional attitude to their work. The nature of these assessments does not make them tractable to analysis for a hypothesis-driven study such as one described in this paper. It was hypothesised that if the interview were assessing characteristics which were not learnt via the curriculum, (as per the school's strategy) there would be no association between interview score and academic performance in either knowledge tests (MCQ) or conceptual thinking tests (GAMSAT) or of knowledge application tests (progress test).

Analysis

Simple correlation statistics between student performance at interview in the GAMSAT test and student performance in both end-of-year knowledge and progress tests were calculated.

Results

Table 1 lists the attribute 'themes' which emerged from the literature review. Clinical, academic and technical competencies were the most frequently cited theme in the literature alongside 'sensitivity to others'. Professionalism and ethical behaviour were also highly popular themes in the literature. The mapping of the attributes derived from the literature to the behaviours measurable in the situational interview are shown in Table 1.

Candidates entering the first year class in September 2007 had interview scores in the range 16.75 to 35.75. Figure 1 illustrates the distribution of scores of those successful at interview.

Figure 1
figure 1

Distribution of interview scores for Peninsula Dental School first cohort

Full size image

Figure 2 illustrates the students' performance in the dental progress test at the end of the first year and Figure 3 the distribution of student scores in the knowledge based multiple choice assessments which took place at the end of their first year of study.

Figure 2
figure 2

Distribution of 1st Year progress test scores

Full size image
Figure 3
figure 3

Distribution of 1st Year test scores in life sciences

Full size image

Correlation co-efficients indicating the strength of association between the interview, the GAMSAT performance and first year outcome scores can be seen in Table 2. Table 2 indicates that correlations between interview scores and knowledge, understanding and traditionally measured 'aptitude' were low, and failed to reach statistical significance.

Table 2 Correlation between interview and first year outcome scores
Full size table

Discussion

The strategy employed by Peninsula Dental School aims to ensure that the curriculum develops the skills and competence of professionals while the recruitment system, selects people with the appropriate 'character'. The literature review revealed that clinical academic and technical competency were widely considered to be key attributes of a 'good' dentist. These, we consider to be mutable rather than innate characteristics of an individual and are thus attributes which it is the role of the dental curriculum to develop. However, the literature revealed just as much emphasis on sensitivity to others, and almost as much to professionalism and ethical behaviour.1 These attributes have been identified as core competencies in the General Dental Council's revalidation process.5 Professionalism of course is open to interpretation and its definition is widely debated6 but generally refers to the skill, competence or character of a member of a profession or an occupation which carries high status and requires considerable training.

The results reported strongly suggest that Peninsula has succeeded in utilising an interview process which identifies in applicants, characteristics which are not readily captured in more traditional selection strategies based on achievement of academic criteria which are generally perceived by the academic world to be desirable. This is shown by the low correlations between interview, GAMSAT scores and academic performance, and concurs with evidence indicating divergence between GAMSAT results and interview scores in predicting Year 1 performance in medical students.7 It also fits with data which indicate an inverse relationship between broader, non-cognitive parameters such as experience in leadership positions and a likelihood of failure.8,9

There is undoubtedly controversy about the reliability of interviews in the student selection process, but this is a global view representing a broad range of interview techniques. When considered in greater detail, strategies which have a highly structured format, and which employ trained interviewers, have a greater predictive value when it comes to assessing an individual's 'non-cognitive' strengths.10,11 Because we are a new school, it will be several years before mapping interview scores to broader professional outcomes becomes feasible. Nevertheless, the interview strategy employed here is based on the best available evidence. It utilises a structured format, the interviewers receive formal training and it represents a genuine attempt to introduce into the selection process enquiry into a candidate's perception of broader social and professional issues. Details of Peninsula's educational philosophy have been presented elsewhere,12 and schools sharing a similar approach have found a structured interview to be central to their admissions process. In addition to the broader attributes required of a clinician, experience of other schools with a similar educational philosophy have shown that it represents an opportunity to identify those students who are likely to perform best within the particular educational milieu on offer.9

Finally, it is clear from both the published research, and from the professions regulatory body the GDC, that more is expected of oral healthcare professionals than academic and clinical skills alone.13 If a dental school is to be a true servant to the public and profession it must place equal weighting on broader non-cognitive, interpersonal and communication skills, and less on academic skills, knowledge and performance in examinations.