Introduction

Competition for admission to dental school within the United Kingdom is intense and recruitment of the most appropriate candidates is imperative. As such, a range of 'screening tools' is required for the selection of prospective dental students.1,2 Within the UK, all applications to dental schools are conducted online through the Universities and Colleges Admissions Service (UCAS) and while substantial variations remain between and among both non-graduate- and graduate-entry dental schools in their admissions process, most are based upon attained/anticipated educational achievements, personal statements, professional references which are graded according to non-academic abilities (for example, work experience, inter-personal- and practical skills), and some form of interview process. While some authors have demonstrated a relationship between previous educational attainment and academic performance during a dental degree programme course,3,4,5 admissions processes have been criticised due to lack of reliability, validity, potential bias and the resource required to appraise applicants.6,7,8,9,10 In addition, 'grade inflation', which is the tendency of academic grades for work of comparable quality to increase over time, and an increasing awareness of the 'softer skills' required in dentistry may have impacted on the selection process in the discipline.

There is, however, the demand for a just and transparent selection process to be developed given that there is a need to select students who have the qualities of a dental care professional. In addition, there is the desire to widen access to dental schools from individuals from deprived and minority backgrounds and to be able to discriminate between candidates who attain equivalent academic grades at the highest level.11

Such concerns led to the development of the UK Clinical Aptitude Test (UKCAT) first introduced in 2006 as part of the admissions process by a consortium of 23 UK medical and dental schools. The UKCAT was conceived to improve the fairness and objectivity of the admissions process for medicine and dentistry and employs an online test to assess a range of cognitive abilities identified as important to the practice of medicine and dentistry. There is no curriculum content to the test given that it examines innate skills, although the UKCAT consortium recommend that prospective applicants spend up to 30 hours reviewing the official guide, practice app and practice questions in the Candidate preparation toolkit. The component parts are in multiple choice format and are completed within a specific time-frame which includes the following:

  • Verbal reasoning which assesses the ability to critically-evaluate information that is presented in a written form

  • Quantitative reasoning: which assesses the ability to critically-evaluate information presented in a numerical form

  • Abstract reasoning: which assesses the use of convergent and divergent thinking to infer relationships from information

  • Decision analysis: which assesses the ability to make sound decisions and judgements using complex information

  • Situational judgement test: which measures the capacity to understand real-world situations and to identify critical factors and appropriate behaviour in dealing with them.

The test has been designed to offer universities the ability to select students on the basis of characteristics relevant to success in their chosen profession.12 One prospective cohort study of nearly 9,000 applicants to medical school determined that the UKCAT may lead to a more equitable distribution of offers of study from certain under-represented socio-economic demographic groups.13 Currently, within the UK, the majority of dental schools use the UKCAT to augment the selection process, although each school will assign a different 'weighting' to the results of the test in choosing prospective dental students.14

Within medicine, there is minimal and conflicting evidence concerning the predictive validity of the UKCAT in the early years of the course11,15,16,17 and more recently, two authors have noted some modest supportive evidence for the UKCAT in the latter clinical years of the medical degree.18,19 To date, however, there is limited data looking at the predictive power of the UKCAT in the selection of dental students. One UK-based study determined that such an assessment tool was not predictive of subsequent academic performance for non-graduate dental students.20

As such, it would appear that further research is required to determine if the UKCAT is an appropriate selection tool in relation to other more traditional pre-admissions methods and their respective predictive values for subsequent academic performance.

Materials and methods

Ethical approval was granted from the College of Life Sciences and Medicine Ethical Research Board (CERB), University of Aberdeen to review admission data for all students admitted to the University of Aberdeen Dental School and Hospital from 2010 to 2014. UKCAT scores for all four domains were collated in addition to the percentile score obtained for each student which indicates student ranking relative to other candidates who sat the UKCAT in the same year group. UCAS forms were reviewed for a pre-admission academic score (PAS) (0–25) based on previous academic qualification grades. Grades were awarded according to class of first degree, for example, first, second or third class. Those with masters and PhD's were awarded greater values. All values were assigned by the university admissions officer. A minimum value of 19 was required for further admissions consideration. The UCAS forms of those individuals matching the minimum academic tariff were scored (UCAS, 0–20) independently by two admissions selectors, specifically reviewing career aspirations, experience of patient care, inter-personal- and practical skills; a supporting reference and selectors scores were averaged. UKCAT, PAS and UCAS scores were combined and students ranked with the highest scoring applicants (n = 60) invited for multiple mini-interview (MMI) which was scored from 0–100. The same assessor was allocated to a MMI station (each lasting five minutes in duration) within each year of study. No minimum MMI score was required and all stations were weighted equally.

Offers of a place to study dentistry were initially made to the twenty individuals ranked highest; where applicants declined the opportunity, further offers were made according to ranking.

Details of the basic demographic data (gender, age) were recorded. Summative assessments consisted of multiple short answer papers (MSA), single-best answer papers (SBA) and objective structured clinical/practical examinations (OSCE/OSPE). Scores for all end-of-term and end-of-year examinations were converted to the University Common Assessment Scale (CAS, 0–20). No re-sit assessments were included in the analysis.

Both the MMI and OSCE stations included the following stations which assessed the clinical abilities and also, the 'softer' skills required to be a dentist. The MMI looked at previous work experience; problem solving and analytical skills; communication and inter-personal skills; manual dexterity and enthusiasm; and interest and motivation to study dentistry, while the OSCE stations included, for example, application of rubber dam, facebow use, radiographic analysis, needle/drill desensitisation and management of medical emergencies.

Statistics

Given the limited sample size, data for all years of study were combined. Student age, PAS, UCAS, MMI, UKCAT scores and percentiles, and CAS scores were normalised (Kolmogorov-Smirnov) and data were analysed by Pearson correlation and multiple regression (IBM® SPSS® Statistics 21). Strength of correlations were compared using Cohen's effect size interpretations (small ≥ 0.10, medium ≥ 0.30, large ≥ 0.50 and very large ≥ 0.70).21 All assessment reliabilities were calculated for written assessments (MSA and SBA) and practical assessments (OSCE and OSPE) using Cronbach's alpha as a measure of psychometric robustness (IBM® SPSS® Statistics 21).

Results

Data were available for 71 students (F: 44; M: 27) with a mean age of 27.8 (SEM = 0.14) years. Twenty-seven written and practical assessments were included in the analysis and Cronbach's alpha for each suggested broadly acceptable levels of reliability (α = 0.66 and α = 0.83, respectively). Descriptive statistics for PAS, UCAS, MMI, UKCAT scores and percentile are given in Table 1. Student age, MMI, UKCAT scores and UKCAT percentiles demonstrated a correlation with CAS scores (r2 = 0.119, P = 0.001, r2 = 0.136, P = 0.001, r2 = 0.077, P = 0.019 and r2 = 0.118, P = 0.001, respectively) (Table 1). Student gender, PAS and UCAS were not predictive of CAS scores.

Table 1 Descriptive statistics and correlations between pre-admissions measures and assessment scores

Multiple regression analysis revealed statistically-significant predictors with positive correlations of 0.08, 0.13 and 0.10 observed between student age, MMI and UKCAT percentiles and assessment scores. There were no significant associations between any other admission selection tools and assessment data. The final regression models explained only 4% of the variance of the assessments (Table 2).

Table 2 Multiple regression statistics

Discussion

This study investigated the predictive value of the admissions process and the UKCAT over four years in a graduate-entry dental school and should be seen as a pilot study. Limitations to the study include the single-site nature of the project and also, the comparatively small number of students, certainly compared with research conducted within medical schools, and, as such, the generalisability of the results should perhaps be viewed with caution. Additionally, data for all years were combined given that on average twenty students/year were available for analysis. The authors acknowledge that student progression is nonlinear and that the predictive value for a student in the early years of study could be different from a student about to graduate.

While the correlations were small, the supportive evidence determined for student age, the MMI and UKCAT percentiles can perhaps be considered as beneficial when considering the validity of the selection process. Increasingly, within the United Kingdom, dental schools have adopted the use of MMIs for student selection because of their ability to differentiate between prospective candidates academically and also, due to the predictive value of subsequent academic-and skills performance on a dental course.22,23

In relation to the UKCAT, limited and divergent evidence exists to support the predictive value within medical schools.11,15,16,17,18,19 Concerning dentistry, one previous study conducted in a non-graduate entry school looked at use of the UKCAT and confirmed that scores predicted the likelihood of a candidate being offered a place to study dentistry, although had no predictive value in year one assessments.20 The findings of the current study in relation to the UKCAT confirmed some predictive validity in the assessment process and could be explained in part by the graduate-entry nature of the course; that is, such students already held at least one previous degree and would be familiar with summative assessments. Given that more than half of the dental schools in the United Kingdom use the UKCAT to supplement more traditional student selection methods and the cost implications for prospective students (currently between £65 and £100 for tests held within and outwith the United Kingdom respectively),12 it would seem prudent to determine the potential validity of the test in the selection of the candidates with a 'natural talent' for dentistry.

Neither the PAS nor UCAS scores provided any predictive validity with neither yielding statistically significant positive associations. Concerning PAS scores, which were determined based on previous academic qualification grades, one previous study identified that students with medical- or anatomy-related degrees and those who had been healthcare professionals in other disciplines performed better academically. This could perhaps be explained given the overlap in biosciences knowledge.22 However, a prior degree in any subject may have some effect on the approach to study, for example, in relation to prioritisation of learning and time management.24 Regarding UCAS scores, others have determined previously that such admissions tools are not predictive of medical students' course performance.7,15,25,26

Conclusions

Despite the limitations of this study, the findings would appear to provide some support for use of the MMI and UKCAT percentiles in dental school selection. The Admissions Committee within this school should continue to review data from the admissions process to ascertain those selection methods which are most predictive of subsequent performance. While acknowledging different weighting applied to the variety of admissions processes within dental schools and the variance in both curricula and assessment methodologies, multi-centre research would appear to be beneficial in this area of dental education research.