Introduction

Research has consistently shown situational judgement tests (SJTs) to be a reliable and valid selection method in healthcare professions, to identify a range of professional, non-academic attributes (such as professional integrity, teamworking and resilience).1,2 Given this evidence, SJTs have received a great deal of attention both in the academic literature and through stakeholder commentary. While there is a long history of research evidence on SJTs in other settings,3,4,5 it is acknowledged that, as a relatively new method within dental selection, stakeholder reactions may not be wholly positive. As such, we highlight four key considerations regarding the use of the dental foundation training (DFT) SJT, to inform further debate.

1. SJTs measure non-academic attributes important for success in dental training

It has been widely recognised in research and practice internationally that there is a critical need for robust measures of non-academic attributes at the point of selection into training for healthcare roles. For example, recent UK government enquiries6,7 highlight concerns regarding the decline in compassionate care within healthcare, emphasising the critical role of the workforce in ensuring the provision of high quality and safe healthcare services. Undoubtedly, an important first step is ensuring that the right individuals appointed to any training place have the 'appropriate attributes to work in healthcare',8 including ethical and moral judgement. Thus, the value of selection methods which assess these types of attributes cannot be understated, especially at a point in the education pathway where emphasis is on aptitude for entering clinical practice, such as DFT.

SJTs are designed to measure important inter- and intra-personal attributes for a given role, and do not require clinical knowledge to score highly.3 Historically, researchers in this field used the term 'non-cognitive' to describe these professional, non-clinical attributes.9 More recently, however, the research literature has termed these 'non-academic' attributes, and this is what the DFT SJT is measuring. In relation to this, Affleck et al.10 state that 'to regard [the SJT] simply as a test of putatively non-cognitive professional attributes is perplexing', referring to Patterson and colleague's article in this Journal11 which provided proof of concept for the DFT SJT. Although Affleck and colleagues appropriately raise the point that the term 'non-cognitive' is problematic as it may imply that no thought is required to perform well on the SJT, this is arguably a matter of semantics. Certainly they put forward an important consideration, which is also supported in the wider research literature: SJTs assess individuals' judgement regarding situations which occur in the workplace, which obviously requires some element of cognition (that is, situational judgement), but does not require clinical knowledge. That these judgements require some awareness of what it means to be an ethical dental practitioner, as Affleck et al. also suggest, is implicit in the nature of SJT design and content.

2. The content and context of SJTs must be updated regularly

Some stakeholders have voiced concerns that the consensus about what is the most appropriate thing to do in challenging work-based situations may change over time; meaning that response keys for some scenarios may become outdated during the lifecycle of an SJT.10 Certainly this is a valid point, and any method used in a high-stakes setting, such as selection into DFT, should be regularly reviewed and its content updated accordingly. Practically, an important design consideration for SJTs is to ensure that the context, content and difficulty of the test is commensurate with candidates' level of training.3 For these reasons, and in line with best practice,12 the content and relevance of existing DFT SJT items are regularly reviewed by both SJT item-writing experts, as well as by expert panel members, including training programme directors, educational supervisors and postgraduate dental deans. These experts provide input from the perspective of an employer of a foundation dentist who is being employed to work and train in a general dental practice. Any items that are no longer appropriate are retired or significantly updated. Similarly, new content is created and piloted each year within the SJT, meaning that the content of the test is refreshed to reflect the latest thinking in practice on a yearly basis. Again, this process involves creating new SJT items in close consultation with expert panel members, which undergo extensive and rigorous design and review processes before being piloted. Only items which perform well psychometrically during piloting are entered into the pool of operational test items. (See Patterson et al.3 for a detailed outline of the development stages for SJT content.)

3. SJTs are efficient and cost-effective

A number of authors in recent publications have called for face-to-face interviews in place of SJTs in selection systems for postgraduate healthcare training positions.10,13 We agree that interviews (either structured or multiple-mini interviews) are an important part of the selection process for both the candidate and the employer to assess person-organisation fit,14 and to allow candidates to provide rationales for their given responses. However, while it is important to retain face-to-face methodologies in a selection process, using interviews in lieu of SJTs would represent a plethora of challenges in terms of both practicalities and validity. Taking DFT as an example, it would be impractical to attempt to assess all of the attributes that the SJT measures (teamworking, resilience and coping with pressure, professional integrity, and empathy and communication) in a single interview station at the selection centre, and attempts to do so are likely to have low reliability and validity.15 It is worthy of note that the DFT selection centre includes two interview stations (in addition to the SJT) which provide the opportunity for probing questions and for candidates to provide rationales for their reasoning. Clinical skills are a pre-requisite for successful completion of undergraduate dental degrees, hence these are not assessed at the point of selection into DFT.

Moreover, the benefits of SJTs as a complementary method to face-to-face interviews are well documented.3 SJTs are a standardised way of assessing large numbers of applicants, such as for DFT which has c. 1,400 applicants annually, as they are machine-marked, and are a significantly more practicable, and less resource-intensive, solution than adding another face-to-face interview to the selection centre. The latter would be very costly to a health service already severely constrained by available resources. As such, recent suggestions that the use of SJTs in dental selection may represent an 'overreliance' on the methodology10 are somewhat misguided.

4. Practice tests should be available for SJTs

A common concern for many applicants is whether access to coaching will increase their scores on an SJT. However, the emerging literature does not support coaching as an effective method for increasing performance on SJTs16,17 Practically however, for recruitment and selection purposes, it is important that as much information as possible, including appropriate practice material, is available to all candidates. This has links to procedural and distributive justice theory, which relate to candidates' perceived fairness of the selection process itself, and fairness of the outcome respectively.18,19 Positive candidate reactions to selection methods are important for a number of reasons, including the fact that candidates who perceive processes to be unfair may legally challenge an organisation,20 and candidates who have negative perceptions and are unsuccessful may criticise the process and potentially reduce further applications.21

Affleck and colleagues10 have called for a practice test for the DFT SJT, including a rationale for the appropriateness of different response options. We concur with this suggestion, because – as stated by Affleck et al. – although practising the SJT content itself should not improve candidates' scores, it is beneficial for candidates to be able to familiarise themselves with the format of the test to enable them to focus on the content of their answers when they sit the operational SJT. This is an avenue of development that is currently being explored in the DFT context.

Conclusions

In summary, SJTs represent a relatively new selection methodology in the postgraduate dental setting, and it is therefore both understandable and expected that stakeholder reactions will be mixed. Importantly, there is a wealth of evidence demonstrating the reliability, validity and robustness of SJTs in similar high stakes, high volume settings. We have aimed to demonstrate that the concerns raised by researchers and stakeholders in the context of the DFT SJT are already addressed (or shortly due to be addressed) in the design, delivery and ongoing improvement of the SJT. One possible area for development is the creation of a practice test for the DFT SJT.