Introduction

'A fundamental element of this development is a shift in emphasis from the formative and summative assessments of undergraduate training to the process of self-appraisal which is necessary for successful professional development.'

The outcome of SHO training should be an experienced and competent dental professional. Competence is not purely about technical skills in the clinic, but about integrating all the relevant competencies to the job in hand. These include drawing upon and applying knowledge gained from previous experience, keeping up to date with information, and adopting the right attitude to the task and to the patients. These are reflected in the 'three-circle model' described by Harden1 in relation to the clinical competence of doctors, and this has been adapted for dentistry2. The SHO training period is therefore not solely about learning more advanced procedures but is also concerned with gaining all round competence in everything that is done. This has been described as “the right kind of professional doing the right task in the right way”. A fundamental element of this development is a shift in emphasis from the formative and summative assessments of undergraduate training to the process of self-appraisal which is necessary for successful professional development.

A previous study described an investigation of Senior House Officer (SHO) and Consultant educational supervisors' perceptions of the SHO training programme in Dundee Dental Hospital and School, using an action research methodology to identify areas for change3. Among several areas of change identified by both trainers and trainees, were the need to develop the SHO experience into more than an apprenticeship by improving the trainer/trainee relationship and encouraging trainee-centred learning and self-appraisal.

'On Track' was developed as a resource to support these changes and to help the SHO develop as a professional. 'On Track' was a paper-based and electronic educational resource produced by SCPMDE (now NHS Education in Scotland) in collaboration with the Faculties of Dental Surgery of the Royal College of Surgeons of Edinburgh and the Royal College of Physicians and Surgeons of Glasgow. It was piloted at the Edinburgh Dental Institute and Dundee and Glasgow Dental Hospitals. This paper describes the rationale behind its inception, the development of the resource and draws conclusions about the implementation and value of such resources.

The 'On Track' programme and its components

'On Track' was produced in two formats – paper and electronic. The paper format has two components – an A4 file and a smaller pocket-sized file. The larger file contains all the key information required for the SHO post including guidance about gaining in experience and competence, an explanation of the purpose and structure of the appraisal process, an outcomes grid, a competence list, a framework for reflection and tips on how to complete it, information on the facilities and educational programme in each centre and general information on topics such as gaining study leave and examination regulations. The smaller file is designed to fit into a pocket and contains information useful on an everyday basis in the clinic, such as telephone numbers, local protocols and a timetable of local educational events. It also contains pages to jot down short notes on any interesting points as they arise throughout the day. These can be used to jog the memory later when time is taken to reflect on what has been seen or heard and what has been learned from it.

The electronic version is carried on a Psion Revo handheld PC. It contains all the information in the paper version with the added functionality of a database and other features of the Psion, which allow information to be stored and retrieved. One third of the SHOs were given 'On Track' in this format in order to explore the potential of this medium for the delivery of educational resources.

In addition, there was an 'On Track' web site. A feature of this was a bulletin board to allow SHOs to communicate with colleagues in other institutions.

Aims of 'On Track'

'On Track' was designed to guide and support the SHO and educational supervisor by:

  • Defining the outcomes of SHO training.

  • Facilitating the relationship between the SHO and his/her educational supervisor.

  • Encouraging reflection and self-appraisal.

1 Defining the outcomes of SHO training

'On Track was designed to guide and support the SHO and educational supervisor by defining the outcomes of SHO training, facilitating the relationship between the SHO and his/her educational supervisor and encouraging reflection and self-appraisal'

It is important that both the SHO and educational supervisor are clear about the learning outcomes of SHO training from the start and that they know exactly what is expected of them. This should address not only what the SHOs are expected to do by the end of their posts, for example the treatment procedures they will be able to undertake, but also what they will bring to these tasks and how they will develop as professionals. These elements of a competent professional are based on Harden's three-circle model (Figure 1) and summarised in the outcomes grid (Figure 2).

Figure 1
figure 1

Harden's three-circle model.

Figure 2
figure 2

The SHO dentist outcomes grid.

The learning outcomes grid, which has been described previously2, and a more detailed list of competences, was drawn up in consultation with senior clinicians and covers a broad range of topics, not simply clinical skills. They encompass the diverse topics the SHO must integrate in order to provide optimum care for their patients and to develop as a competent dental professional and are divided into three broad categories:

  • Doing the task, that is caring for your patients. This involves clinical information gathering, diagnosis and treatment planning, and treatment procedures.

  • What the SHO brings to the task, that is, application of basic clinical sciences, clinical reasoning and judgement, communication, health promotion, attitudes, ethical stance and legal responsibilities, and information handling.

  • Who the SHO is as a professional, that is, their role in the health service and their personal development.

2 Facilitating the relationship between the SHO and his/her educational supervisor

The educational supervisor has a vital role to play in helping the SHO's professional development throughout his/her time in post, guiding, monitoring and supervising as required and helping the SHO to develop self-appraisal skills necessary for continuing professional development throughout his/her career. 'On Track' facilitates the establishment of such a constructive relationship by providing a plan for regular meetings between the SHO and his/her named educational supervisor, and structured guidance on the objectives of these meetings. This then provides a framework on which to build a meaningful dialogue, resulting in effective, directed support for the SHO. The first meeting happens at the beginning of the post to formulate a learning plan for the SHO by reviewing his/her past experience and identifying relevant areas for future learning. Again 'On Track' assists this process through the detailed list of competences referred to above. This provides a simple checklist for identifying the SHO's past clinical experience and what should be achieved, deemed either essential or desirable, throughout his/her time in the new post.

Regular follow-up meetings are then arranged to monitor progress, address specific training needs and, through appraisal, help the SHO to develop into a reflective professional. Appraisal, which can be defined as “a confidential, educational review process for the benefit of the trainee”4, benefits the SHO by explicitly acknowledging his/her training needs and identifying means of addressing them. 'On Track' provides an outline for appraisal allowing for a variation in the structure and timing according to the location. However, as a recommendation, four appraisal opportunities are suggested; the first to draw up a learning contract, the last, at the end of the post, to review the learning outcomes and two in between, to review progress against the learning plan so that adjustments can be made if required. Emphasis is placed on setting realistic, achievable objectives at each of these meetings, and on the need for adequate resources to be available to support their realisation.

3 Encouraging reflection and self-appraisal

While the supportive role of the educational supervisor is one cornerstone of a worthwhile SHO experience, the SHO also has to recognise that s/he has a large personal responsibility for his/her professional development. Taking time to consider actions and outcomes in the light of available evidence and using this information towards the best care for patients are characteristics of a professional and the further development of such qualities is an important aspect of SHO training. In order to achieve this in the context of day-to-day work, it is important that the SHO takes time to reflect on experiences, good and bad, and more importantly to learn from them. Simply undertaking an operative task, attending a lecture and passively listening to the speaker or observing a senior colleague work is not enough to gain the maximum educational benefit from such learning opportunities. The key to maximising learning is to reflect on what has been seen, heard, or done. For example, a seemingly routine case may in fact have taught a lot about communication with anxious or difficult patients, or a postgraduate lecture may change opinion on a particular treatment procedure. Learning will become much more meaningful and effective when the SHO takes time to record what has happened and what s/he feels that s/he has learned from it. The encouragement and promotion of such reflective learning is one of 'On Track's' fundamental messages. It also provides a starting point for this process by including a framework around which to structure such reflections, along with some completed examples to give further guidance.

Development of 'On Track'

'On Track' was developed by a steering group comprised of individuals heavily involved in postgraduate medical and dental education. The members were carefully chosen to bring both experience in the everyday practicalities of dental SHO training and expert knowledge in educational design and delivery to the project. The programme was also developmentally tested with both SHOs and dental consultants with training responsibilities, details of which are outlined below. The feedback and comments received from this testing were incorporated into the final version of 'On Track'.

SHOs

The SHOs devoted a Scottish national SHO study day to reviewing and commenting upon a draft of the programme. In particular they were asked to consider the following questions:

  • Does the outcomes grid accurately reflect the role of the SHO?

  • How easy to use is the framework for reflection?

The SHOs were supportive of 'On Track' and its aims. Several stated that while they had been told they had an educational supervisor they either did not know who this was or had never met with them. They felt this had been to their disadvantage therefore fully supported the emphasis on creating a supportive relationship with an educational supervisor. They recognised however that it would only work if consultant staff were equally aware of this need and committed to addressing it. They all felt there should be a formal mechanism for regular feedback on performance and that this should begin right at the start of the post in order that any problems be identified and addressed early.

The majority felt that the outcomes did accurately reflect the role of the SHO, that they were very comprehensive and included items they would not necessarily have thought of themselves. Some concerns were expressed, however, about the presentation stating that it perhaps contained too much information and the layout was not 'user friendly'. These comments were taken into account in the presentation of the final version. There was also some misunderstanding regarding the breadth of clinical experience expected, with some thinking that experience of all the clinical competences listed would be required. This is clearly not possible therefore it was emphasised in the final text that those expected of an SHO in a particular post would be identified at the first meeting with the educational supervisor.

With regard to 'On Track's' emphasis on reflective learning, the SHOs generally felt that reflecting on their experiences using the framework provided would be a beneficial exercise. Most thought the framework presented would be straightforward to complete but that it would be helpful to have completed examples. Some such examples were therefore worked up for inclusion in the final programme (Figure 3). Some SHOs however felt that simple discussion with their educational supervisors would be more beneficial. The completed frameworks are intended to act as the basis for such discussions though, and greater emphasis on this role was included in the final text.

Figure 3
figure 3

Framework for reflection – example from Paediatric clinic.

Dental consultants

Dental consultants with responsibilities for SHO training in Scotland devoted a large part of an annual educational meeting to discussion of an early draft of the 'On Track' programme. They were invited to comment on all aspects of content and design of the programme and particularly to give feedback on the list of competences under development. Each consultant was asked to consider the procedures they would expect an SHO to complete during his/her time in that consultant's department, and to rate whether competence in each procedure by the end of post was essential or desirable. This exercise resulted in a comprehensive and realistic checklist of the clinical experience SHO posts in each dental speciality should afford.

The consultants were also supportive of 'On Track's' aims and particularly appreciated the guidance and structure given for appraisal. Some, however, felt the suggested timetable of appraisal meetings would not fit with the needs and demands of their particular SHO posts. In the final version of the programme therefore it was stressed that this was to give guidance only and should be adapted to suit individual circumstances as required, as long as the initial and final meetings took place as proposed.

Developmental testing with both of the programme's target groups, therefore, resulted in useful feedback on the different elements of the programme, which was incorporated into the final version thereby improving its usefulness and value to those it aimed to benefit.

Implementation of the 'On Track' programme

The implementation of 'On Track' had two main strands. It was introduced firstly to educational supervisors at a national meeting when all its components were carefully explained and the educational supervisors' responsibilities outlined. This was further supported by practical training in appraisal through the use of role play, delivered by an external training consultancy. It was then introduced to the SHOs in Scotland approximately one month later, as part of their induction into their new posts, through a series of 'roadshows' involving three members of the development team in Dundee, Edinburgh and Glasgow. Both SHOs and educational supervisors attended these roadshows where, again, 'On Track', its purpose and aims, and all its constituent components were described. The SHOs and their supervisors then met and, in a light-hearted exercise, drew up a 'learning plan' for the educational supervisor based on a hobby or interest. This served the dual purpose of breaking the ice and giving both more insight into the benefits and process of drawing up such a plan. The session ended with a question and answer forum to clarify any outstanding queries anyone might have. Limited resources meant that planned further intervention and follow-up meetings to support the adoption of 'On Track' in the various centres were not possible.

Evaluation of the 'On Track' programme

A free form qualitative evaluation of 'On Track' was undertaken approximately six months after its launch through a series of one-to-one interviews with all SHOs in two of the centres and all the educational supervisors from one centre. In addition there was a group meeting with educational supervisors in a second centre.

Response to 'On Track' was largely positive. The major benefit perceived by both SHOs and educational supervisors was that the structure provided for the appraisal process had ensured that an initial early meeting to draw up a learning plan had taken place. In most cases the educational supervisor had arranged appointments for these meetings but, in some cases, the SHO had had to request it and they stated that they would not have done this had they not had the 'On Track' programme. Both trainers and trainees felt that these meetings had been beneficial and the SHOs particularly felt better prepared for their post as a result. The occurrence of follow-up appraisal meetings was patchier, with some having had two or three by the time of evaluation and others having had no further meetings. The main reason given for this was lack of time on both the part of the educational supervisor and sometimes the SHO. Educational supervisors spoke of the many conflicting demands on their time and could not see a solution to this problem other than the input of additional resources to give them protected time for their SHO training responsibilities.

Another major theme arising from the evaluation was a mixed response to the concept of reflective learning5. A minority of SHOs had completed a number of the reflective frameworks provided and had found this a helpful exercise, especially at the beginning of their post. The majority, however, had not regularly completed the frameworks, several stating that they thought things through in their heads all the time and did not see the benefit of committing these thoughts to paper. Others said they would rather jot things down in a notebook than fill in the structured framework, which they felt to be a little artificial. Self-appraisal and reflective learning was to many a new concept and it is clear that further thought needs to be given to the best means of promoting and supporting it. A resource that focuses on the development of reflective learning and self-appraisal also depends on the support of educational supervisors and it was clear from previous comments that many did not have the time to devote to this.

Evaluation of the electronic format of 'On Track' did not provide clear evidence as to the benefits of this means of delivery, especially with consideration to the cost of such provision. Several of the SHOs who chose this format were unfamiliar with such devices and found it difficult to use. The constraints of the Psion also meant that information had to be spread over many pages, sometimes making it difficult to access easily. Additionally, many reported that it was easier to write notes on paper while on the clinic than to type it into the Psion, and that these notes were then not transferred. However, many of those who had the Psion were enthusiastic about it despite reporting these problems, because of its other features.

A different pilot programme based on the evaluation of performance6 was launched during the same year as 'On Track'. This caused some confusion among those educational supervisors and SHOs exposed to both, which consequently had a negative impact on both programmes.

Conclusions

While the implementation of the 'On Track' programme met with mixed success, the experience allows several important conclusions to be drawn.

Firstly, this study endorses the importance of specifying learning outcomes as a key element of an educational programme and of communicating these effectively to both trainers and trainees. 'On Track' provided one way of doing this but other means by which this can be achieved may also be valid.

'On Track' represents a move towards a more formal approach to the organisation of postgraduate education, which combines the advantages of a traditional apprentice-style training together with a more structured educational programme to enhance the value of SHO training. 'On Track' facilitates the relationship between the two by providing a framework for meetings, formal appraisal and reflective learning, all of which use everyday clinical practice and what can be learned from it as their basis.

Evaluation of 'On Track' has provided insufficient evidence to comment on the advantages and disadvantages of the different formats for delivery of educational resources. As technology continues to advance, however, it is entirely possible that electronic formats will become increasingly accepted.

The experience of implementing 'On Track' emphasises the importance of overall co-ordination and planning of postgraduate training and the resources developed to support it. The introduction of two schemes at the same time acted to the disadvantage of both as, while they had different aims, they were sufficiently similar to cause confusion on the part of both trainers and trainees.

Finally, therefore, the introduction of an educational innovation requires a great deal of planning not only in its initial conception and development, but also in its implementation. It is only with the ongoing commitment and support both from those expected to use it on a day to day basis and from those who produced it that it will be adopted into everyday practice and fully realise the benefits it was designed to make possible.