Key Points
-
Revalidation is intended to assure the public that dental professionals continue to remain up to date and fit to practise.
-
It should be possible to delegate a significant proportion of the administration of revalidation compliance activity.
-
Current clinical governance systems should be sufficient to permit revalidation to operate.
Abstract
Aim To devise and operate a pilot scheme of revalidation for general dental practitioners.
Method A representative group of dental practitioners was convened to advise on an approach to piloting revalidation. Ten general dental practitioners volunteered and completed portfolios of evidence of fitness to practise. The portfolios were assessed by a panel of three calibrated experts, using a specially developed assessment tool. A single decision 'evidence presented allowed revalidation to be recommended' was made. A timesheet was used to record the time spent producing the portfolio.
Results Eight portfolios were assessed as sufficient for revalidation purposes. Two dentists were required to make supplementary submissions of evidence before they were found to be acceptable. An average of eight hours of dentist time and six hours of delegated time was spent producing the portfolios.
Conclusions The small number of dentists in this pilot were able to use the portfolio satisfactorily. The dentists were all volunteers and so may not necessarily be fully representative of the profession. The time spent completing the portfolio was not considered excessive. The assessors were adequately prepared and calibrated for their work.
Similar content being viewed by others
Introduction
In recent years there has been growing public concern that dental care might not be meeting required standards. Additionally there has been a greater willingness on the part of patients to complain and to litigate, with a doubling of complaints and claims to one dental protection society during the 1990s.1 As a result regulatory authorities have moved to improve protection of the public from poorly performing and out of touch healthcare professionals. Poor performance procedures,2 recertification3 and the commitment to revalidation,4,6 have all been part of the dental profession's response.
The term 'revalidation' has sometimes been held to mean the same as recertification.5 However, developments in the regulation of the practice of dentistry have introduced clear differences in the respective processes, which are now distinguished as separate strands of a quality assurance process for the practice of dentistry in the United Kingdom. Recertification for dentists in the UK is defined by the General Dental Council (GDC) as completing the required number of hours of Continuing Professional Development (CPD) and being able to produce certificates to prove this upon request.3 Currently, to maintain a dentist's name on the register requires participation in 250 hours of continuing professional development over five years. This means an average of 50 hours per year. Of the 50 hours, 15 (75 hours over five years), needs to be verified by an approved educational body.3,7,8
Revalidation is more demanding than recertification,8 requiring the individual to demonstrate that the activities undertaken in the revalidation period have had the effect of maintaining fitness to practise. Thus revalidation, in addition to participation in continuing education, encompasses participation in a range of activities such as: clinical audit, presentation of evidence of clinical performance, structured reflection upon practice and evidence of consequent change in practice (implicitly for the better). The medical profession has been moving towards revalidation since the year 2000.9
In 2002 the GDC committed itself in principle to the introduction of a scheme of revalidation.6 Accordingly a pilot scheme of revalidation for use by general dental practitioners was devised to explore the acceptability and practicalities of revalidation for the profession. Non-dental involvement in the assessment process was sought to engender wider ownership. Information on the required time implications was also collected.
The objectives of the study were to
-
Explore how revalidation might work in practice
-
Assess the acceptability of a scheme to the profession
-
Ensure that the scheme fits with clinical governance requirements
-
Assess the likely time required to produce the evidence required to facilitate revalidation.
Methods
A steering group for the project, comprising representative dentists from the profession, reached a consensus that a portfolio format was most likely to achieve a cost effective and acceptable result. Portfolios do have the potential to be used in the assessment of clinical performance over a period of time.10,11,12 Figure 1 shows a summary of the sections of the portfolio, together with the outcome measures used to assess that section. The measures for the three blank sections at part 5 were taken to be the research data, publications or lectures themselves.
Practitioners were recruited to participate in the pilot scheme by inviting 20 vocational trainers in South East Scotland to participate. Ten dentists volunteered and all completed the scheme. Each dentist was provided with a ring binder portfolio, sub-divided by headed index cards with pro-forma sheets and plastic document pouches in each section to make recording of information and reflection straightforward. Portfolios were supplied at a half day induction course. This course introduced the concept of revalidation, showed how the scheme would fit with clinical governance requirements13 and explained how participation would be facilitated. A questionnaire was sent out after the induction course to allow post course evaluation (Fig. 2), which asked participants to rate how well the course had met its objectives, using a Likert scale.
The scheme was run over the second six months of 2002. To identify any problems or difficulties, the scheme organiser made two pastoral contacts with all participants in the middle of the second and fourth months, but none were volunteered. Minor issues that arose during the operation of the scheme were dealt with directly — all were successfully resolved.
The eventuality of a participant not achieving a recommendation that revalidation was appropriate was addressed by creation of a remedial process. The scheme organiser proposed a draft Personal Development Plan (PDP) to address the deficiencies in their evidence to support revalidation. The dentist then submitted a codicil to the portfolio detailing actions taken, with documentary evidence where appropriate. Finally the assessors checked that the codicil met the requirements laid out in the PDP and gave a decision of 'still unable to recommend revalidation' or 'able to recommend revalidation after considering supplementary evidence submission'.
Assessment of the portfolios was carried out by a panel of three assessors. The panel was selected with a view to ensuring that the members were fully conversant with current trends and requirements in CPD for general dental practitioners, were aware of clinical governance requirements in general dental practice and that there was 'clinical governance informed' non-dental input. The assessors were given the following remit:
'To assess whether or not revalidation could be recommended and this decision was to be based on an assessment of whether the dentist had demonstrated (via documentary evidence) that they remain up to date in their practice of dentistry. Also, that they remain “fit to practise” according to the General Medical Council (GMC) definition of the term. The GMC definition is: “Fitness to practise — The level of performance, conduct, and health necessary in order to maintain registration”.'9
A valid assessment is one that measures what it claims to measure and what is important to measure.14 Reliable assessments are ones where the same marker reaches the same conclusions when presented with similar evidence.14 An assessment tool was devised in the form of a grid. National clinical audit criteria were used as a base for setting standards to assess the quality of audit activity.15 Where appropriate, supporting competencies derived for the assessment of Vocational and General Professional Training16 were included. This tool was piloted by the one of the authors (YM), prior to sending it out to the assessors for calibration.
Before carrying out the assessments, the assessors were asked to participate in a validation and calibration exercise. They were sent a portfolio completed with real data and deliberate flaws. This had three purposes: to familiarise them with the use of the assessment tool, its layout, and also to validate the assessment. It also facilitated a period of discussion and calibration before the 10 pilot portfolios were assessed. In this way a consistent approach was assured. The assessors were supplied with a questionnaire to determine their views on the revalidation pilot scheme process and the assessment process. They were asked to return these a week after the assessment day, leaving them some time for reflection.
All 10 participants were provided with grid sheets and instructions for recording how much time was spent compiling the various sections of the portfolio. They were asked to use these to record how much of their time and how much delegated time was spent compiling and reflecting upon the evidence.
Results
Eight of the 10 participants returned the post-induction course evaluation questionnaires. The results are summarised in Table 1
Initially, eight of the 10 portfolios were found to have included sufficient evidence to allow the assessors to form the view that recommending revalidation would be justified. Two were found to have insufficient documentary evidence to support such a recommendation. Consequently the remedial process was undertaken for these practitioners. The deficiencies noted were minor and simple remedial plans were soon fulfilled, allowing a view recommending revalidation to be formed by the assessors. Thus all 10 participants ultimately provided sufficient evidence to satisfy the assessors.
The time spent compiling the portfolio by dentists in the pilot is shown in the histogram below (Fig. 3) as averaged data. The measures of average show that the time spent on tasks was evenly spread between dentists and staff. However, the very low median for delegated tasks reflects a very large range, with many dentists not delegating any tasks at all in certain categories (eg Dental Reference Officer — DRO-reports) and sharing tasks equally in other categories (eg Audit).
The panel of assessors completed their calibration task of assessing a trial portfolio in a consistent fashion. Difficulties and minor differences of interpretation were resolved at the pre-assessment meeting.
Discussion
The method used to select the mode of evidencing fitness to practise was a consultative one, involving a steering group. Using the same group, essentially a Delphi method was employed to devise what should be included in the portfolio. Some representatives of the profession suggested what should be included, then some volunteers from the profession tried it out and reported their experiences.
A portfolio is a collection of evidence maintained and presented for a specific purpose.17 Guidance on what to include in the portfolio was widely sought.18,19,20,21 GDC guidance on professional conduct Maintaining standards22 was employed to devise criteria against which to assess the portfolio. The first section was designed to collect identity information on the dentist to whom the portfolio relates and an affirmation by the dentist that the evidence contained in the portfolio relates to them. Subsection two consisted of volunteered information in this pilot, but would need to be strengthened in any substantive scheme — Chief Dental Officer (CDO) references would need to be checked, and the same with disciplinary and tribunal appearances. Numbers of NHS patients can be cross-referenced against Dental Practice Division (DPD) annual reports, but not very easily. This information relates only to NHS practice, so consideration would need to be given to alternative data sources for predominantly and wholly private practitioners. This also applies to the sub-sections in the performance assessment section relating to DPD, DRO reports and practice inspections. The section on assessment of performance was structured to include two different types of input from patients: unsolicited feedback in the form of complaints23 and compliments, but also an assessment by the patients via a randomly distributed questionnaire.24 The use of patients' views in assessing what constitutes good dental practice has been shown to be significantly at variance with a dentist's perspective.25 A section on peer review was included as it is suggested as a principle method of advancing clinical governance in general medical practice26 and has been shown to be effective in changing dentists' practice.27,28,29 A section on audit was included as it is an essential component of any clinical governance system. It can also be based on the assessment of patient outcomes — a direct reassurance to patients that dentists are performing well.30
Structured reflection is a key component of educational processes17 and of the development of professional judgement.31 Reflective practice has also been favourably reported in medical journals.32 A subsection specifically designated for reflection upon one or more events that had led to change in the practice was incorporated to encourage dentists to start recording this type of process. It was called 'critical event analysis' in the portfolio, but might be more readily recognised as 'significant event analysis'33,34 in future. The CPD section was created as a discrete component, but was positioned between the performance assessment section and reflection on practice section to try and indicate a flow of processes and activities through performance assessment to CPD and on through reflection on practice to a focus on the Personal Learning Plan. Thus reflection upon activity may lead to audit (which can count as CPD), which in turn may stimulate research or team training or initiate a further cycle of personal learning. This shows the difficulty in recognising and rewarding CPD in practice: 'Learning in the context of practice is a continuous experience, best acknowledged through processes rather than events.'35 Learning in practice has been recognised as a core process promoting quality.36,37 Approximately one quarter of the profession hold other registrable degrees38 so it seemed that this would be appropriate to record in the portfolio as a marker of CPD. In order to try and follow the current move towards multidisciplinary, team based learning39 a sub-section on training as part of the team was created in the 'reflection on practice' section. The completion of a personal learning plan has an educational role in appraisal and revalidation40 and two formats were reviewed40,41 before adopting as a template the one that was demonstrated to link with appraisal and revalidation in the medical profession.40
Patterns of prescription and treatment provision were assessed in the subsection on Dental Practice Division reports, while standards of treatment provision and probity were assessed simultaneously in the Dental Reference Officer (DRO) subsection. The standards employed by DROs are the judgements of established and experienced practitioners (Limen referencing).
The number of dentists participating in this pilot was small, at 10. However, it is in the nature of pilots to be small — ideally a larger study should follow. This requires waiting until the GDC has made some decisions about the format, nature and requirements for revalidation. All the participants were volunteers and so the results may not be fully representative of the profession, which may include some or many dentists who are less well disposed to the concept of revalidation. Persuading such dentists to participate in a substantive study of revalidation prior to its becoming mandatory would be challenging.
The responses to the post induction course questionnaire show that the participants were successfully introduced to the concept of revalidation. The results also show that the relationship of revalidation to clinical governance was well explained. The participants understood how to use the portfolio. The critical incident/significant event analysis exercise was well received, even though this is a new concept for most general dental practitioners. The process of assessing the completed portfolios was the area least well understood, but overall the induction day succeeded in meeting its educational objectives.
The average amount of time spent by dentists producing their portfolios was nine hours 16 minutes and the average amount of time delegated to carry out tasks was eight hours 17 minutes. Two practitioners had included all the time that they had spent conducting clinical audit and CPD in that time category rather than the time spent reflecting upon them and changes in practice as a result. These timings were reduced to the mean value of the other five. The adjusted values are seven hours 58 minutes for dentists' time and delegated time to six hours nine minutes. Thus the anticipated time commitment for a portfolio system would not seem to be too onerous at approximately eight hours of dentist time supported by six hours of delegated time per year.
Conclusions
Elements of clinical governance can be used to support the mechanism of revalidation. Allowing for the caveats made regarding the small number and the volunteer status of those involved in the pilot, the dentists in this pilot scheme were able to assemble the required documents and carry out suitable reflection. The assessors were adequately prepared and calibrated prior to assessing the portfolios. The time allowed for assessing the portfolios appeared to be about right.
Other important aims of the pilot were to gauge the acceptability of the scheme to the profession both as subjects and assessors. The results of a qualitative survey of attitudes is presented in a subsequent paper.
References
Tidman, S . Risk management. The Dentist February 2001; 12.
GDC. Report and recommendations of the career redevelopment review group on a performance review scheme for the dental profession. London: GDC, 1998.
GDC. Reaccreditation and recertification for the dental profession. London: GDC, 1997.
Wilson N . Revalidation — hostage to fortune? Br Dent J 2002; 193: 1.
Buckley G . Revalidation is the answer. Br Med J 1999; 319: 1145–1146.
Wilson N . New horizons in the regulation of dentistry. Br Dent J 2003; 194: 7–9.
GDC. Lifelong learning. London: GDC, 2000.
Newble J . The certification and recertification of doctors: issues in the assessment of clinical competence. UK: Cambridge University Press, 1994.
GMC. Revalidating doctors — ensuring standards, securing the future. London: GMC, 2000.
Snadden D, Thomas M . Portfolio learning in general practice vocational training — does it work? Medical Education 1998; 32: 401–406.
Snadden D, Thomas M . The use of portfolio learning in medical education. Medical Teacher 1999; 20: 192–199.
Snadden D . Portfolios — attempting to measure the unmeasurable? Medical Education 1999; 33: 478–479.
Wilson N . Lifelong learning. Br Dent J 2000; 188: 469.
Donabedian A . The quality of care — how can it be assessed? JAMA 1988; 26: 1743–1748.
National Centre for Clinical Audit (NCCA). Criteria for clinical audit. London: NCCA, 2002.
Prescott L . Competencies for dental vocational training and general professional training in Scotland. Edinburgh: NES, 2002.
Mathers N, Challis M, Howe A, Field N . Portfolios in continuing medical education – effective and efficient? Medical Education 1999; 33: 521–530.
Freeman R, Lewis R . Planning and implementing assessment. London: Kogan Page Ltd, 1998.
Pendleton D, Schofield T, Marinker M . (Eds) In pursuit of quality — Approaches to performance review in general practice. London: RCOGP, 1986.
Education Development Unit. Revalidation for clinical general practice — Evidence and standards (Document C). Dundee: Tayside Centre for General Practice, 2001.
Jolly B, Grant J. A practical guide to assessment and appraisal for higher specialist training. London: Joint committee for Education in Medicine, 1997.
General Dental Council. Maintaining standards. London: GDC, 2000.
Scottish Office. Complaints — Listening ... Acting ... Improving, guidance for general dental practitioners. Edinburgh: HMSO, 1996.
Prescott L . Scottish dental assessment project; 2nd annual progress report 83. Edinburgh: NHS Education Scotland, 2001.
Burke L, Croucher R . Criteria of good dental practice generated by general dental practitioners and patients. IDJ 1996; 46: 3–9.
Grol R, van Eijk J, Mesker P, Schellevis P . Audit: a project on peer review in general practice. Family Practice 1985; 2: 219–224.
Maidment Y . An assessment of peer review and its effect on changing the knowledge and practice of participants. University of Edinburgh: DDS Thesis, 2002.
Fleming W, Golding L, Macdonald M, Fleetwood-Walker P . Evaluation of the peer review scheme for GDPs in England. Birmingham: Soundings Research, 1996.
Clyde J . Continuing education in general dental practice. Edinburgh: SCPME, 1991.
Norcini J J . Recertification in the USA. Br Med J 1999; 319: 1183–1185.
Tripp D . Critical incidents in teaching — Developing professional judgement. UK: Routledge, 1993.
Mckay A J . Revalidation: the catalyst for change in continuing professional development. J R Coll Surg.Edinb 2000; 45: 71–73.
NHS Clinical Governance support team 3. Analysing significant events — to make things better not attribute blame at Langbaugh Primary Care Group, Cleveland. NHS Modernisation Agency.
Pringle M . Significant event auditing. Scandinavian J of PH Suppl 2000; 18: 200–202.
Grant J, Stanton F . The effectiveness of CPD. London: Joint Centre for Education in Medicine, 1998.
SCOPME. CPD for doctors and dentists — Recommendations. pp 20–23. London: SCOPME, 1998.
DOH. CPD — Quality in the new NHS. London: DOH, 1999.
GDC. Dentists register 2002. London: GDC, 2002.
Pringle M . Participating in clinical governance. Br Med J 2000; 321: 737–740.
Challis M . AMEE Medical Education Guide No.19 — Personal Learning Plans. Med Teacher 2000; 22: 225–235.
Highland Health Board. Personal Development Plan. Inverness, 2003.
Acknowledgements
The authors wish to thank Dr Mike Winter, Martyn Steed and Sandy Stewart for acting as assessors in the project. Acknowledgement is also given to Dr Linda Prescott for agreeing to use of the Patient Assessment Questionnaires. Many thanks to Elaine Humphreys for helpful advice at various stages. The project was funded by Scottish Council for Postgraduate Medical and Dental Education, and its successor — NHS Education Scotland.
Author information
Authors and Affiliations
Corresponding author
Additional information
Refereed paper
Rights and permissions
About this article
Cite this article
Maidment, Y., Rennie, J. & Thomas, M. Revalidation of general dental practitioners in Scotland: The results of a pilot study Part 1 – feasibility of operation. Br Dent J 200, 399–402 (2006). https://doi.org/10.1038/sj.bdj.4813427
Published:
Issue Date:
DOI: https://doi.org/10.1038/sj.bdj.4813427
This article is cited by
-
Feasibility of revalidation operation
British Dental Journal (2006)
-
Revalidation of general dental practitioners in Scotland: The results of a pilot study Part 2 – acceptability to practitioners
British Dental Journal (2006)