Introduction

Peer review has been an activity of recognised importance in continuing professional development (CPD) for over a decade and it has been identifiable as a definable process since 1969 when Schonfeld described it as 'a critical examination and subsequent evaluation, by a group of competent dentists of what has occurred elsewhere'.1 It is listed as a verifiable core activity in the General Dental Council's recertification scheme2 as well as being central to plans to implement clinical governance in dental practices.2 The role of continuing professional education and continuing medical education is widely recognised throughout Europe as important in maintaining standards in the medical profession.2 The dental equivalents have been recognised as central to maintaining standards in the dental profession. To be effective, adult learning needs to be:

  1. A

    interactive

  2. B

    'relevant to their work' eg involving problem solving. A point emphasised by Brown and Uhl: 'Most adults learn most efficiently and effectively in a problem-solving situation'2

  3. C

    built on their own experience.

More generally, passive involvement is ineffective and unlikely to result in behaviour change.2 This study was carried out to assess the general dental practitioner's perception of the effectiveness of peer review in producing change in knowledge and practice, compared with other methods of CPD. Peer review, although continuous, tends to occur as fixed term projects. The balance of passive and interactive learning will vary from one project to another and in this respect it is similar to the other CPD methods with which it is compared. Thus it is important that evidence of peer review's effectiveness, or lack of it, is produced to inform future continuous professional development provision. Peer review has been reported as an acceptable and constructive form of self-appraisal/continuing education in England and Wales.2

Method

Initially, a pilot questionnaire form was drafted and submitted to five general dental practitioner colleagues who made recommendations to improve the clarity of individual questions. Then a postal questionnaire was sent to 268 general dental practitioners, who had participated in peer review over the first 24 months of the scheme's operation in Scotland. The mailing list was compiled from lists of participating dentists held by the chairmen of the regional peer review assessment panels, after ethical considerations had been addressed by NHS Management Executive. This produced the provisos that participants would be anonymous and that repeated follow ups were not to be made. A reminder was sent to non-responders after four weeks, with a follow up eight weeks after the initial mailing.

The questionnaire comprised six sections covering: demographic data, professional experience, access to and participation in postgraduate dental education, perception of knowledge acquisition and changes in dental practice, perceived effects of peer review and attitudes towards peer review. Dentists were asked to rate on a visual analogue scale, by striking through a number from one to six arranged in a horizontal line across the page, how effective postgraduate education was in updating knowledge. They were then asked the same question with respect to how effective each mode was in changing their practise of dentistry. Anonymity was ensured by having a secretary open the envelopes and remove a coded sheet from the questionnaires, allowing her to remove their name from the database used to send a follow up.

Various IBM-compatible personal computers running Microsoft Excel were used to create a database into which responses were entered. The transcription of the responses on to the database were checked and any errors corrected. Data were exported to Minitab for analysis and some manual calculation. Standard statistical methods were employed, including Chi-squared test and McNemar's Chi-squared test. The Chi-squared test was used to compare the distribution of a categorical variable in the sample population (eg sex) with the distribution of another categorical variable (effective in changing knowledge). The test is for a difference in the two distributions. McNemar's Chi-square test was employed where pairs of individuals were matched, to take account of the matching, which standard Chi-square tests do not. The visual analogue scores were analysed against a series of characteristic profile measures (gender, age, whether or not single handed, possession of a higher qualification and distance from a postgraduate centre) using Chi-square tests.

Quantitative data results

Of the 164 questionnaires returned, six were incomplete and unusable, leaving 158 for analysis — giving a response rate of 59%. Only 21% (33) of respondents were female, compared with 29% of dentists registered in the UK in 1997,8 at the end of the survey period. The age profile of the dentists in the sample was skewed towards younger practitioners, with 67% (105) aged under 45 years (Tables 1 and 2). Most of the sample were principals in practice — 72% (114). Postgraduate qualifications were held by 27% (45) of the dentists.

Table 1 The distribution of dentists by sex
Full size table
Table 2 The distribution of dentists by age
Full size table

The visual analogue scale scores given by the dentists were grouped into two categories: 'not effective in producing change in knowledge' (scores of one to three) and 'effective in producing change in knowledge' (scores of four to six), as illustrated in Tables 3 and 4. The results of the scores for each type of continuing dental education are shown in Figure 1. In Figure 2 the equivalent scores, treated in the same way, for perceived effect on changing practice are shown.

Table 3 Effect of continuing dental education mode on change in knowledge
Full size table
Table 4 Effect of continuing dental education mode on change in practice
Full size table
Figure 1
figure 1

Fig.1 The perceived effect in changing knowledge of continuing education modes

Full size image
Figure 2
figure 2

The perceived effect in changing practice of continuing education modes

Full size image

The significance of differences in the numbers of dentists saying that no change occurred in their knowledge between peer review and the other CPD activities was tested with McNemar's test.2 The same was carried out with respect to perceived change in practise. The results in Table 5 show that section 63 courses, private courses and reading journals are all perceived to be significantly better at changing knowledge. The test was also significant for Computer Assisted Learning (CAL) the other way, showing that peer review was perceived as better at changing knowledge. The same test on perceived effectiveness in changing practise showed that all the other modes tested were felt to be better than peer review, except CAL.

Table 5 The P values of McNemar's test, comparing perceived effectiveness of peer review with the other CPD modes surveyed
Full size table

Discussion of quantitative data results

The 158 useable replies received equate to 1% of general dental practitioners in Scotland. These were dentists who had chosen to participate in peer review, so there is a risk of bias due to this being a self-selecting group. Peer review was introduced in Scotland four years later than in England. It was run for a limited period that was extended once and then discontinued. Although peer review activity continues amongst some Scottish general dental practitioners, it has developed along different lines from England and so the results are not necessarily extrapolatable to a UK level. There was no difference between the sexes in the perceived effect of other forms of continuing dental education in changing knowledge, apart from attending study groups, which male dentists found to be more effective in changing knowledge (P < 0.05).

Dentists aged over 44 were significantly less likely to find CAL effective in changing knowledge (P < 0.05). Those dentists possessing a postgraduate qualification were significantly more likely to find conferences (P < 0.01) and symposia (P < 0.05) effective in changing knowledge. They also found private courses very significantly more effective (P < 0.001) and section 63 (MADEL-funded) courses significantly less effective (P < 0.01).

The only difference in response that varied between the genders for perceived effect in changing practice was that women found private courses to be less effective than the men (P < 0.05). Those aged over 44 were much more likely to find CAL less effective in changing practise than younger dentists (P < 0.001). Those possessing postgraduate qualifications were significantly less likely to report section 63 courses effective in changing practise (P < 0.01), but much more likely to find conferences effective (P < 0.01). Dentists practising on their own all the time were much more likely to report videos effective in changing practise (P < 0.01).

Comparing the data in Figures 1 and 2 it is apparent that all modes of continuing professional development scored higher on altering practise than knowledge, except for videos.

Qualitative data results

Free form qualitative comments were invited at the end of the questionnaire and 48 dentists (41%) made comments. Of these 42% (66) related positively to the communication that had occurred, with such statements as:

'Peer review was a worthwhile exercise as it brought together practitioners from several different types of practice and allowed us to use a frank exchange of views'

demonstrating a widely felt perception that peer review was good for professional communication. The single most common disadvantage cited by participants was the lack of, low level of, or difficulty in obtaining funding — all of the 32 (67% of) comments on this aspect agreed that funding was inadequate:

'There must be proper funding that genuinely reflects the time and cost of being out of the practice.'

Particular references were made to the problems of rural practices in sourcing postgraduate education generally:

'Continuing dental education is good provided it is easily available … we have problems getting lecturers to come to us though that would be cheaper'

and peer review in particular:

'However, in rural areas it can be a problem getting in contact with other interested dentists, if priority is given to groups with different people [ie not the same group twice].'

Several dentists identified peer review as addressing the isolation of single handed practitioners:

'Peer review is vital for single-handed practitioners.'

Straying from the subject was felt to be a problem by six of the dentists. Other structural problems were mentioned by another three:

'Peer review done properly and well organised is really excellent and providing the group is not too large, good for interpersonal responses. Perhaps a visit from an outside observer from time to time to see format and assess progress would be a good idea.'

The relationship of peer review to clinical audit was not clear to some dentists, who grouped them together:

'Peer review is, along with Clinical Audit, a very good way of keeping up to date in a non-threatening and very open way'

and others who viewed them as separate entities:

'Peer review generally is very useful — it is based on planet Earth and not on cloud 9! Audit on the other hand tends to be a bit too academic to have practical significance to general practice.'

Others had insight into local activity in both areas:

'Clinical audit performed by groups of dentists and properly funded would retain the advantages of peer review and audit. Peer review is accessible by many. Clinical audit as presently happening in (locality) lacks attraction and is likely to be performed by a minority.'

It was apparent from the comments that some practitioners had found peer review a way into audit activity, but there were not that many (four) and there was some suggestion that these individuals may have got involved anyway.

Discussion

The free form comments highlight that single-handed practitioners are perceived as needing professional peer contact. Other studies have also suggested that single handed practitioners are more isolated and at greater risk of being out of date.2 This perception was further supported by a theme present in most of the comments, that the value of interaction with colleagues in a structured activity is professionally stimulating. Given the comment about difficulties of dentists in rural areas getting in contact, the perception of the effectiveness of videos may be a function of circumstance, rather than a true expression of preference. In Scotland distance from postgraduate centres is a geographical fact and it would be useful to find out if this affects perceptions.

A number of dentists had little idea how clinical audit differed from peer review. However, the start of clinical audit as a term of service requirement in 2002 and the accompanying scrutiny of audit projects by regional assessment panels should help clarify this for these dentists. The unstructured comments generally described peer review in a constructive and positive light. The very low level of outright hostility to peer review and its effectiveness, together with the unlikelihood of being able to run a truly randomised trial to compare educational methods suggest that these results are as close to what the profession feels as it is likely to be possible to obtain.

Conclusions

Peer review in Scotland was perceived to be significantly more effective in increasing knowledge than other modes of postgraduate dental education, except for courses — both section 63 and privately run ones, and reading journals. The dentists felt that peer review was not significantly more effective than other modes of postgraduate dental education in changing practising behaviour. Despite this it was felt to be an acceptable method of CPD in Scotland. The deployment and development of NHS funded peer review activity in Scotland has differed significantly from other parts of the UK, so these conclusions may not necessarily reflect the situation elsewhere. Peer review in Scotland is no longer funded as a discrete activity by the NHS, although private funded and unfunded activity continues.