Introduction

In the undergraduate periodontal clinic at Glasgow Dental Hospital and School, students are taught to use a comprehensive periodontal chart (Fig. 1) on which can be recorded the position of the gingival margin in relation to the amelo-cemental junction, and probing pocket depth. The recorded measurements can then be used to calculate loss of attachment. Furcation involvement, tooth mobility and bleeding on probing can also be recorded. Experience has shown that a proportion of students find this initially difficult, either conceptually or in practice. Furthermore, infrequent use of the chart often results in degradation of knowledge and understanding over time. For this reason pocket charting was the subject of one of a number of computer assisted learning (CAL) programmes developed within the department.

Figure 1
figure 1

Comprehensive periodontal pocket chart

Full size image

CAL programmes have a number of potential advantages; they allow students to learn at their own pace and, within limits, at a time of their choosing. Additionally they might reduce the demand on teachers to teach certain subjects, thus freeing them for other activities. However, prior to the introduction of CAL programmes into the curriculum it would seem sensible to evaluate them; firstly, to determine whether they are capable of supporting learning to the end that intended learning outcomes are attained, and secondly to determine the most appropriate context for their use. Recent upgrading of the programme in question presented the opportunity for just such an evaluation.

Evaluations of the usefulness of CAL have frequently relied upon user feedback questionnaires that have assessed subjective aspects of the programme such as ease of use.1,2,3 There have been a number of attempts at more objective assessment of computer assisted learning. Lechner et al.4 used a protracted CAL programme in place of a series of lectures to teach removable partial denture design. Having completed the programme, students produced designs of a high standard under examination conditions. Relatively few studies have compared the effectiveness of CAL programmes with that of traditional methods of teaching clinical dentistry. These studies have tended to focus on programmes that aim to teach diagnostic and clinical decision-making skills, often using clinical case simulations.5,6,7 Written tests and performance in role-play situations have been amongst the methods of assessment employed and these have shown CAL programmes to be at least as effective as traditional methods.

The aim of the present study was to assess the effectiveness of a computer-assisted learning programme to teach the theoretical and practical aspects of periodontal pocket charting compared with traditional small group teaching methods.

Methods.

The written content of the CAL programme was adapted from existing course material, with a small-scale evaluation in August 2000 resulting in a decision to revise the programme. After considering various options it was agreed that the best way forward was to develop the programme for delivery over the internet. This had the advantage not only of being platform independent but also in allowing for easier updating in the future. The programme used computer animation as well as static graphics to help reinforce aspects of the periodontal pocket-chart (fig. 2). Screen size was optimized to a resolution of 800 by 600 pixels; this allowed full screen display without the need for scrolling for most users.

Figure 2
figure 2

Screen shot of CAL package

Full size image

The investigation took place in the context of a clinical techniques course at the end of the third year, where this subject would normally be taught. Sixty-one students were randomly assigned to either receive a traditional small group tutorial or to work through the web-based CAL programme. The students had received no prior teaching on pocket charting. Thirty-one students were randomized to the tutorial group while 30 used the CAL programme. The groups participating in either intervention at any one time typically consisted of three or four students. One of the authors (RAM) was present in the computer laboratory in order to answer questions concerning the use of the programme. Students undertaking the CAL programme were first asked to complete a computer experience questionnaire to determine how confident and skilled they perceived themselves to be in computer use as well as requesting details of their previous experience with computers. Students were also required to complete a questionnaire after finishing the CAL programme, relating to ease of use.

Tutorials were delivered by seven different tutors, each of who was an experienced teacher in the periodontal department. Tutors were given no guidance other than departmental teaching policy as set out in a handbook, which they all received.

All students were assessed using confidence logs, a practical exercise and a written test, with the sequence of events as follows:

  1. 1

    Tutorial or CAL programme

  2. 2

    Confidence log immediately following the above

  3. 3

    Practical test immediately following the above

  4. 4

    Confidence log immediately following the above

  5. 5

    Confidence log three weeks later

  6. 6

    Written test immediately following the above

The confidence log was adapted from that described by Draper et al.8 The log asked the students to rate their confidence as, 'very confident', 'confident', 'some confidence', 'little confidence' or 'no confidence', for the following intended learning outcomes:

  1. 1

    I can examine, assess and record data for gingival margin position.

  2. 2

    I can examine, assess and record data for pocket depth.

  3. 3

    I can use the above data to calculate loss of attachment (LOA).

  4. 4

    I understand how to grade furcation involvement.

  5. 5

    I understand how to grade tooth mobility.

  6. 6

    I understand the function of the periodontal pocket chart.

  7. 7

    I know how to complete the periodontal pocket chart.

A practical test was devised using a custom-made model of a dental quadrant. The models were constructed using anatomically accurate reproduction teeth, clear epoxy resin and pink acrylic gingiva. Each model had examples of recession, gingival overgrowth and a variety of probing depths (Fig. 3). Students were required to complete a pocket chart for the model. It is important to note that the model was not designed to simulate the clinical situation precisely but to determine whether students had understood the concept of charting gingival margin position and probing depth, and the use of these recordings to correctly calculate loss of attachment. Eight such models were constructed, numbered, and randomly assigned to individual students. Although all the models were similar there were minor variations. The models were each evaluated by one of the authors (VB) prior to the test and ten features of each model were selected as being highly relevant to the knowledge being tested. Criteria were thus developed for each model; the student chartings were then evaluated against these criteria and a score out of ten awarded. Prior to the test, all students were given brief verbal instructions on reading the periodontal probe which was used (PCP 12, Hu-Friedy®).

Figure 3
figure 3

Custom-made model of dental quadrant

Full size image

Three weeks after the teaching session the students were asked to complete a short, written, multiple-choice test covering various aspects of the periodontal pocket chart. In the three weeks intervening between the first and second sets of assessments the students had continued their clinical techniques course but received no additional formal teaching on pocket charting

Results

Of the 63 students who began the study, four students failed to complete all evaluations. The results presented are therefore based on 59 students who completed all evaluations. Twenty-eight of these were in the CAL group and 31 in the tutorial group.

All students selected to work through the CAL programme had completed at least one course in computing skills prior to this study. Only 7% considered themselves to be computer novices.

Students indicated that they spent most of the CAL session concentrating on learning pocket charting skills; the use of the technology did not deflect from this aim. When asked if they would use the package again 96% stated that they would. Furthermore, all stated that they would recommend the programme to other students.

The time allocated for the tutorial was 30 minutes. The same time was allocated for the CAL programme, although most students completed the programme at least twice, in less than 20 minutes.

Figure 4 depicts the results from the practical test. The distribution of scores is skewed and analysis by means of the Mann-Whitney test showed no significant difference between the CAL and tutorial groups. The median score for both groups was seven, with very few students scoring four or less.

Figure 4
figure 4

Scores in the practical exercise

Full size image

An example of the data collected from the confidence log questions is presented in Table 1. In the interests of brevity, data from all of the questions will not be presented. An exact version of the Chi-square test was used to analyse the data and this revealed very few significant differences between groups. For the statement 'I understand how to grade furcation involvement' in the first confidence log, a higher percentage of the tutorial group were 'confident' than in the CAL group and a greater percentage of the CAL group had 'little confidence' compared with the tutorial group (p=0.04). If the responses 'very confident' and 'confident' are grouped together and compared with responses which were either 'some confidence', 'little confidence' or 'no confidence', differences were observed with respect to the statements 'I understand the function of the periodontal pocket chart' and 'I know how to complete the periodontal pocket chart'. In both cases, after completion of the practical test, students from the tutorial group tended to be more confident than those in the CAL group (p=0.026, p=0.036, for each statement respectively). Interestingly, in relation to assessment of the gingival margin position, confidence amongst both groups decreased immediately after the practical exercise and then recovered somewhat three weeks later.

Table 1 Confidence log statement: I can examine, assess and record data for pocket depth.
Full size table

Performance in the written test revealed no difference between the groups. The median score in each group was six, out of a possible seven.

Discussion

Using the measures described, there were few differences in outcome between the CAL programme and traditionally delivered tutorials. One possible criticism of the study is that no attempt was made to standardize the teaching of the seven different tutors. However, this reflects the reality of how teaching is often delivered. In any case, standardization and consistency are features of a CAL programme which might be considered to contribute to the superiority of this method of teaching. The results do not support this contention. A possible Hawthorne effect on the tutors, who were aware that they were being compared with a computer, must also be considered.

Scores in the practical exercise were high and the subsequent confidence log seemed to indicate that the exercise had reinforced most concepts. The exception was in relation to recording gingival margin position, where students found the amelo-cemental junction difficult to locate. In fact, this is a fair reflection of the difficulty of this procedure clinically. The function of the chart, and student confidence in their ability to complete it, was reinforced by the practical exercise to a greater extent in those who had received a tutorial. Over-interpretation of this finding should be avoided but it is possible that, in respect of these issues, some tutors imparted a greater depth of understanding to some of their students than was possible with the CAL programme. It is interesting that, when students were asked for their subjective comments, both written and verbal, regarding the whole process, a number of them expressed concern that they might have been disadvantaged by not having received a tutorial. It would seem that students still value the opportunity for personal interaction with a teacher, which occurs in a traditional setting.

The slight difference in confidence with respect to understanding how to grade furcation involvement alerted the authors to the need to improve the illustrations in this part of the programme. This highlights the importance of this type of evaluation with respect to the introduction of new CAL programmes.

Kay et al.9 have demonstrated the ineffectiveness of a CAL programme in changing dentists' treatment planning behaviour. Clinical treatment decision-making is a profoundly complex skill. Completing a periodontal chart is, perhaps, less so and this study suggests that CAL programmes and traditional methods may be equally effective in introducing this subject to undergraduate students. However, the perception amongst some students of the importance of contact with a teacher, should be borne in mind at this stage of the learning process in particular.

Holt and Oliver10 have recently questioned the value of controlled experiments in educational studies. These authors argue that there are so many potentially confounding factors that can impact on the educational experience, that generalization derived from such studies is, at best, unwise. They contend that qualitative data are also required, particularly if those specific characteristics of individual learners that may have impacted on the outcome are to be identified. There is clearly merit in such views. However, the methods used to evaluate a given educational intervention will depend upon the nature of the intended learning outcomes and on the target of the intervention. Educators of undergraduates will need to know what methods are most appropriate for large numbers of students and at what stage in the curriculum various methods are best employed.

The current study has attempted to evaluate a CAL programme with intended learning outcomes that have been summarised by the statements in the confidence log. These outcomes relate to the knowledge and understanding that underpins a particular clinical competence or skill. For a skill to become embedded it requires practice and feedback. Minton11 has said that skills are learned experientially and require a 'model in the head'. Clearly, CAL programmes cannot furnish their users with every aspect of practice that the development of a skill requires, but they could possibly assist in providing a model in the head. Unfortunately, if procedures are not performed very often the model may not be retained. The authors recognise the need for caution in the interpretation of the results presented in this paper. Whilst not following directly from these results it would seem reasonable to suggest that CAL programmes may be ideal for the purposes of revision of concepts, in other words renewal of the model in the head, as required. This, together with practice or use of the skill, should result in it becoming firmly embedded.