Main

In an era of mandatory continuing professional development (CPD), the possibility of using new technologies has to be considered and evaluated carefully. Such new technologies offer the potential for enhancing learning and delivering post-graduate education in a more convenient and cost-effective manner.

This project was concerned with the effectiveness of e-learning versus traditional lecture delivery. Specifically, questions arose about:

  • The nature of learning from e-learning and lecture delivery

  • The experience of the learner with learning by each method

  • The significance of the subject matter for learning by each method, and

  • The impact of individual differences.

The project evaluated participant response to the introduction of CD-ROM delivery of continuing professional development (CPD) for groups of trainee and practitioner dentists. The subject content under evaluation related to the 'pillars of clinical governance' as described in the white paper on health.1 Briefly, the material concerned the ten component processes of clinical governance detailed in the white paper, including the use of evidence-based practice, the dissemination of good ideas, clinical risk reduction programmes, learning from complaints. (Full details are given in http://www.smile-on.com/cg/cgscheme.php.) Specifically we wanted to determine whether differences existed between two groups of learners: one using e-learning, the other the traditional face-to-face lecture method.

The use of new technology to promote learning in professional contexts is growing and changing. Salmon2 has identified a number of models for the development of e-learning, the first of which — namely, programmes based on the delivery of content — describes the model applied in this research project.3 We return to Salmon's work later in this paper when making our recommendations.

Rowntree, in evaluating developments in e-learning materials, identified the need for material evaluation, critical commenting and developmental testing combined with continuous monitoring during the introductory phase of electronic teaching media.4 This project was designed to follow Rowntree's advice, using a thorough research process to evaluate, with the aim of enhancing the learning achieved. This research project constituted the monitoring component of Rowntree's model and involved participant evaluation of the two different delivery methods noted above.

The research concerned two specific groups: the first group consisted of dentists in their first years of training, participating in a vocational training induction (VTI) programme; the second group consisted of their trainer mentors who were all established practice managers, well respected in their field. The research was carried out in two locations, the first an inner city area, the second a more affluent urban area.

The research was also designed to enhance work towards research-based practice into teaching and learning by exploring new ways of encouraging high quality learning in postgraduate courses. We planned to achieve this by evaluating the way in which the new technologies used to enhance learning were understood by the learners and in so doing, to determine learner preference in different teaching environments.

Central government policy has given strong support to the use of technology in dentistry and this of course extends to its use in CPD.5 There is a national commitment to using ICT in learning and teaching and elements of online learning provision are likely to become an aspect of all National Health CPD programmes. The National Health Strategy Information for Health makes a commitment to establishing a clinical reference website for access to national reference material (www.NELforHealth) as part of an information superhighway for dentistry. The use of ICT connectivity is further articulated in the announcement of plans to establish a national electronic library for health to encourage the comprehensive use of ICT in training and CPD.6

The developing use of reusable learning objects (RLOs) such as CD-ROM teaching materials in health professional education and training is a phenomenon that is gaining momentum. The use of RLOs is seen to support the social, economic and professional objectives of CPD by providing quick and easy access to knowledge acquisition, understanding and use, in a changing world.7

However, for technology to be effective it is essential to incorporate the best pedagogy into the learning environments. The best methods can only be identified through research and evaluation such as those reported here. Educational research has identified a number of changes to the learning experience that occur when e-learning is adopted. Laurillard applies the term 'adaptivity' to describe how the learner adjusts to a different learning environment,8 and Jameson has researched the impact of technology-enhanced teaching on those not confident in the use of ICT who experience an initial period of disorientation.9 Still more research is required into the pedagogy of technologically designed learning opportunities so that the new tools can be designed with the needs of different learners, and different learning situations in mind.

Method

The research took place in two different locations with two cohorts of participants in each location. The inner city VTI group and their mentors were given a traditional face-to-face lecture lasting around 45 minutes. Identical numbers of urban counterparts were given the same time to work through the same body of knowledge delivered through a CD-ROM. Knowledge retention was tested by means of a written test with 10 set questions given to all participants. An additional research tool was employed which involved the use of a group discussion with each of the four groups to elicit their views on the learning experience to which they had been party. The group interview data was analysed using a thematic approach to draw out emerging similarities about the perceived advantages and disadvantages of the two delivery methods.

Results

Retention

The aim was to determine whether differences existed between the e-learning and lecture groups for both trainees and for trainers. As a subsidiary question, it was felt important to determine whether e-learning or lecture presentation appeared to offer advantages or disadvantages for either trainees or trainers.

The group that scored best overall were the e-learning trainers (x̄ = 1.48). Next were the lectured trainers (x̄ = 0.96). Next were the lectured trainees (x̄ = 0.93), and last the e-learning trainees (x̄ = 0.78).

These findings do not present clear cut advantages for e-learning or lectured groups, and a one way ANOVA (Table 1) confirms that there is no clear advantage for either set.

Table 1 ANOVA
Full size table

However, post hoc t tests reveal significant differences between groups in the following pairings, as revealed in Table 2.

Table 2 Differences between e-learning and lecture for the two groups
Full size table

In other words:

  • For the trainees, lecturing was significantly more successful than e-learning

  • For the trainers, e-learning was significantly more successful than lecturing, and

  • Trainers performed significantly better than trainees overall.

It should be noted that these were small groups and one should be wary of drawing too much in the way of conclusions from these findings. However, if a tentative conclusion can be put forward, it is that e-learning may have benefited those with a higher baseline knowledge (ie the trainers), while those with a more basic knowledge seemed to benefit more from a personalised lecture.

One finding that emerges in favour of e-learning is that there is consistency of coverage of the topics. By contrast, in one or two of the topic areas measured by the questions, it was notable that the lecture groups appeared to have retained little information. There was some evidence here, particularly in the trainee lectured group, of the 'primacy and recency' phenomenon known to experimental psychology, wherein beginning and end material (particularly when given in a list) is retained better than that in the middle. Topics at the very beginning and very end of the lecture appeared to be remembered better than those in the middle. Interestingly, this was far less evident in the e-learning groups.

Group feedback on the methods

The group discussion produced some revealing information, the narrative of which is summarised below. The responses received appeared to tally with the amount of experience already gained in electronic learning media, with those having the most experience providing a more positive response.

The perceived advantages of RLOs in learning and teaching

The participants identified the benefits of e-learning in relation to the freedoms of time, pace and place.10 Specifically, time saved in travel to the course venue was valued. The visual presentation of the material was deemed to be an improvement on a paper-based approach especially if completion of the activities was verified by certified accreditation for the CPD record.

The topic of clinical governance was judged to be appropriate for CD-ROM delivery. The pictorial element and use of video clips which could be re-visited to confirm understanding was seen to be beneficial. The opportunity to check understanding was appreciated, especially as this could be done without the recourse to another person. The avoidance of public embarrassment was valued particularly by mature trainee mentors.

The perceived disadvantages of RLOs in learning and teaching

The VTIs were generally more positive about the benefits of RLOs, many having used similar materials during their university training programmes. The key objection to the electronic medium was voiced by two of the mentors who were concerned about the isolation that CD-ROM learning might entail. The likely loss of personal contact with fellow professionals was felt deeply with reference made to what Wenger would identify as communities of practice.11 The loss of social interaction to discuss professional understanding within a supportive domain was highlighted. Concern was expressed about the isolation of e-learning approaches. This argument has both social and pedagogic dimensions: Knowles identified that adult learning involves negotiated understandings, and for the adult learner especially learning is a social activity, with knowledge constructed through the active engagement of learners.12

An additional critique of the learning design related to the elements of personal time. One mentor expressed strong concern that easy access at any time equated with legitimacy given to the expectation that CPD could be allowed to impinge on his private time. In the light of the criticisms discussed above, the overriding preference was for blended learning involving a mix of e-learning and face-to-face delivery.

Discussion

Certain of the results obtained here — such as the superior performance of the trainers over the trainees — are explicable in terms of superior previous knowledge. That superior knowledge was taken as a 'given', and not controlled for in the design of the study. What was interesting in the findings was the effect of that additional knowledge. It appeared, perhaps by enabling and contextualising particularly of the electronically presented information, to enable the latter to be better understood and retained, something that was more difficult for the more naïve group of trainees, who appeared to benefit from the more personal presentation of the lecture material, with its asides, anecdotes and mini-explications. It is perhaps necessary for e-learning to find ways of incorporating the kind of idiosyncratic asides and variation that can make lectures interesting and memorable.

This project was a small one, and as already noted one should beware of attempting to draw definitive conclusions. Assessing larger groups over a number of regions would enhance the validity of the work, particularly given the wide variability in the nature and quality of lecture performance, which gives rise to the possibility of large variation in the memorability of lectured material. That variability would be valuable in itself to assess on a topic as discrete as clinical governance, but particularly so in relation to the lessons that it could provide for the design of e-learning programmes. How, in other words, does the inspiring lecturer provide 'value added'? And, if the latter can be isolated, how can this be incorporated into e-learning?

As part of a research-informed approach to teaching and learning, an important element of this project has been to encourage the involvement of the learner's voice in the design and management of the learning experience. In gaining an awareness of the needs and wishes of learners it is the intention that adaptations will be made to the teaching methods evaluated here.

Findings from both the retention study and the group feedback have pointed to the benefits of the ease and speed of use afforded by e-learning alongside the need for the interactivity and questioning afforded by traditional lecture. If the latter (that is, questioning and interaction) can in some way be incorporated into the e-learning methods on trial here, substantial benefits may be provided by e-learning and this provides great potential for the teaching of certain subjects in CPD. Such interactivity could be provided, for example, by chatrooms and other on-line engagement, either with an expert or with peers — and the incorporation of the latter to the simple CD-ROM format used in this piece of research suggests avenues for further research.

The use of chatrooms and on-line engagement is clearly an important avenue for further development and assessment. Anecdotal evidence appears to suggest that the benefits of any kind of on-line discussion depend on the opportunity for sustained 'debate', the efficacy of which in turn rests on issues such as the 'critical mass' available for discussion: large groups would seem to offer more opportunity than small ones. There needs to be some sense of a community of learners — a community willing to ask questions, to admit to ignorance, to offer insights. Here, the optimum size of group, and the best format for the structuring of the discussion in relation to the topic under consideration are all potential subjects for further research in relation to e-learning.

To support the recommendations arising from this research we return to the work of Salmon, who when identifying a number of models for future development in e-learning, cites the use of learning through participation in online communities. Such an approach, involving specially designed platforms to support the CD-ROM delivery involves the addition of a chat room discussion board, to allow for synchronous communication at a time appropriately set aside for CPD.

The use of virtual electronic spaces to support the professional world of dentistry could also be productive in the development of communities on-line where good practice and concerns can be openly discussed. Such an approach would provide for the learning community in both a pedagogic and professional sense.

The findings here point to the very great benefits that may accrue in future from e-learning, particularly on topics (such as the one used here) that do not demand close personal support or scrutiny, as would be the case with elements of clinical learning. Given the adaptation and enhancement of e-learning that research of the kind reported here enables, e-learning promises improved learning in more flexible settings, with vigorous communities of practices engaging.