Main

Tay Sachs disease (TSD) is a neurodegenerative disorder, fatal in children usually by the age of five, with an autosomal recessive pattern of inheritance.1 The identification of the enzymatic deficiency2 of hexosaminidase A in those affected with TSD enabled the worldwide implementation of carrier screening programs within the high risk Ashkenazi Jewish community.3,4 In Australia, the Melbourne TSD carrier screening program involves senior secondary school students and has been offered successfully since 1997.5 Education is an important aspect of screening programs, allowing students to gain an understanding of relevant information and enabling an informed decision to be made regarding testing.6 Until 2001, students in the Melbourne program received an education session in the form of an oral presentation accompanied by visual aids and written material.7

In any screening program, there are resource implications to be considered for delivery of effective and consistent educational messages. Another educational approach, increasingly being adopted in a large range of areas, is computer-based instruction (CBI).8 In terms of educational approaches to genetics, CD-ROM resources have been developed for a range of topics and audiences.9–15 In addition to these resources, genetic educational web sites are also accessible.15,16 These resources have been designed to cope with the global increase in genetic knowledge and the need to educate large numbers of people, who do not have a science background, on genetic issues so they can make informed decisions. In the setting of a carrier screening program, it is the general population, often with little or no knowledge of the genetics, that is the target audience. It is important for this audience to be educated not only about general genetic concepts, but the characteristics of a particular medical condition as well as the implications for themselves, for their family, and for having children, within the one educational session. This is a challenging task compounded by the notion that genetics has been described as “one of the most difficult scientific concepts for students to learn.”17 However, Jacobson and Kozma17 also noted that well-designed technological tools, such as CBI tutorials, provide students with an understanding of science in addition to an opportunity to “creatively construct, authentically experience, and socially develop and represent their understanding.” Research has shown well-designed educational resources to be effective if their design and development are based on sound theories of learning.18

The potential advantages offered by CBI over an oral presentation prompted our design, development, and evaluation of a CBI resource for use in the Melbourne TSD carrier screening program. The aim of the resource is to present the necessary and relevant concepts of genetics to stimulate active learning and knowledge construction that promote participant informed decision-making. As it is knowledge construction, rather than information acquisition (i.e., active vs. passive learning) that is the goal of this resource, the design and development were based on elements of learning theories, cognition, and constructivism. A cognitive theory of learning assumes that people process information through two channels.19,20 The visual/pictorial and the auditory/verbal channels each have a limit for the amount of information they are able to process20,21; however, the use of both channels promotes active learning and knowledge construction.22,23 This information leads to the theory of constructivism, which states that learning is meaningful when it is formed through knowledge construction based on the individual’s own characteristics and prior experiences.24,25

We report here on the design, development, formative, and summative evaluation of a CBI resource for a genetic carrier screening program that is based on educational principles and serves as a model for more general CBI genetics resources.

METHODS

CBI resource development

The development and evaluation involved students from the target audience (senior secondary school students aged 15 to 17) as well as a team of content and production experts. This study was approved by the Royal Children’s Hospital, Ethics in Human Research Committee, Melbourne, Australia on 19th July, 2002, EHRC ref no. 22038A.

A resource prototype was developed with the educational content drawn and adapted from the oral presentation used within the TSD screening program. A map of the resource layout was first produced that informed the production of a planning grid. The grid included text to be presented on each screen as well as possible graphical representations and interactivity questions, thereby providing a set of instructions in an easily interpretable format for the programmer. This planning grid was subjected to formative evaluation involving content experts. Through consultation with an instructional designer, computer programmer, and graphic designer regarding visual appearance, navigation, and interactivity aspects, an a-version of the resource was produced. This a-version, which was developed for cross-platform delivery, was the first trial version of the resource and was shown to students from the target audience for formative evaluation. After further evaluation and modification, the final b-version of the resource was produced.

Formative evaluation

Two processes of formative evaluation were used during the development of the resource: (1) content expert evaluation of the resource prototype; and (2) user group evaluation of the resource a-version.

Content expert formative evaluation

Participants were experts in the area of genetics education, clinical genetics, genetic counseling, and multimedia education. They reviewed the educational material of the resource planning grid for validation of content and layout. Their feedback was discussed through semistructured interviews to revise changes.

User group formative evaluation

Eight students were invited to participate in the user group formative evaluation. Each student had been offered the TSD carrier screening program the previous year and was studying biology in their final year of secondary school. These students attended a Jewish secondary school in Melbourne, Australia.

The evaluation involved a process of triangulation incorporating a strong qualitative data focus complemented by quantitative data. Triangulation allows for the formation of a more complete representation of the participants’ views.26 Participants were required to have viewed the resource before contributing to a focus group discussion. Therefore, while participants were working through the resource, they completed the first of two questionnaires. After this, further in-depth qualitative data collection was obtained through the method of focus group discussion using both the content data derived from the open-ended comments of the first questionnaire and the second questionnaire as a guide.

Questionnaire details

In this study, two questionnaires were modified27 for specific evaluation of this resource. The first questionnaire was self-administered and required users to rate and comment on introductory objectives and directions, content and structure, interactivity, navigation, graphics, visual appearance, and the package overall. Data were analyzed using a statistical software package (SPSS for Windows, version 11.0) with descriptive statistical procedures, while a content analysis was performed on the open-ended comments.28 The second questionnaire explored both positive and negative aspects of the resource.

Focus group details

Participants were initially asked to comment on their general impression of the resource and then explored problem areas and suggested improvements of the resource, using the second questionnaire administered by the facilitator. Conclusion of the discussion aimed at determining whether or not students would use the resource and understood the educational material presented.

Summative Evaluation

In order to assess the effectiveness of the resource, a comparison of knowledge outcomes was made with the successful oral educational session5 over two consecutive years. Knowledge outcomes were measured with a questionnaire scale consisting of seven multiple choice questions that covered general genetic concepts and inheritance as well as clinical and population features of TSD.

Students (aged 15–17 years) offered the TSD carrier screening program in Melbourne Jewish day schools, were randomly allocated to receive either the CBI resource or the oral presentation educational session. A total of 302 students completed a questionnaire knowledge scale both before and 2 to 10 days after the educational session on the day of genetic testing. Knowledge outcomes were assessed using a calculation of a percent knowledge score for each student. Both descriptive and comparison of means statistical procedures to the 0.05 level of significance (SPSS for Windows, version 11.0) were used in analysis.

RESULTS

User group formative evaluation

Users were presented with the a-version of the resource to review. The open-ended comments of the first questionnaire were categorized for content analysis.28 These categories were objectives and directions, content, structure, interactivity, navigation, text, graphics, interface, and the resource as a whole. The results are expressed as a percentage, which indicate the percent of times a particular category is mentioned from all responses. Participants were asked to consider the best and worst aspects of the resource. For the best aspects, the percentage of times these categories were mentioned included interactivity (50%), structure (37.5%), and graphics (37.5%). The worst aspects mentioned were navigation (87.5%), content (37.5%), and objectives (25%). These results were supported by the quantitative data, which showed that navigation scored poorly, whereas graphics were a favorite (Fig. 1).

Fig 1
figure 1

Formative evaluation. Students rated seven aspects of the resource for their effectiveness through scales of the quantitative data questionnaire.25 Mean score calculated from each of the students’ scores was converted to a percentage of the highest score possible with 100% being extremely effective. Generally each aspect scored above 60% with the graphic aspect being the most effective with 82.6%; however, the navigation proved problematic for the students with the lowest score of 47.0%. These data were supported by the content analysis and focus group discussion data collected.

Full size image

During in-depth discussion, students expressed the following thoughts: “the spreadsheets [meaning screen] were very clear and has concise, relevant information…,” “there were not enough [graphics] but the ones that were there were very good,” “It was a bit difficult to navigate around the site and to revert back to a page that I had previously seen” “…it was a pretty good package…,” and “I think it would be effective learning tool for use in classes.”

Resource b-version

A revised final b-version of the resource was produced following the user group formative evaluation. The focus was on incorporation of further graphics as well as revision of the navigation. More graphics were designed, obtained, and incorporated into almost every screen of the resource to enhance visual stimulation. The navigational problems of the a-version were addressed with the page titles of the section maps being highlighted and hyperlinked to enhance the ease of navigation.

The resource is entitled “Genetics in the community: Tay Sachs disease” and is delivered on CD-ROM. The interface was adapted from the Melbourne TSD screening program web site, www.taysachs.net, and lies within an Internet Explorer framework to enable future delivery via the internet. The sequence of the five sections, described here, was created in a linear progression for the user to gain maximum knowledge and understanding, but users can navigate through each section independently via the main menu. For clarity, organizational purposes, and enhancement of visual stimulation, each of the five sections has a different color scheme. Fig. 2 displays a screen from the resource as a representation.

Fig 2
figure 2

Screen grab. One screen of the five sections of the resource “Genetics in the Community: Tay Sachs disease” is displayed here. Each section has a different color scheme; however, all have this format. This screen, from the Introduction section, displays the interactive rollover world map explaining the ethnic origin of a range of common genetic conditions.

Full size image

1. Introduction

This section introduces the user to the resource and provides a general overview of how genes and environment determine a global variation in people (Fig. 2). This concept emphasizes that different conditions are more prevalent in different ethnic groups and not just one ethnic group, thereby minimizing the potential stigma for the Ashkenazi Jewish Community.

2. TSD: The facts

The clinical description, genetic basis, and inheritance of TSD are described in this section. At the conclusion, users can confirm their knowledge of TSD by completing multiple choice questions, which provide on-screen and immediate feedback to their submitted answers.

3. Screening

This section describes how individuals can be tested for their TSD carrier status through enzyme and DNA testing. It provides information regarding the screening process and the involvement of genetic counselors. Frequently Asked Questions (FAQs), which were formed from knowledge of student concerns raised in previous years of the program, are answered and two multiple choice feedback sections are included. Users can also view a video depicting a couple speaking of their experiences with TSD screening, using actors portraying a real-life situation.

4. Introductory genetics

General genetic concepts, such as DNA, chromosomes, genes, proteins, and inheritance are described in this section. This section is included for those people with little or no prior knowledge of genetics and also contains multiple choice feedback interactivity, concluding with guidance to the web resources section or main menu. This section on genetics is considered a “learning loop” and may be omitted by those with prior knowledge.

5. Web resources

For those users requiring or interested in further in depth information, a list of hyperlinked internet web sites is presented.

Summative evaluation: Knowledge outcomes

Before education, the students scored a mean of 75.6 ± 1.04% on the knowledge scale. After education, for both those receiving the CBI and the oral presentation, there was a statistically significant increase in the combined mean score to 86.8 ± 0.84%. This corresponded to a mean difference of 11.2 ± 0.98%; P < 0.0001 (Fig. 3).

Fig 3
figure 3

Summative evaluation. Mean percent knowledge score is shown for 302 students offered the TSD carrier screening program. Before the education session of the program, the overall students scored a mean of 75.6 ± 1.04%. This score increased significantly (11.2 ± 0.98%, P < 0.0001) to 86.7 ± 0.84% after education, for both the CBI resource and oral presentation combined (hatched bars). Comparison of educational methods, oral presentation (black bars) or CBI resource (white bars), are presented for both before (oral 74.8 ± 1.55%, CBI 76.1 ± 1.46%) and after education (oral 85.7 ± 1.00%, CBI 87.6 ± 1.21%). Each method proved to be effective for students’ knowledge construction, with no significant difference between methods (P = 0.996).

Full size image

When comparing the mean difference in before and after knowledge scores for those receiving the CBI resource (11.42 ± 1.49%) and those receiving the oral presentation (11.43 ± 1.38%), there was no significant difference (P =0.996;Fig. 3). This outcome was maintained for both years of measurement with there being no significant difference recorded between the years (P = 0.735).

Of the seven individual knowledge questions, there were three particular questions in which students showed a greater understanding after the education intervention. These three questions covered concepts of how TSD affects an individual, the impact of being a TSD carrier on the individual and recessive inheritance. There was no significant difference between either educational method for the mean difference in knowledge score for each individual question (data not shown).

DISCUSSION

The increased use of computers over the past few decades in both the home29 and at school30 has resulted in a fundamental shift in the perceptions of young people and their use of technology.31 Bandura concluded that the young people of today have both the cognitive and social abilities to use multimedia optimally and with self-efficacy.31 There is also some evidence to suggest that CBI use in the classroom has a positive effect on student motivation, self-esteem, and classroom behavior.32,33 These findings provide strong evidence to suggest that using CBI resources for health education are both feasible and relevant.

Originally, multimedia focus was on technological capacity rather than promoting learner thinking; however, more recently, an interest in research and theory has driven the design and assessment of multimedia resources and uses.18 “Genetics in the Community: Tay Sachs disease” was designed with mini-malist technological requirements to enable widespread use. Although the technological capacity of the school systems delivering the resource was a consideration, incorporation of the learning theories of constructivism and cognitive learning were important. The idea is for the students to make sense of the material presented based on their own experiences and to build a mental representation to allow them to decide whether or not they wish to be tested for their TSD carrier status. Meaningful learning based on a cognitive theory of learning assumes that learners use the dual channels of visual/pictorial and auditory/verbal for information processing,19,20 they have a limited capacity to process information in each,20,21 and that learners engage in active processing of information.22,23 Therefore, the resource incorporated text, graphics, audiovisual material, interactive quizzes, and animations, which the students in this study emphasized as being important in keeping their attention. The minimum time required to complete the resource was restricted to 10 minutes, to maximize students’ capacity for information processing. This is much less than the oral presentation, which is given within a 45-minute lesson session. To facilitate the students’ active processing of information by engaging and motivating learning through situating learning in an authentic, real-world context34,35 is especially relevant, as the students are learning about genetics and health while being offered a genetic test for their carrier status.

Because we are considering the use of this resource as an alternative educational approach to the oral presentation in the TSD carrier screening program, it is important to look at the potential benefits of the resource. Fletcher36 outlines a number of benefits of CBI such as the nonrequirement of students to gather at specific times and places in order to learn, or for there to be active engagement between the instructor and students simultaneously. Therefore, the resource can be used by a wide range of individuals in their own time and place. Individuals can regulate their own learning,33 which guards against overloading of individuals’ working memories.37 Also, the time taken to reach a given objective has been found to be related to a student’s prior knowledge.38 Therefore, as this resource is given to the entire year 11 group (15–17 year olds) in the screening program, those students with a strong biology background are able to progress through the introductory genetics section at a faster pace than those with no prior knowledge. CBI also allows the interaction between the student and the computer as well as with other students at the time of learning.39 This is an effective way to enhance deep learning via “community knowledge building.”40 However, as this study suggests, there is the option of teacher facilitation during set classes and incorporation into the curriculum for biology, health issues, or contemporary issues in English studies. Financial benefits can be seen when CBI allows for the substitution of human resources with computer instruction.36 This resource will not only provide a decision-aiding, problem-solving, and instructional message but it will do so consistently over a period of time, as the knowledge outcome results of this study have shown. When the material remains constant, Orlansky and String, suggest the use of CBI to be faster but may be no more effective in knowledge construction than traditional instruction.41 This study has supported this notion with the CBI resource shown to be equally effective as the lengthier oral presentation in knowledge construction for screening program participants. The resource could easily be adapted for screening programs for other genetic conditions such as fragile X syndrome, hemochromatosis, thalassemia, sickle cell disease, and cystic fibrosis.

Although there are many potential advantages for the use of this resource, some limitations include not only the technological capacity possessed by the schools and individuals to deliver the program, but also the physical space to do so. The resource is now available via the internet, allowing students to access the information at a time and place of convenience (http://www.taysachs.net).

The main objective of the education session within the screening program is to promote understanding to allow informed decision-making. Relevant concepts in this process include the effects of TSD, impact of carrier status, and mode of inheritance. Although students showed a significant increase in knowledge overall, it was these three concepts in particular that showed the greatest improvement in student understanding. The knowledge outcome comparison of the resource with the oral presentation has shown “Genetics in the Community: Tay Sachs disease” to be comparable to the successful oral presentation of the program.5 The added benefits described in this report favor the future use of the resource in TSD screening programs and reflect the value of considering educational theory when designing genetics multimedia resources in general.