Introduction

Problem-based learning (PBL) was first implemented at McMaster University Medical School in the 1960s where it has revolutionized the field of medical education since then1. PBL is defined as a student-centered pedagogy in which participants are allocated in groups of up to 8 persons under non-directive tutors and given tasks or challenges that reflect situations relevant to the working environments they will experience2. In this way, the learners are empowered to integrate theory and practice and apply knowledge and skills to develop a viable solution to the problem3,4. PBL represents a paradigm shift from traditional pedagogy, which is more often lecture-based learning (LBL). LBL pedagogy focuses on factual knowledge and memorization, providing little chance for application of knowledge acquired from basic science to the working situation. There are many other advantages of PBL over LBL, including flexible knowledge, improved communication, collaborative skills and self-directed learning skills and a more enjoyable and motivational format5.

Originally, PBL was devised in an attempt to develop a teaching method for use for clinical medical education. A variety of medical schools in Europe (the Netherlands, Sweden, Norway and the United Kingdom), Asia (Japan, Korea and China), Canada, the United States, South Africa, Australia and New Zealand already have or are introducing PBL as a learning modality3,6,7,8. Now it has also been used in multiple disciplines including public health and preventive medicine9. For example, Dyke et al. had used PBL approach for teaching epidemiology and found that the PBL students got a far richer learning experience than their counterparts in the traditional stream10. As the public health system in China has been developing rapidly for the past decade, PBL is increasingly popular among preventive medicine educators striving to produce graduates who can solve complex public health problems efficiently. In recent years, a growing number of Chinese studies have investigated the effectiveness of PBL pedagogy in preventive medicine education. As one of the most well-known public health schools in China, the School of Public Health from Nanjing Medical University (NMU) has adopted a PBL in preventive medicine education from September 2012. NMU has established multidisciplinary cases throughout the course schedule to improve the positive outcomes, such as motivating learning, problem solved skill, collaborative skill and critical thinking. To the best of our knowledge, however, there are no reports or reviews summarizing the trials done to evaluate the effectiveness of PBL for preventive medicine education.

The uncertainty about the effectiveness of PBL and the heterogeneity in the published literatures provided the impetus for this study. Thus, the aim of the study was to perform a meta-analysis on all published studies to obtain an overall estimate of the effectiveness of PBL on learning outcomes of preventive medicine education in China.

Results

Study characteristics and quality assessment

The study selection process is depicted in Fig. 1. The search strategy yielded 110 entries, of which 64 in Chinese were considered to have potential value and were subjected to further examination. Based on the inclusion and exclusion criteria, we excluded 52 articles (23 were reviews or editorials; 16 no relevant quantitative outcomes; 9 no comparison with LBL; 3 no full text and 1 duplicate publication). Three additional articles were included from the reference review. Ultimately, a total of 15 articles involving 2,061 subjects published from 2006 through 2013 were included in this meta-analysis11,12,13,14,15,16,17,18,19,20,21,22,23,24,25. Characteristics of the selected studies are presented in Table 1.

Table 1 Characteristics of included studies
Full size table
Figure 1
figure 1

Flow chart summarizing identification of included studies.

Full size image

The methodological quality of each included study is summarized in Table 2. The quality scores ranged from 4 to 6. A majority of the studies are cluster randomized trials, while only one is randomized controlled trial (RCT). Blinding is a critical measure to protect against bias, however, none of the studies mentioned the application of this method. Some other bias protection measures such as control for important factors and incomplete data bias were often used in the publications. Whether the outcome assessors and data collectors were blinded to subjects' assignments or not was not mentioned in the studies.

Table 2 Methodologic quality of studies included in the meta-analysis
Full size table

Quantitative synthesis

The effects of PBL methods were evaluated by synthesizing theoretical examination scores, pass rate, the improvement on learning attitude, problem solved skill, self-directed learning skill and collaborative skill in this meta-analysis. The effects of PBL on theoretical examination scores were reported by all the included studies. When all the eligible data were pooled into the meta-analysis, PBL was associated with a significant increase in students' examination scores (SMD = 0.62, 95% CI = 0.41–0.83, Pheterogeneity < 0.001) (Fig. 2). Four studies reported pass rate of the theoretical test, when they were pooled together, there was a lack of heterogeneity (P = 0.83) and the PBL effect was significant (OR = 2.88, 95% CI = 1.44–5.75). For the attitude- and skills-based outcomes, the pooled PBL effects were also significant (OR = 3.62, 95% CI = 2.40–5.16, Pheterogeneity = 0.444 for learning attitude; OR = 4.80, 95% CI = 2.01–11.46, Pheterogeneity < 0.001 for problem solved skill; OR = 5.81, 95% CI = 3.11–10.85, Pheterogeneity = 0.048 for self-directed learning skill; and OR = 4.21, 95% CI = 0.96–18.45, Pheterogeneity < 0.001 for collaborative skill) (Table 3).

Table 3 Summary of effect sizes for PBL and LBL
Full size table
Figure 2
figure 2

Forest plot for the effects of PBL on examination scores compared with LBL.

Full size image

We then evaluated the effects of PBL in students' examination scores according to the quality scores. For quality scores = 6, SMD = 0.70, 95% CI = 0.37–1.04, Pheterogeneity < 0.001; scores = 5, SMD = 0.53, 95% CI = 0.37–0.70, Pheterogeneity = 0.153; scores = 4, SMD = 0.38, 95% CI = 0.03–0.73, Pheterogeneity = 0.924. The result indicated that studies with high quality scores would yield significantly larger effect sizes than those with low quality scores.

Test of heterogeneity

There was significant heterogeneity (P < 0.001) among included studies. We assessed the source of heterogeneity by quality scores, source of participants and sample size (participants more than 50 in both PBL and LBL groups). As a result, quality scores (P = 0.035) and source of participants (P = 0.048), but not sample size (P = 0.146), were found to contribute to the substantial heterogeneity.

Sensitivity analyses

Sensitivity analyses indicated that Lu's study was the main origin of heterogeneity. However, when this article was excluded the overall result (SMD = 0.57, 95% CI = 0.38–0.77, P < 0.001) and the heterogeneity (P < 0.001, I2 = 72.6%) was not materially influenced. Further exclusion of any single study got similar results with a range from 0.56 (95% CI = 0.37–0.75) to 0.67 (95% CI = 0.46–0.87), suggesting that the results of this meta-analysis are statistically reliable.

Publication bias

Begg's funnel plot and Egger's test were performed to evaluate the publication bias of the literatures. As a result, the funnel plots did not show any obvious asymmetry. Then, the Egger's test was adopted to provide statistical evidence of funnel plot symmetry. The results also indicated no evidence of publication bias (P = 0.432).

Discussion

Many medical schools internationally are changing their curricula and moving to PBL programs26. While in China, the application of PBL pedagogy is still in the initial stage, especially among preventive medicine education. In this meta-analysis, we found that compared with LBL, PBL was more effective in all domains including theoretical knowledge, attitude and skills. Nevertheless, the results of attitudes and skills were clearly less precise than that of the theoretical knowledge. It may be due to the less eligible studies and smaller sample sizes. Moreover, the assessment of theoretical knowledge by a test or exam seems more objective than that of attitudes and skills by the participants' sensation.

PBL is one of the best described interactive learning methods27. According to the questionnaire results and the tutors, students showed more enthusiasm for PBL rather than LBL. It is reported that learning is most effective when students are actively involved in PBL28. Therefore, our results were consistent with previous conclusions, which may improve the application of lesion-based theory to practical knowledge as well as achieving specific goals. Although the heterogeneity amongst studies was large (I2 = 80.1%), we found that the sources of heterogeneity were from quality scores and source of participants. Furthermore, the results of the subgroup analysis only partially explained the heterogeneity. Variations in other aspects such as study design, involved discipline and the duration may also play a part.

In the analysis stratified by quality scores, elevated effects were more pronounced among studies with higher quality score, suggesting the importance of methodological quality during PBL intervention. Overall, the methodological quality of the included studies in this meta-analysis was not high. First, most of the studies are not randomized controlled trials (RCTs). In China, students are divided into classes when they entered the university, so the researchers prefer to allocate according to the classes rather than the individuals. Second, as an investigation in the field of education, it is impossible for the researchers to use the blinding method during the whole curriculum. Third, there is no standard criterion for evaluating the effectiveness of the PBL pedagogy. Whether the outcome assessors and data collectors were blinded to subjects' assignments or not was also poorly reported. In a word, high risks for selection bias, performance bias, as well as measurement bias do exist in the literatures we included.

In conclusion, our meta-analysis shows that PBL in preventive medicine education appears to be more effective than LBL in improving knowledge, attitude and skills. For future studies assessing the implementation of PBL, high quality of problems, uniform and objective outcome measurement and well-designed RCTs are required. Only through studies which are strategic planned and conducted with carefully quality controlled, can we ever expect to achieve the practical goals.

Methods

Search strategy

Relevant biomedical sciences and educational databases [PubMed, EMBASE, China National Knowledge Infrastructure (CNKI) and Chinese Biomedical Literature database (CBM) were searched from inception through September 2013. The search strategy was to use and combine the following key words: “problem-based learning”, “PBL”, “preventive medicine”, “epidemiology”, “medical statistics”, “health statistics”, “environmental health”, “nutrition science and food hygiene”, “occupational health and medicine”, “child and adolescent health”, “toxicology”, “hygienic toxicology”, or “social medicine”. In addition, references of all included articles were also identified by a manual search and studies matching the eligible criteria were retrieved.

Inclusion criteria

Studies included in the current meta-analysis have to meet the following inclusion criteria: (1) the study should compare the effectiveness of PBL group and traditional LBL group; (2) the courses of PBL pedagogy should be preventive medicine professional disciplines; (3) both randomized and non-randomized studies were considered and (4) quantitative data about the effectiveness of PBL were available (i.e., they had to report a mean change in knowledge, attitudes or skills that occurred in response to an PBL intervention and was measured using a numeric scale such as an examination or test).

Data extraction and quality assessment

For each study, the following basic information was collected: the first author's name, publication year, the involved disciplines, number of participants in each group, educational background of the participants, mean scores of both groups and the outcome measure. Two of the authors extracted all data independently according to the inclusion criteria listed above and reached a consensus on all the items. If more than one article was published using the same population, we selected the most recent or most informative report.

Recognizing that many nonrandomized studies would be included, an adaptation of the Newcastle-Ottawa Scale was adopted to abstract information on methodological quality. The included studies were judged in terms of a proper method of randomization, allocation concealment, blinding, the comparability of the populations, the completeness of the data and the assessment of outcome. The full score was 8 triangles and a high-quality study was defined as a study with 5 or more triangles.

Statistical analysis

Standardized mean difference (SMD) for continuous data and odds ratio (OR) for dichotomous data were performed to estimate the pooled effects. Heterogeneity assumption was identified by the I2 statistics. A P-value ≤ 0.10 for the I2 test indicated a lack of heterogeneity among the studies and then random-effects model (DerSimonian and Laird method) was used to calculate the summary standardized mean difference (SMD) or OR estimate of each study29. Otherwise, the fixed-effects model (the Mantel-Haenszel method) was used30. Sensitivity analyses were performed to assess the stability of the results, namely, a single study in the meta-analysis was deleted each time to reflect the influence of the individual data set to the pooled SMD. To evaluate the publication bias, Funnel plots and Egger's linear regression test was applied31. All analyses were carried out with Stata software (version 11.0; StataCorp LP, College Station, TX, USA), using two-sided P values.