Assessing the Effectiveness of Problem-Based Learning of Preventive Medicine Education in China

Abstract

Problem-based learning (PBL) is defined as a student-centered pedagogy which can provide learners more opportunities for application of knowledge acquired from basic science to the working situations than traditional lecture-based learning (LBL) method. In China, PBL is increasingly popular among preventive medicine educators and multiple studies have investigated the effectiveness of PBL pedagogy in preventive medicine education. A pooled analysis based on 15 studies was performed to obtain an overall estimate of the effectiveness of PBL on learning outcomes of preventive medicine. Overall, PBL was associated with a significant increase in students' theoretical examination scores (SMD = 0.62, 95% CI = 0.41–0.83) than LBL. For the attitude- and skill-based outcomes, the pooled PBL effects were also significant among learning attitude (OR = 3.62, 95% CI = 2.40–5.16), problem solved skill (OR = 4.80, 95% CI = 2.01–11.46), self-directed learning skill (OR = 5.81, 95% CI = 3.11–10.85) and collaborative skill (OR = 4.21, 95% CI = 0.96–18.45). Sensitivity analysis showed that the exclusion of a single study did not influence the estimation. Our results suggest that PBL of preventive medicine education in China appears to be more effective than LBL in improving knowledge, attitude and skills.

Introduction

Problem-based learning (PBL) was first implemented at McMaster University Medical School in the 1960s where it has revolutionized the field of medical education since then1. PBL is defined as a student-centered pedagogy in which participants are allocated in groups of up to 8 persons under non-directive tutors and given tasks or challenges that reflect situations relevant to the working environments they will experience2. In this way, the learners are empowered to integrate theory and practice and apply knowledge and skills to develop a viable solution to the problem3,4. PBL represents a paradigm shift from traditional pedagogy, which is more often lecture-based learning (LBL). LBL pedagogy focuses on factual knowledge and memorization, providing little chance for application of knowledge acquired from basic science to the working situation. There are many other advantages of PBL over LBL, including flexible knowledge, improved communication, collaborative skills and self-directed learning skills and a more enjoyable and motivational format5.

Originally, PBL was devised in an attempt to develop a teaching method for use for clinical medical education. A variety of medical schools in Europe (the Netherlands, Sweden, Norway and the United Kingdom), Asia (Japan, Korea and China), Canada, the United States, South Africa, Australia and New Zealand already have or are introducing PBL as a learning modality3,6,7,8. Now it has also been used in multiple disciplines including public health and preventive medicine9. For example, Dyke et al. had used PBL approach for teaching epidemiology and found that the PBL students got a far richer learning experience than their counterparts in the traditional stream10. As the public health system in China has been developing rapidly for the past decade, PBL is increasingly popular among preventive medicine educators striving to produce graduates who can solve complex public health problems efficiently. In recent years, a growing number of Chinese studies have investigated the effectiveness of PBL pedagogy in preventive medicine education. As one of the most well-known public health schools in China, the School of Public Health from Nanjing Medical University (NMU) has adopted a PBL in preventive medicine education from September 2012. NMU has established multidisciplinary cases throughout the course schedule to improve the positive outcomes, such as motivating learning, problem solved skill, collaborative skill and critical thinking. To the best of our knowledge, however, there are no reports or reviews summarizing the trials done to evaluate the effectiveness of PBL for preventive medicine education.

The uncertainty about the effectiveness of PBL and the heterogeneity in the published literatures provided the impetus for this study. Thus, the aim of the study was to perform a meta-analysis on all published studies to obtain an overall estimate of the effectiveness of PBL on learning outcomes of preventive medicine education in China.

Results

Study characteristics and quality assessment

The study selection process is depicted in Fig. 1. The search strategy yielded 110 entries, of which 64 in Chinese were considered to have potential value and were subjected to further examination. Based on the inclusion and exclusion criteria, we excluded 52 articles (23 were reviews or editorials; 16 no relevant quantitative outcomes; 9 no comparison with LBL; 3 no full text and 1 duplicate publication). Three additional articles were included from the reference review. Ultimately, a total of 15 articles involving 2,061 subjects published from 2006 through 2013 were included in this meta-analysis11,12,13,14,15,16,17,18,19,20,21,22,23,24,25. Characteristics of the selected studies are presented in Table 1.

Table 1 Characteristics of included studies
Figure 1
figure1

Flow chart summarizing identification of included studies.

The methodological quality of each included study is summarized in Table 2. The quality scores ranged from 4 to 6. A majority of the studies are cluster randomized trials, while only one is randomized controlled trial (RCT). Blinding is a critical measure to protect against bias, however, none of the studies mentioned the application of this method. Some other bias protection measures such as control for important factors and incomplete data bias were often used in the publications. Whether the outcome assessors and data collectors were blinded to subjects' assignments or not was not mentioned in the studies.

Table 2 Methodologic quality of studies included in the meta-analysis

Quantitative synthesis

The effects of PBL methods were evaluated by synthesizing theoretical examination scores, pass rate, the improvement on learning attitude, problem solved skill, self-directed learning skill and collaborative skill in this meta-analysis. The effects of PBL on theoretical examination scores were reported by all the included studies. When all the eligible data were pooled into the meta-analysis, PBL was associated with a significant increase in students' examination scores (SMD = 0.62, 95% CI = 0.41–0.83, Pheterogeneity < 0.001) (Fig. 2). Four studies reported pass rate of the theoretical test, when they were pooled together, there was a lack of heterogeneity (P = 0.83) and the PBL effect was significant (OR = 2.88, 95% CI = 1.44–5.75). For the attitude- and skills-based outcomes, the pooled PBL effects were also significant (OR = 3.62, 95% CI = 2.40–5.16, Pheterogeneity = 0.444 for learning attitude; OR = 4.80, 95% CI = 2.01–11.46, Pheterogeneity < 0.001 for problem solved skill; OR = 5.81, 95% CI = 3.11–10.85, Pheterogeneity = 0.048 for self-directed learning skill; and OR = 4.21, 95% CI = 0.96–18.45, Pheterogeneity < 0.001 for collaborative skill) (Table 3).

Table 3 Summary of effect sizes for PBL and LBL
Figure 2
figure2

Forest plot for the effects of PBL on examination scores compared with LBL.

We then evaluated the effects of PBL in students' examination scores according to the quality scores. For quality scores = 6, SMD = 0.70, 95% CI = 0.37–1.04, Pheterogeneity < 0.001; scores = 5, SMD = 0.53, 95% CI = 0.37–0.70, Pheterogeneity = 0.153; scores = 4, SMD = 0.38, 95% CI = 0.03–0.73, Pheterogeneity = 0.924. The result indicated that studies with high quality scores would yield significantly larger effect sizes than those with low quality scores.

Test of heterogeneity

There was significant heterogeneity (P < 0.001) among included studies. We assessed the source of heterogeneity by quality scores, source of participants and sample size (participants more than 50 in both PBL and LBL groups). As a result, quality scores (P = 0.035) and source of participants (P = 0.048), but not sample size (P = 0.146), were found to contribute to the substantial heterogeneity.

Sensitivity analyses

Sensitivity analyses indicated that Lu's study was the main origin of heterogeneity. However, when this article was excluded the overall result (SMD = 0.57, 95% CI = 0.38–0.77, P < 0.001) and the heterogeneity (P < 0.001, I2 = 72.6%) was not materially influenced. Further exclusion of any single study got similar results with a range from 0.56 (95% CI = 0.37–0.75) to 0.67 (95% CI = 0.46–0.87), suggesting that the results of this meta-analysis are statistically reliable.

Publication bias

Begg's funnel plot and Egger's test were performed to evaluate the publication bias of the literatures. As a result, the funnel plots did not show any obvious asymmetry. Then, the Egger's test was adopted to provide statistical evidence of funnel plot symmetry. The results also indicated no evidence of publication bias (P = 0.432).

Discussion

Many medical schools internationally are changing their curricula and moving to PBL programs26. While in China, the application of PBL pedagogy is still in the initial stage, especially among preventive medicine education. In this meta-analysis, we found that compared with LBL, PBL was more effective in all domains including theoretical knowledge, attitude and skills. Nevertheless, the results of attitudes and skills were clearly less precise than that of the theoretical knowledge. It may be due to the less eligible studies and smaller sample sizes. Moreover, the assessment of theoretical knowledge by a test or exam seems more objective than that of attitudes and skills by the participants' sensation.

PBL is one of the best described interactive learning methods27. According to the questionnaire results and the tutors, students showed more enthusiasm for PBL rather than LBL. It is reported that learning is most effective when students are actively involved in PBL28. Therefore, our results were consistent with previous conclusions, which may improve the application of lesion-based theory to practical knowledge as well as achieving specific goals. Although the heterogeneity amongst studies was large (I2 = 80.1%), we found that the sources of heterogeneity were from quality scores and source of participants. Furthermore, the results of the subgroup analysis only partially explained the heterogeneity. Variations in other aspects such as study design, involved discipline and the duration may also play a part.

In the analysis stratified by quality scores, elevated effects were more pronounced among studies with higher quality score, suggesting the importance of methodological quality during PBL intervention. Overall, the methodological quality of the included studies in this meta-analysis was not high. First, most of the studies are not randomized controlled trials (RCTs). In China, students are divided into classes when they entered the university, so the researchers prefer to allocate according to the classes rather than the individuals. Second, as an investigation in the field of education, it is impossible for the researchers to use the blinding method during the whole curriculum. Third, there is no standard criterion for evaluating the effectiveness of the PBL pedagogy. Whether the outcome assessors and data collectors were blinded to subjects' assignments or not was also poorly reported. In a word, high risks for selection bias, performance bias, as well as measurement bias do exist in the literatures we included.

In conclusion, our meta-analysis shows that PBL in preventive medicine education appears to be more effective than LBL in improving knowledge, attitude and skills. For future studies assessing the implementation of PBL, high quality of problems, uniform and objective outcome measurement and well-designed RCTs are required. Only through studies which are strategic planned and conducted with carefully quality controlled, can we ever expect to achieve the practical goals.

Methods

Search strategy

Relevant biomedical sciences and educational databases [PubMed, EMBASE, China National Knowledge Infrastructure (CNKI) and Chinese Biomedical Literature database (CBM) were searched from inception through September 2013. The search strategy was to use and combine the following key words: “problem-based learning”, “PBL”, “preventive medicine”, “epidemiology”, “medical statistics”, “health statistics”, “environmental health”, “nutrition science and food hygiene”, “occupational health and medicine”, “child and adolescent health”, “toxicology”, “hygienic toxicology”, or “social medicine”. In addition, references of all included articles were also identified by a manual search and studies matching the eligible criteria were retrieved.

Inclusion criteria

Studies included in the current meta-analysis have to meet the following inclusion criteria: (1) the study should compare the effectiveness of PBL group and traditional LBL group; (2) the courses of PBL pedagogy should be preventive medicine professional disciplines; (3) both randomized and non-randomized studies were considered and (4) quantitative data about the effectiveness of PBL were available (i.e., they had to report a mean change in knowledge, attitudes or skills that occurred in response to an PBL intervention and was measured using a numeric scale such as an examination or test).

Data extraction and quality assessment

For each study, the following basic information was collected: the first author's name, publication year, the involved disciplines, number of participants in each group, educational background of the participants, mean scores of both groups and the outcome measure. Two of the authors extracted all data independently according to the inclusion criteria listed above and reached a consensus on all the items. If more than one article was published using the same population, we selected the most recent or most informative report.

Recognizing that many nonrandomized studies would be included, an adaptation of the Newcastle-Ottawa Scale was adopted to abstract information on methodological quality. The included studies were judged in terms of a proper method of randomization, allocation concealment, blinding, the comparability of the populations, the completeness of the data and the assessment of outcome. The full score was 8 triangles and a high-quality study was defined as a study with 5 or more triangles.

Statistical analysis

Standardized mean difference (SMD) for continuous data and odds ratio (OR) for dichotomous data were performed to estimate the pooled effects. Heterogeneity assumption was identified by the I2 statistics. A P-value ≤ 0.10 for the I2 test indicated a lack of heterogeneity among the studies and then random-effects model (DerSimonian and Laird method) was used to calculate the summary standardized mean difference (SMD) or OR estimate of each study29. Otherwise, the fixed-effects model (the Mantel-Haenszel method) was used30. Sensitivity analyses were performed to assess the stability of the results, namely, a single study in the meta-analysis was deleted each time to reflect the influence of the individual data set to the pooled SMD. To evaluate the publication bias, Funnel plots and Egger's linear regression test was applied31. All analyses were carried out with Stata software (version 11.0; StataCorp LP, College Station, TX, USA), using two-sided P values.

References

  1. Gwee, M. C. Problem-based learning: a strategic learning system design for the education of healthcare professionals in the 21st century. Kaohsiung J Med Sci 25, 231–239 (2009).

    Article  Google Scholar 

  2. Smits, P. B. et al. Problem-based learning versus lecture-based learning in postgraduate medical education. Scand J Work Environ Health 29, 280–287 (2003).

    Article  Google Scholar 

  3. Shankar, P. R., Nandy, A., Balasubramanium, R. & Chakravarty, S. Small group effectiveness in a Caribbean medical school's problem-based learning sessions. J Educ Eval Health Prof. Epub ahead of print (2014).

  4. Oda, Y., Onishi, H. & Sakemi, T. Effectiveness of student tutors in problem-based learning of undergraduate medical education. Tohoku J Exp Med 232, 223–227 (2014).

    Article  Google Scholar 

  5. Onyon, C. Problem-based learning: a review of the educational and psychological theory. Clin Teach 9, 22–26 (2012).

    Article  Google Scholar 

  6. Mansur, D. I., Kayastha, S. R., Makaju, R. & Dongol, M. Problem based learning in medical education. Kathmandu Univ Med J 10, 78–82 (2012).

    CAS  Article  Google Scholar 

  7. Hoffman, K., Hosokawa, M., Blake, R., Jr, Headrick, L. & Johnson, G. Problem-based learning outcomes: ten years of experience at the University of Missouri-Columbia School of Medicine. Acad Med 81, 617–625 (2006).

    Article  Google Scholar 

  8. Lee, Y. M., Mann, K. V. & Frank, B. W. What drives students' self-directed learning in a hybrid PBL curriculum. Adv Health Sci Educ Theory Pract 15, 425–437 (2010).

    Article  Google Scholar 

  9. Savery, J. R. Overview of Problem-based Learning: Definitions and Distinctions. Int J Probl Learn 1, 9–20 (2006).

    Google Scholar 

  10. Dyke, P., Jamrozik, K. & Plant, A. J. A randomized trial of a problem-based learning approach for teaching epidemiology. Acad Med 76, 373–379 (2001).

    CAS  Article  Google Scholar 

  11. Zhang, J., Zhao, X., Zhang, Q. & Zhu, L. Application of PBL in the teaching of preventive medicine on rural doctors [in Chinese]. J Xingjing Med Univ 36, 1049–1051 (2013).

    Google Scholar 

  12. Li, G. & Zhang, L. The exploration of PBL teaching method applied in medical statistics teaching [in Chinese]. J Educ I Jilin 29, 53–55 (2013).

    Google Scholar 

  13. Gao, L., Li, X. & Jin, Y. Application of PBL teaching method in occupational health and occupational medicine teaching and its evaluation [in Chinese]. Chinese J Med Educ Res 12, 577–580 (2013).

    Google Scholar 

  14. Zhao, Y., Xu, G. & Zhao, X. The application of PBL teaching method in experimental teaching of toxicology [in Chinese]. J Henan Med Coll Staff Work 25, 211–213 (2013).

    Google Scholar 

  15. Zhang, Z. et al. Application of PBL teaching method in the teaching of preventive medicine [in Chinese]. Basic Med Educ 14, 506–509 (2012).

    Google Scholar 

  16. Zhang, Q., Zhuang, Y., Han, H. & Yao, R. Evaluation of the problem-based learning effects on the teaching of child and adolescent health [in Chinese]. J Bengbu Med Coll 37, 345–347 (2012).

    Google Scholar 

  17. Wang, S., Shan, Y. & Du, X. Evaluation of the problem-based learning effects on the teaching of preventive medicine on undergraduate students [in Chinese]. China High Med Educ 3, 95–96 (2011).

    Google Scholar 

  18. Lu, Z. et al. Practice of problem-based learning in the teaching of epidemiology on postgraduate students [in Chinese]. China J Mod Med 21, 1126–1128 (2011).

    ADS  Google Scholar 

  19. Hu, D. & Liu, Q. Effect evaluation about problem-based learning (PBL) in experimental teaching of health statistics for graduate students [in Chinese]. China High Med Educ 2, 110–111 (2011).

    ADS  Google Scholar 

  20. Sun, R., Wang, J. & Sun, F. Evaluation of the problem-based learning effects on the teaching of medical statistics [in Chinese]. Educ Chinese After-school 14, 95–96 (2010).

    CAS  Google Scholar 

  21. Zhang, Y., Wang, W., Liu, L., Wu, Y. & Yi, D. Practice of problem-based learning in theory teaching of medical statistics on eight-year program medical students [in Chinese]. J Shangxi Med Univ 12, 977–980 (2010).

    Google Scholar 

  22. Zhang, Y., Wang, W., Liu, L., Wu, Y. & Yi, D. The preliminary study of Problem-Based Learning teaching reform on medical statistics [in Chinese]. Res Med Educ 9, 1697–1699 (2010).

    Google Scholar 

  23. Wu, S., Cao, P. & Lei, J. The comparison of PBL and traditional method effects on the experimental teaching of medical statistics [in Chinese]. Guid Sci Educ 4, 47–48 (2010).

    CAS  Google Scholar 

  24. Zhang, Y., Guo, C. & Huang, J. The application of problem-based learning to health statistics in advanced vocational education [in Chinese]. J Anhui Health Voc Tech Coll 8, 89–90 (2009).

    CAS  Google Scholar 

  25. Deng, S. & Guo, R. Application of PBL teaching method to preventive medicine [in Chinese]. J Youjiang Med Coll Nat 4, 677–678 (2006).

    Google Scholar 

  26. Azer, S. A. Introducing a problem-based learning program: 12 tips for success. Med Teach 33, 808–813 (2011).

    Article  Google Scholar 

  27. Williams, B. Self direction in a problem based learning program. Nurse Educ Today 24, 277–285 (2004).

    Article  Google Scholar 

  28. Carrero, E., Gomar, C., Penzo, W. & Rull, M. Comparison between lecture-based approach and case/problem-based learning discussion for teaching pre-anaesthetic assessment. Eur J Anaesthesiol 24, 1008–1015 (2007).

    CAS  Article  Google Scholar 

  29. Mantel, N. & Haenszel, W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst 22, 719–748 (1959).

    CAS  PubMed  PubMed Central  Google Scholar 

  30. DerSimonian, R. & Laird, N. Meta-analysis in clinical trials. Control Clin Trials 7, 177–188 (1986).

    CAS  PubMed  PubMed Central  Google Scholar 

  31. Egger, M., Davey Smith, G., Schneider, M. & Minder, C. Bias in meta-analysis detected by a simple, graphical test. BMJ 315, 629–634 (1997).

    CAS  Article  Google Scholar 

Download references

Acknowledgements

This study was partly supported by the Project of Teaching Reform for Jiangsu Higher Education (2013JSJG038) and the Priority Academic Program Development of Jiangsu Higher Education Institutions (Public Health and Preventive Medicine).

Author information

Affiliations

Authors

Contributions

Conceived and designed the experiments: D.X., Z.Z., W.M. Performed the experiments: D.X., Z.Z., W.M. Analyzed the data: D.X., Z.L., C.H. Contributed reagents/material/analysis tools: D.X., Z.L., N.C., H.Z. Wrote the main manuscript text: D.X., Z.L. Reference collection and data management: D.X., Z.L. Statistical analyses and paper writing: C.H., T.N. Study design: D.X., Z.Z., W.M. Prepared figures 1–2: D.X. All authors reviewed the manuscript.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. The images in this article are included in the article's Creative Commons license, unless indicated otherwise in the image credit; if the image is not included under the Creative Commons license, users will need to obtain permission from the license holder in order to reproduce the image. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ding, X., Zhao, L., Chu, H. et al. Assessing the Effectiveness of Problem-Based Learning of Preventive Medicine Education in China. Sci Rep 4, 5126 (2014). https://doi.org/10.1038/srep05126

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing