Assessing the effectiveness of problem-based learning in physical diagnostics education in China: a meta-analysis

Problem-based learning (PBL) has been extensively applied as an experimental educational method in Chinese medical schools over the past decade. A meta-analysis was performed to assess the effectiveness of PBL on students’ learning outcomes in physical diagnostics education. Related databases were searched for eligible studies evaluating the effects of PBL compared to traditional teaching on students’ knowledge and/or skill scores of physical diagnostics. Standardized mean difference (SMD) with 95% confidence interval (CI) was estimated. Thirteen studies with a total of 2086 medical students were included in this meta-analysis. All of these studies provided usable data on knowledge scores, and the pooled analysis showed a significant difference in favor of PBL compared to the traditional teaching (SMD = 0.76, 95%CI = 0.33–1.19). Ten studies provided usable data on skill scores, and a significant difference in favor of PBL was also observed (SMD = 1.46, 95%CI = 0.89–2.02). Statistically similar results were obtained in the sensitivity analysis, and there was no significant evidence of publication bias. These results suggested that PBL in physical diagnostics education in China appeared to be more effective than traditional teaching method in improving knowledge and skills.

Study characteristics. The characteristics of these 13 included studies are shown in Table 1. All of them were published in Chinese between 2007 and 2014, and evaluated the effects of PBL compared to traditional teaching in physical diagnostics courses (also named clinical diagnostics in China). The sample sizes ranged from 20 to 230 students in PBL group and 25 to 229 in control group. The major of students in almost all studies was clinical medicine. The school system was different, however nearly half of the studies were carried out in the five-year system medical students. Nine studies described that the students in each group were matched for the basic medical knowledge test scores or college entrance exam scores. The duration of study was one semester, which was mentioned in 9 studies. Both knowledge scores and skill scores of physical diagnostics were used as outcome measures in all studies. The scores in the knowledge exam of physical diagnostics were used to assess how well the students mastered the related theoretical knowledge, and the scores in the practical skill test of physical diagnostics were used to evaluate the students' clinical skills including history taking, physical examination and diagnostic reasoning. Study quality. The summary of the methodological quality of each study is shown in the left panel of Fig. 2, and the graph of each quality item presented as percentages across all studies is shown in the right panel. Nine studies were designed as randomized controlled trials, and the other 4 studies were unclear. All studies reported complete outcome data and were free of selective reporting and other bias. The allocation concealment and blinding were not stated in these studies.
Effects of PBL on knowledge scores. The effects of PBL compared to the traditional teaching on knowledge scores were reported in all 13 included studies involving 2086 medical students (PBL group = 957, control group = 1129). Seven studies suggested that there was a statistically significant difference between the PBL and the control group in students' knowledge scores. Because a high degree of heterogeneity was observed across all of the 13 studies (I 2 = 95%, P < 0.001), the random-effects model was applied for the meta-analysis. The analytical results showed a significant difference in knowledge scores (SMD = 0.76, 95%CI [0.33, 1.19], P = 0.0006) in favor of PBL, compared with the traditional teaching (Fig. 3). The summary effects also favored the PBL group when using the fixed-effects model (SMD = 0.81, 95%CI [0.72, 0.91], P < 0.001). In addition, we performed a sensitivity analysis by excluding individual studies sequentially. Statistically similar results were obtained (SMD = 0.58~0.84, P < 0.01), suggesting the stability of this meta-analysis. In the stratified analysis by school system, we pooled the data from 6 studies in which target population were all five-year system students using the random-effects model (I 2 = 97%, P < 0.001). A statistically significant difference was also observed in knowledge scores ( (Fig. 4). The summary effects also favored the PBL group when using the fixed-effects model (SMD = 1.40, 95%CI [1.29, 1.51], P < 0.001). In addition, we performed a sensitivity analysis by excluding individual studies sequentially. Statistically similar results were obtained (SMD = 1.28~1.60, P < 0.01), suggesting the stability of this meta-analysis. In the stratified analysis by school system, we pooled the data from 5 studies in which target population were all five-year system students using the random-effects model (I 2 = 92%, P < 0.001). A statistically significant difference was also observed in skill scores (SMD = 1.08, 95%CI [0.47, 1.69], P = 0.0006) in favor of PBL, compared with the traditional teaching.
Publication bias. Funnel plots for knowledge scores (Fig. 5) showed no clear evidence of publication bias, and the test using Egger's method did not suggest publication bias, either (t = − 0.47, P = 0.649). Funnel plots for skill scores (Fig. 6) also showed no clear evidence of publication bias, and the test using Egger's method did not suggest publication bias, either (t = 0.14, P = 0.896).

Discussion
Although PBL has been widely adopted in medical education all over the world, the application of this teaching method in China is still at the elementary stage. Given that different education systems or cultural backgrounds could influence the effectiveness of PBL method 27 , the target population should be limited to Chinese medical students in the Chinese education system in order to evaluate the potential effectiveness of the PBL approach in China. This meta-analysis of studies in the field of Chinese physical diagnostics education assessed whether there were differences in quantitative outcomes between PBL pedagogy and the traditional teaching method.
The results of current meta-analysis show that the students in the PBL groups have better knowledge scores than those in the traditional teaching method groups, which are consistent with the findings of recent PBL studies in Chinese pediatric, pharmaceutical and dental education 10,28,29 . However, in western countries, some education researchers have questioned the performance of PBL students in the knowledge exams and have argued that PBL has a negative effect on the acquisition of factual knowledge 1,6,7 . Given the differences in higher medical education between China and the western countries, there are several possible reasons for the inconsistent results. Firstly, PBL pedagogy is totally different from the traditional teaching method in which students achieve knowledge passively from their teachers. Chinese students have received the traditional teaching model since primary education. PBL teaching model is a novelty for them and has greatly stimulated their interest in learning 30 . Secondly, most Chinese medical schools use unified textbooks and syllabuses for all students, which is very different from  the western countries. In the present study, both PBL and traditional students used the same textbook and syllabus of physical diagnostics. PBL students appeared to be more proactive in learning, which led them to get higher knowledge scores. Thirdly, teachers and other faculty personnel in a PBL program usually engage more with students 31 . Chinese teachers always emphasize the importance of course examinations and the keen pursuit of high scores promotes the students to excel in the exams 30 .
Despite the explosion of technological advances, the basic clinical skills of history taking, physical examination and counseling remain the most important and effective diagnostic tools 8 . Therefore, the development of above skills in the physical diagnostics education is considered crucial for medical students. According to the results of our study, there is a robust positive effect from PBL on the skill scores of students, and no single study reports negative effects. These results are consistent with the previous findings that PBL can effectively enhance clinical skills 1,[32][33][34][35] . The use of problems as a vehicle for developing problem solving skills is one of the key characteristics of PBL, while the important aspects of physical diagnostics education involve the development of students' clinical diagnostic reasoning skills and ability to solve clinical problems, which represent the advantages of PBL. Hence, the superiority of PBL relative to the traditional teaching method could be more evident when considering the clinical skills.
Obviously, there is a high heterogeneity among the included studies. In the stratified analysis by school system, significant heterogeneity also exists, which might have resulted from the following factors. The definition and implementation of PBL could vary widely among medical schools and educators 36 . The educational quality of different medical schools in China could be very uneven, and the medical students from different schools could have different learning background. There are still no unified criteria for the evaluation of the effectiveness of PBL on knowledge and skills. In addition, many other factors that are difficult to measure and control could also affect the success of PBL 37 . Therefore, the random-effects model was applied in this meta-analysis. The high levels of heterogeneity in the effect sizes indicate that more than one population is generating the effect sizes and that the present mean effect sizes are not broadly generalizable. Further research that focuses on moderator variables is required to reduce and resolve the heterogeneity.
It is generally accepted that randomized controlled trials provide the highest level of evidence for the effectiveness of an intervention. However, it is not always possible to use a very rigorous randomized controlled trial approach for educational research. Although most of the studies included in this meta-analysis were designed as randomized controlled trials, none of them described the allocation concealment or blinding method. Overall, the methodological quality of the included studies is not high. In the analysis of knowledge scores (Fig. 3), two of the effect sizes are very high and the mean effect size is large. Similarly, five of the effect sizes are high in the analysis of skill scores (Fig. 4), and the mean effect size is also quite large. Although statistically significant results were also obtained after removing those corresponding studies (data not shown), moderating variable analyses were not performed due to limited studies. Thus, the results of current meta-analysis need to be interpreted with caution. In addition, some of the included studies also used questionnaire surveys as an additional measurement to assess the effectiveness of PBL in physical diagnostics education. However surveys are limited in reliability and validity, subjective and prone to rater biases, and we failed to perform the meta-analysis of those outcomes due to inadequate data.
In summary, the present meta-analysis shows that PBL in physical diagnostics education in China appears to be more effective than traditional teaching method in improving knowledge and skills. Based on the limitations of this meta-analysis, we believe that further well-designed studies on this topic are needed.  Study selection. Included studies met the following criteria: (1) target population: medical students in Chinese medical schools; (2) study design: controlled trials in physical diagnostics education; (3) interventions: PBL teaching in the experimental group and traditional teaching in the control group; (4) outcome measurements: students' knowledge scores and/or skill scores of physical diagnostics. In addition, we excluded studies with insufficient data for calculating effect sizes.

Methods
Data extraction and quality assessment. Two reviewers independently identified trials and reviewed the titles, abstracts and citations (Jianmiao Wang and Jianping Zhao). Two reviewers independently assessed studies for inclusion based on the criteria for population, intervention, study design and outcome measurements (Jungang Xie and Xiansheng Liu). Data were also independently extracted by two reviewers (Yongjian Xu and Weining Xiong). From each study, the following information was extracted: first author's surname, publication year, course name, sample size, student characteristics, intervention method, duration of study and outcomes. We also contacted study authors for missing data. The methodological quality of each study was evaluated with the risk of bias table according to the Cochrane Collaboration.
Statistical analysis. Standardized mean difference (SMD) for continuous outcomes, with 95% confidence interval (CI), was calculated for each study. Studies were then pooled together using SMD as appropriate. The Z-test was used for overall effect and two-sided P < 0.05 was considered as statistically significant. The Q-statistic was calculated to examine result heterogeneity among studies, and P < 0.10 was considered significant. The fixed-effects model was used when the effects were assumed to be homogenous; otherwise, the random-effects model was applied. The I 2 statistic was also calculated to efficiently test for the heterogeneity, with I 2 < 25%, 25-75% and > 75% to represent low, moderate and high degree of inconsistency, respectively 38 . Publication bias was examined in funnel plots and tested with Egger's weighted regression method 39 . The meta-analysis was performed using Review Manager 5.0.23 (Cochrane Library Software, Oxford, UK) and STATA 12.0 (STATA Corporation, College Station, Texas, USA).