Introduction

Latin America is home to over 10% of the world’s population but accounts for only 5% of the global scientific output (Holmgren and Schnitzer, 2004; Leon-De La et al., 2018; Zacca-Gonzalez et al., 2014; UNESCO, 2015), as measured by quantity of research publications and patent awards. A key indicator of performance in science, technology, and innovation (STI), these output metrics suggest significant underrepresentation of the region on the global stage.

Even outside of Latin America, Latinxs tend to be underrepresented in scientific careers. In the United States, while Latinxs comprise over 17% of the country’s population, they receive under 6% of graduate degrees in science, technology, engineering, and mathematics (STEM) fields (Taningco et al., 2008). Importantly, while Latinxs are among the fastest growing demographics in the developed world, their participation in higher education and science is not projected to keep pace with this growth (Chapa and De La Rosa, 2006). Several intervention strategies have been deployed in an attempt to minimize these gaps, but disproportionately low Latinx representation in STEM remains a conundrum for policymakers and science educators.

Competitive STI performance is a critical determinant of sustainable economic development. Within Latin America, Bolivia represents an interesting case for examining individual STI engagement. Bolivia’s 36 indigenous groups comprise over 70% of the total population, making this relatively small territory the most diverse country in Latin America (Montenegro and Stephens, 2006; Kohl and Bresnahan, 2010). Moreover, three Bolivian cities are projected to be the fastest-growing Latin American economies by 2030, as measured by middle-income population growth, with a fifteen-fold hike expected in the city of Santa Cruz (EIU, 2017). However, the country accounts for only 0.22% of Latin American STI research outputs and has negligible rates of patent awards (UNESCO, 2015; Van Noorden, 2014). Despite a decade of political stability and government education expenditures exceeding 8% of GDP—highest in the region and far above any developed country (Arauco et al., 2014; Dobrota et al., 2015; Esteves and Feldmann, 2016)—Bolivia consistently places last in the region according to World Intellectual Property Organization (WIPO) (Acs et al., 2017; Crespo and Crespo, 2016). The country does not implement standardized education metrics further obfuscating causal analyses and international oversight (Zhang et al., 2015) and raising a need for alternative methods of analysis.

Several efforts have been carried out in Bolivia to increase student retention. K-12 education in Bolivia is mandatory for all Bolivian citizens, and the public system is free for all students. Since 2006, the Bolivian government has implemented a nation-wide conditional cash transfer program, locally known as ‘Bono Juancito Pinto', to incentivize continuing education (Yanez et al., 2018). Public university enrollment requires only a nominal student fee, and its funding is guaranteed as a percentage of revenue from nationalized oil and gas sales. Public universities have complete administrative autonomy from the central government. Yet, despite secured funding and freedom to appoint their governing body and faculty, Bolivian universities perform poorly in world rankings and experience low student retention rates (Ferreyra et al., 2017).

Bolivia illustrates a central challenge in science education in developing countries, where resource limitations and outsize political influence in curriculum development often compromise schooling priorities in three distinct ways. First, resource limitations beget a bias toward curricula with direct practicality in the local context over curricula reflecting cutting-edge findings in developed nations (Eilks and Hofstein; 2014, Knamiller, 1984). Second, lessons favored by particular political attitudes may receive priority over more foundational subject matter. The 13-pronged national development directive calls for an emphasis on ‘wisdom of the ancients’, mentioning ‘science and technology’ only in a call for national sovereignty. Moreover, the 2010 education reform ‘Avelino Siñani—Elizardo Perez’ highlighted conceptual elements of decolonization, community education, and ‘productive education’, disregarding teaching methodology and training of teachers (Salinas et al., 2015; Paco, 2013). Lastly, pressures to cover particular topics, and quickly progress to the next, often result in superficial student understanding. In other words, science education policies largely focus on what to teach, but neglect how to teach (Rogan and Grayson, 2003). Thus, a typical Bolivian student is incentivized toward simple repetition of facts, but discouraged from asking questions or engaging in meaningful discussions with faculty (Salinas et al., 2015; Punch, 2004; Osuna, 2013).

Participatory learning traces its roots back to the 20th century, when John Dewey proposed that science teaching must follow an investigational approach, valuing students’ active engagement (Dewey, 1910). Project-based teaching has been shown to be particularly effective at transmitting scientific concepts (Bradforth et al., 2015; Waldrop, 2015), especially to students from underrepresented communities (Stark and Freishtat, 2014; Hrabowski, 2011). In this approach, students are given complex tasks to be resolved over a defined period of time and are expected to conclude with a finalized product or presentation (Barron et al., 1998; Serpa et al., 2018). Such ‘hands-on' approaches would be particularly adapted to the needs of students in developing nations, yet their implementation has been limited (Sife et al., 2007). The majority of studies on the impact of project-based teaching in underrepresented communities have been performed in the developed world (Hrabowski, 2011; Kanter and Konstantopoulos, 2010), with comparatively few studies carried out in Latin America.

In Brazil, for instance, high schools reported increased students’ motivation and critical thinking skills when actively participating in projects framed within the context of local problems (Alves et al., 2012), as well as during science-related activities in an informal setting (De Oliveira et al., 2012). In Mexico, inclusion of forest and water management education in the school curriculum of an indigenous community resulted in a strong feeling of empowerment among locals (Ruiz-Mallen et al., 2009). Similarly, the introduction of a project-based computer science education in Chile resulted in increased motivation of college students (Muñoz et al., 2015). However, these studies focused on small uniform populations with regards to age, gender, socioeconomic status, and education level of the participants. Therefore, studies examining diverse student groups are needed to understand the full potential of project-based education in Latin America.

Participatory approaches can directly influence local culture and policy. Since 2001, the Colombian Administrative Department of Science, Technology and Innovation has executed Ondas, a nation-wide participatory science education program for primary school students from impoverished areas (Manjarrés, 2007). Together with other initiatives, this program increased community involvement and appreciation for science and innovation (Betancur, 2011; Manjarrés, 2007). As result, in 2011, the Colombian government allocated 10% of the country’s royalties from renewable natural resources to STI development. These policies had a major impact in Colombia and, particularly, in the city of Medellin, propelling it from the world’s most dangerous city in the 1990s, to the most innovative city in 2013 (Brand, 2013). In Bolivia, however, significantly less work has been done in participatory education. Our group has previously led sexual violence education initiatives in Bolivian schools, eventually prompting a restructuring of the police and inspiring the enactment of Law 348 (Mostajo et al., 2012), considered one of the most progressive regulations in South America (Anthony, 2018).

Here, we sought to determine whether a grassroots approach in participatory learning, termed Clubes de Ciencia Bolivia (CdeCBo), could impact individual students’ motivation toward STEM careers. We used an experimental, pragmatic method to assess comprehension of complex topics transmitted to students using participatory learning as a paradigm of science teaching. By exploiting national and social media outlets, we were able to reach over 50% of Bolivian youth with internet access to obtain over 900 student applicants. We found that, despite high baseline motivation and confidence, as well as vastly positive self-perception of education quality, students performed poorly when tested on basic knowledge in biology (N = 109). We then show that upon completion of hands-on, hybrid lecture and project-based workshops with exposure to the most recent scientific developments, students demonstrated rapid learning of new knowledge, achieving high scores in confidence and enthusiasm in relation to their own prospects in pursuit of STEM careers. Our results indicate that hands-on workshops, fostering personalized collaboration between highly-trained instructors and local students, are an effective means of driving STEM motivation across a diverse youth population, and suggest that such platforms may prove fruitful in a broad range of developing economies.

Methods

Survey design

A survey was designed to interrogate the experience and perceptions in science education of Bolivian high school and college students nationwide. The survey contained 5 sections. (1) basic demographic information, such as department of residence, grade in which they were enrolled, and type of institution attended. (2) Personal information, such as education of relatives and after school work. (3) Perceived education quality and confidence in future success in STEM disciplines. (4) Essay questions addressing their scientific curiosity and interest in science. (5) Previous academic record in four STEM subjects: biology, chemistry, physics, and mathematics. High achievers were defined as those with scores equal to or above 80%, and low achievers as those below 80%. Some participants had not taken a course on one or more of these subjects and were thus indicated as ‘no previous course’. Survey participants answered all questions online (www.clubesdecienciabolivia.com). The survey was not anonymous, however the CdeCBo admissions committee only had access to basic demographic and essay questions (See Selection of Students). Participants were made aware of this and had the option not to respond to questions not evaluated by the admissions committee. A complete list of questions can be accessed in Table 1 and S1.

Table 1 Application essay questions
Full size table

Course design

Each course was designed by a scientist based in the United States or Europe in collaboration with a Bolivian investigator. Both instructors taught the course together in Santa Cruz. Each course consisted of 10 h of discussion-based lectures and 20 h of project-based laboratory work over the course of 5 days. After the instructional section of the course, students were required to present course-relevant projects, designed by the instructors, such as building homemade microscopes, surgically creating a “cyborg” cockroach, and CRISPR-Cas9 genome engineering in bacteria.

The six Life Sciences courses analyzed in this study were: Model organisms (17 students); Microscopy (17 students); Neuroscience (21 students); Genome engineering (20 students); Microbiology (17 students); Regenerative Biology (17 students). A description and a syllabus for each course are available in Table 2 and S2.

Table 2 CdeCBo course summaries
Full size table

Beyond medical schools, very few universities in Bolivia offer undergraduate degrees in life sciences. Universidad Catolica de Bolivia (UCB) has recently launched an undergraduate degree in Biotechnology. Three UCB courses have curricular overlap with CdeCBo courses: Molecular and Cellular Biology overlaps with CdeCBo’s Model organisms course; Microbial Genetics overlaps with CdeCBo’s Genome engineering course, and General Microbiology overlaps with CdeCBo’s Microbiology course. The curricula of all courses above are included in Tables S2 and S3.

Fundraising strategies

A major issue in developing countries is the lack of laboratory equipment at universities and research institutions (Dolgin, 2018). Yet, several options exist in the United States to help scientists in the developing world, including universities’ equipment recycling programs and non-profit organizations that administer equipment donations (Dolgin, 2018). We took advantage of these programs and obtained several pieces of equipment and laboratory supplies from Harvard University. In addition, we received educational kits as a donation from The Odin.

While PhD students in the United States and Europe often have teaching assistantship requirements, they are rarely given the opportunity to design and execute their own course curricula. Importantly, Ph.D. training grants in the United States include funds to be used towards external training of students, including attendance of conferences and courses, which can be used to train students in teaching. We partnered with departments at Harvard University and Johns Hopkins University to support the costs associated with courses led by their graduate students, Model Organisms (D.G.O.) and Neuroscience (G.A.C.), respectively.

In Bolivia, our fundraising approach was different. Bolivian regulations require private companies to donate a percentage of their earnings to nonprofit organizations. These donations can be made in cash or services, i.e., in-kind donations. As a registered foundation, we received several donations. In-kind donations included flights (Boliviana de Aviacion and U.S. Embassy in Bolivia), lodging, event planning services (Ximena Ximenez Eventos), and laboratory supplies (U.S. Embassy in Bolivia). Laboratory and classroom spaces were provided free-of-charge by Universidad Privada de Santa Cruz de la Sierra (UPSA). Cash donations were used to purchase additional lab equipment and offset organizational costs.

A complete list of all equipment and supplies, as well as their sources, is available in Table S4.

Instructor selection and training

Foreign instructors were recruited by contacting Ph.D. programs in the United States and Europe; through our professional network; our Facebook page ‘Clubes de Ciencia Bolivia'; and international scientific conference presentations. Foreign instructors were either Ph.D. candidates or postdoctoral fellows. In the United States, graduate students become Ph.D. candidates after passing their qualifying exam. The equivalent qualification in Europe is completion of a Masters program and enrollment in a Ph.D. program. All foreign instructors were required to have had at least one semester of college-level teaching experience prior to teaching in Bolivia. Additionally, respective Ph.D. program directors or principal investigators were contacted to request letters of recommendation. Local instructors were recruited by word of mouth. All local instructors had completed at least one year of graduate level education and had at least one semester of teaching experience. All instructors were tested using the Science Teaching Efficacy Belief Instrument (STEBI) (Riggs and Enochs, 1990).

Foreign and local instructors were paired based on research experience and interest. Together, they developed courses based on their area of expertise, complementing each other’s strengths. Communication between instructors was done on their own time via video calls and emails. The organizers then approved courses after discussing the content in detail. Resources on teaching strategies were made available to all instructors.

Prior to the start of the course, we held a mandatory team retreat in Bolivia for all instructors (Video 1). We invited education experts to train instructors on project-based learning. During the retreat, the instructors were briefed on the incoming student body. They were then trained on the following topics: science teaching and participatory learning, team development, the Bloom’s taxonomy of questions, and challenges of science education in Bolivia. Instructors had workshop sessions to develop their testing methods and compare them to other courses. Finally, role-playing exercises were used to simulate difficult scenarios in the classroom and teaching laboratory.

Student recruitment

We used three different channels to recruit students: press, social media, and visits to schools and universities. We opened applications on 6 August 2017, Bolivian Independence Day, a major holiday and one of the top newspaper circulation days. We coordinated a simultaneous press release in the top three newspapers in the country: El Deber (Santa Cruz); Los Tiempos (Cochabamba) and Página Siete (La Paz). Each newspaper wrote, free of charge, a two-page article describing the program and inviting students to apply. El Deber featured this article on the cover. In addition, we coordinated appearances on several TV shows. Moreover, we orchestrated publication of interviews with instructors in varied press media throughout the application period ending 15 October 2017.

On social media, we created a Facebook page named ‘Clubes de Ciencia Bolivia’, which at the start of the application period had over 33,000 followers. Weekly page viewing statistics show that our online presence reached over 45% of 18–24-year-old Bolivians with Internet access, suggesting that this approach was optimal to target a large audience. Working with local filmmakers, we created videos for recruitment, featuring instructors introducing their course, as well as multimedia presentations illustrating the potential for positive impact from scientific engagement in national development (Video 2).

To recruit additional students, particularly in rural areas, we organized on-site visits to schools across the country. Our program had had two previous editions and over 300 alumni, several of which volunteer with the organization. These volunteers are based in six of nine departments in Bolivia. Volunteers were trained online and given a standard presentation. They then visited schools and taught potential students how to apply.

To facilitate the recruitment of students, we incorporated an application platform online, which could be accessed via cell phone or computer. Cell phone penetrance in Bolivia is high. While no official numbers are available, a study in hospitals of the cities of La Paz and El Alto found that over 96% of patients ages 18–29 owned a cell phone (Kamis et al., 2015). Recent efforts from the Bolivian government have delivered cell phone service in rural areas of the country. Moreover, Quipus, a government-subsidized company has provided computers to schools all around Bolivia. Altogether, we believe that the combination of press coverage, social media presence, school visits, and an accessible application platform ensured the targeting of a diverse student pool.

Selection of students

For this study, 113 students were selected from a pool of 903 eligible applicants. Only 109 students are part of the study, as 4 did not attend all sessions of their respective course. The gradable component of our online application form focused on essay questions rather than quantitative aspects (i.e., school grades). A complete list of all essay questions is available on Table 1.

Each application was reviewed and ranked by one member of the admissions committee. The admissions committee was blind to previous academic performance and emphasized representation of all corners of the country. The members of the admission committee were instructed to take a holistic approach to evaluate each applicant, taking into account their environment and potential. When the member felt unsure about a particular application, a second or third member was asked to evaluate the application. The members of the admissions committee included the organizing team, instructors, and education professionals.

Students were required to rank their top 3 courses from a total of 16 available courses. These included the 6 courses in biology-related fields analyzed for this study, as well as courses in computer science, engineering, entrepreneurship, and social sciences. Students were considered part of the ‘Initial Interest’ group if they were assigned to one of the 3 courses they selected. Students were part of the ‘No Initial Interest’ group if they were assigned to a course different from their 3 selections. In total, 32 (29.36%) students were in the ‘No Initial Interest’ group, while 77 (70.64%) students were in the ‘Initial Interest’ group. Instructors were blind to which students were part of each group, although they were aware of the experiment. In order to allow for student preparation, the students were notified of their final course assignment at least 14 days prior to the start of the program.

Student consent

The Clubes de Ciencia Bolivia Review Committee reviewed and approved the design and execution of this study. In accordance with Bolivian regulations, students were informed about the study prior to the application process. In order to participate, students were required to consent to make pre- and post-test results available for anonymized publication. Upon admission, students had to submit two documents: a copy of an identification document proving their Bolivian citizenship and a letter from their institution proving that they were registered students and were in the year they indicated in the application. In the case of students under the age of 18, legal age in Bolivia, parents had to consent and send a signed and notarized authorization. A template letter was provided in the admission package.

Testing

In each course, the instructors administered a five-question test focused on the material to be covered in class lectures. Students were tested at the beginning and end of the course. The pre-tests were not returned to the students before the post-test, nor did they have access to the answer key. Students were unaware that the pre- and post-tests were identical ahead of time. Each test was graded by each of the two instructors of the course. The average of both grades was then computed for analysis. A complete list of test questions is available in Table 3.

Table 3 Questions tested in each course
Full size table

Selection and testing of the reference group

As mentioned above, three courses exist in Bolivia that have a curricular overlap with some courses of CdeCBo (Tables S2 and S3). The Biotechnology degree at UCB was established in 2018 and currently the oldest students are in their second year of university (ages 19-20). Microbial Genetics and Molecular and Cellular Biology are upper level courses, so no student has taken those courses yet. General Microbiology is a first year-level course and 14 students have completed it. 11 agreed to participate in our study. Of these 11, 2 students had previously completed CdeCBo courses although not the CdeCBo Microbiology course. In order to remove any potential bias, we grouped both former CdeCBo alumni separately from the other 9. All 11 students were administered the same test as CdeCBo’s Microbiology students. Each test was graded by the same instructors of the CdeCBo Microbiology course. The average of both grades was computed.

Statistics

For analyses that considered two groups of students, two-tailed paired or unpaired t-tests were performed using PRISM version 7.0, depending on whether the groups being compared contained the same set of students or different students. Gaussian distributions were assumed. Significance was set at a threshold of p < 0.05.

For analyses that considered three or more groups of students, we performed one-way ANOVA with Tukey’s correction using PRISM version 7.0. Gaussian distributions were assumed. Significance was set at a threshold of p < 0.05.

To compute the correlation between educational level, type of institution attended, previous academic achievement, or gender and confidence in success in science, we performed a multinomial logistic regression.

Finally, correlation analyses between two parameters were performed by computing Pearson correlation coefficients (Gaussian distributions were assumed) with PRISM 7.0.

Results

Bolivian students rate their scientific education highly

To gain insight into the lack of relative representation of Bolivia in STEM fields, we conducted a country-wide online survey targeting precollege (11th and 12th grades) and college students in the context of an application to our science outreach program (Methods). Respondents included students from eight of nine Departments in Bolivia, aged 15–22. A total of 903 students responded to the survey, which included questions designed to gauge participants’ perceptions of their educational experience, interest in science, and career goals. The sample group comprised 48.7% males and 51.3% females, equally spread across public (53.5%) and private (46.5%) institutions. Moreover, 30.0% of respondents were enrolled in high school, while 70.0% were in college (Tables S1 and S5).

Surprisingly, respondents both at the pre-college and college levels rated their educational experience highly (7.75 ± 0.08 precollege, n = 268 and 7.59 ± 0.05 college n = 614, on a 1–10 scale; p = 0.0736) (Fig. 1a). While this observation was independent of gender (7.59 ± 0.06 n = 430 males and 7.69 ± 0.06 n = 451 females, p = 0.2433), respondents attending private institutions reported a slightly higher satisfaction level with the quality of their education (7.52 ± 0.06 public, n = 467 and 7.76 ± 0.06 private n = 406, p = 0.0067). Overall, our survey results indicate that Bolivians perceive their educational experience as positive, regardless of education level, gender, or type of institution attended (Fig. 1a).

Fig. 1
figure 1

Perceptions, confidence, and aspirations of Bolivian youth in science. a Bolivian students rank their education highly. Bars represent mean ± SD. b Highest educational level desired. c Confidence in good performance in future STEM subject tests does not depend on educational level or type of institution attended. d Confidence in good performance in future STEM subject tests is affected by previous achievement in Biology. e Confidence in good performance in future STEM subject tests is affected by gender. f Correlation between confidence in future performance in STEM subject tests and perceived education quality. Each dot represents the response of a group of students. Unpaired t-test across groups in a. Multinomial logistic regression when comparing more than two groups in (c–e). *p < 0.05; **p < 0.01; ***p < 0.001. N = 903

Full size image

We then asked respondents to indicate the highest level of education they aspire to achieve, as well as their understanding of requirements to pursue a career in science. The vast majority (91.8%) of respondents reported to understand the required training to pursue a career in science (Figure S1A) and hoped to obtain a graduate level of education (77.6% Doctorate and 15.7% Masters, compared with only 5.2% Bachelors and 1.6% Associate level of education) (Fig. 1b). These numbers are in great disagreement with current data showing that Bolivia has a vanishingly low number of researchers (<1 Ph.D. level scientist per 1000 workers) (UNESCO, 2015, Van Noorden, 2014).

To understand if our sample population was biased towards high academic achievers, we requested that respondents report their academic grade in their previous biology courses. Of the 876 respondents who opted to report their grades, 23.3% had never taken a biology course, whereas 25.6% had but achieved a low grade (<80%) and 51.1% obtained a high grade (>80%) (Figure S1B). Similar numbers were obtained when inquiring about courses in Chemistry, Physics, and Mathematics (Figure S1B). Of note, no strong correlation (r < 0.5) between academic achievement in any of these subjects and perceived education quality was found (Figure S1C).

Bolivian students have confidence in their scientific aptitude

To gain further insight into students’ perceptions and attitudes towards STEM careers, we asked respondents how confident they were in achieving a high score in a STEM subject test. We found the majority of respondents to be confident that they will perform well in STEM courses (Figs. 1c–e). Notably, no clear distinctions were observed in confidence levels when participants were grouped by education level or type of institution attended (Fig. 1c). Nevertheless, confident participants were more likely to be high academic achievers compared to those with low academic achievement or lacking previous biology course experience, and neutral respondents were the least likely to be high achievers (Fig. 1d). Male participants reported higher levels of confidence than their female counterparts (Fig. 1e). Finally, we found no clear correlation (r = 0.1942) between confidence in future STEM subject tests and perception of education quality (Fig. 1f). Together, our results show that diverse Bolivian student populations perceive their education as being of high quality, aspire to obtain advanced science degrees, and display high confidence in their ability to succeed in mastering STEM subjects.

Life sciences workshops effectively transmitted new concepts

Given the surprising results obtained in this initial nation-wide survey, we decided to test the knowledge of a subset of respondents admitted into CdeCBo in six life sciences topics, namely model organisms, microscopy, neuroscience, genome engineering, microbiology, and regenerative biology. Each course was designed and taught by two instructors: one based in a foreign country (United States or Europe) and one local instructor. All 12 instructors were tested using STEBI (Riggs and Enochs, 1990). In the Personal Science Teaching Efficacy Belief scale the instructors obtained 57 ± 1.25 out of 65 points (Fig. 2a, Table S6), while in the Science Teaching Outcome Expectancy scale, instructors obtained 46.33 ± 1.61 out of 60 points (Fig. 2b, Table S6), consistent with previous reported scores of teaching professionals (Gencer and Cakiroglu, 2007; Riggs and Enochs, 1990). Importantly, no significant differences were observed between foreign and local instructors in either of the scales (Fig. 2, Table S6).

Fig. 2
figure 2

Results of the Science Teaching Outcome Expectancy scale instrument (STEBI) applied to all instructors. a Results of the Personal Science Teaching Efficacy Belief scale. b Results of the Science Teaching Outcome Expectancy scale. Bars represent mean ± SD. Foreign Instructors n = 6, Local Instructors n = 6. Unpaired t-test between foreign and local instructors

Full size image

During the student selection process, the committee strived to ensure that the admitted student body reflected Bolivia’s demographics (Fig. 3a). Overall, eight of nine departments in Bolivia were represented. Due to possible migration within Bolivia, we classified the students based on their department of residence, not department of birth. Nevertheless, only 16 of 109 (14.68%) students were residing in a department different from their department of birth. One department, Pando, was not represented in the application pool and therefore could not be included in the study. In addition, three departments, Potosi, Tarija, and Chuquisaca, were underrepresented in the applicant pool and thus in the student cohort (Fig. 3a). The admitted student pool displayed an equal representation of gender, type of institution in which the students were enrolled, and a balanced distribution of age (Fig. 3b–d). Because the application was open to students from 11th grade in high school to third year of college at the time of application, the student cohort was enriched for college students over high school students (Fig. 3e). As the experiment was performed during summer vacation, some students had graduated high school but had not yet started college. Together with the high school students, we categorized this group as ‘precollege’. Moreover, the committee gave preference to applicants who had not attended our program in past years (77 of 109 participants), diminishing the possibility of any advantage gained from previous exposure to our approach that might confound study data (Fig. 3f). As two of six courses had been taught in previous years, the returning students were not allowed to attend the same course twice. Altogether, to the best of our possibilities, our study analyzed a student cohort representative of the diverse demographics of Bolivia.

Fig. 3
figure 3

Demographic distribution of the student cohort. a The student body represents the population of Bolivia. Left: Map of Bolivia showing the number of students from each department our cohort, as well as their distribution (%). Right: Population distribution (%) in Bolivia according to the most recent census (2012). b–f Other demographic parameters: b Gender, c Age, d Type of institution in which the students were enrolled, e Education level of the students, f Previous participation in non-competitive science outreach programs. N = 109

Full size image

Each course contained a theoretical component, taught using a discussion-based lecture approach (Fig. S2A, Video 1), as well as a project-based laboratory component (Figure S2B, Video 1) (Han et al. 2015, Newman et al. 2013). At the end of the courses, all students were required to present on a topic within the scope of their course that they researched in small groups of 3–4 students during the week (Figure S3, Video 1). We designed questions to assess comprehension of basic principles behind biological processes, such as heredity, immune rejection, or neural communication (Table 3). As we aimed to understand whether students had a basic grasp of these topics, we designed questions targeting knowledge and comprehension, following the principles detailed in the Bloom’s taxonomy of questions (Lord and Baviskar, 2007). We found that incoming students performed poorly in our tests (38.24% ± 20.42, N = 109) (Fig. 4a), revealing a stark disconnect between perceived education quality and aptitude in STEM subjects.

Fig. 4
figure 4

Effective transmission of key concepts in biology. a Overall student performance before and after courses. Left: Combined student performance. Right: Paired test results for each student. N = 109. b–g Student performance for each individual course. b Model organisms and disease modeling. N = 17. c Microscopy. N = 17. d Neuroscience. N = 21. e Genome Engineering. N = 20. f Microbiology. N = 17. g Regenerative Biology. N = 17. Paired t-test; ****p < 0.0001. h Comparison of student performance between the CdeCBo Microbiology course and a semester-long Introduction to Microbiology course taught to first year university students. CdeCBo Microbiology students n = 17, Reference students without CdeCBo experience n = 9, with previous CdeCBo experience in a course not related to Microbiology n = 2. One-way ANOVA p < 0.0001. Tukey's Multiple Comparison Test ***p < 0.001. Bars represent mean ± SD

Full size image

To interrogate the effectiveness of our approach in teaching life sciences, we analyzed test scores obtained after course completion. Overall, post-course test scores were significantly higher (N = 109, p < 0.0001) than those obtained at the beginning of the program (Fig. 4a). In fact, 108 of 109 students increased their test score after participating in the program (Fig. 4a). This pattern was true for every individual course analyzed (Figs. 4b–g). Genome Engineering (Fig. 4e, n = 20) had the lowest pre-test scores (20.12 ± 11.52, ranging from 0 to 48), as well as the lowest post-test scores (56.52 ± 13.04, ranging from 34 to 78), whereas Microscopy (Fig. 4c, n = 17) and Microbiology (Fig. 4f, n = 17) had the highest pre-test scores (50.96 ± 23.12, ranging from 16 to 89.33, and 50.60 ± 14.36, ranging from 20 to 80, respectively) and the highest post-test scores (89.94 ± 8.52, ranging from 74 to 100, and 92.96 ± 12.12, ranging from 60 to 100, respectively). From the course with the lowest post-test scores, Genome Engineering (Fig. 4e, n = 20, p < 0.0001) to the one with the highest post-test scores, Microbiology (Fig. 4f, n = 17, p < 0.0001), all courses yielded significant gains in test scores (Fig. 4, Table 4 and S7), demonstrating that the efficiency of the participatory approach extends beyond a particular topic or instructor.

Table 4 Descriptive statistics for students’ test scores
Full size table

To understand whether the increase of scores was linked specifically to the participatory nature of the courses, we tested college students who have had exposure to similar curricula, but taught using standard Bolivian practices. We administered the same test to 11 second-year college students who have taken the semester-long UCB Microbiology course. Of those students, 9 have never been part of our program, while 2 students are CdeCBo alumni, albeit in topics different from Microbiology. UCB Microbiology students scored significantly lower than CdeCBo students who completed the Microbiology course (Completed university Microbiology students 34.44 ± 10.94% n = 9, Completed university Microbiology students with other CdeCBo experience 25.00 ± 5.00% n = 2, Completed CdeCBo Microbiology course 92.94 ± 2.94% n = 17, p < 0.0001, Fig. 4h, Table S7). Surprisingly, college-level Microbiology students scored no better than CdeCBo students before the courses (p > 0.05, Fig. 3h, Table S7). These observations suggest that a participatory learning approach is more effective at transmitting complex biological concepts than traditional Bolivian practices. Moreover, we find that students who previously attended CdeCBo do not necessarily perform well when tested on topics different from their previous CdeCBo exposure, indicating that the teaching method, and not a bias in student selection, is the main driver of the observed test performance increases.

Possible confounding factors did not influence students’ performance

Next, we investigated whether our results were confounded by external factors. We found no influence of gender or age on test scores obtained either before or after taking a course (Fig. 5a, b). Students coming from public institutions scored slightly better than those enrolled in private institutions on pre-course tests (p < 0.05), yet this difference did not persist in the post-course test scores (Fig. 5c). In Bolivia, public universities tend to rank higher than their private counterparts (Villarroel-Mareno et al. 2012). Therefore, the statistically significant difference in scores between students enrolled in public versus private institutions is likely to have been driven by college students, who outnumbered pre-college students (Fig. 3e).

Fig. 5
figure 5

Confounding factors and student performance. a Gender. Female n = 61, Male n = 48. b Age. Teenagers n = 68, 20+ years old n = 41. c Type of institution in which students were enrolled. Private n = 42, Public n = 67. d Education level of the students. Precollege n = 27, College n = 82. e Previous experience in non-competitive outreach programs. First time in an outreach program n = 77, Returning student n = 32. f Department of residence. Santa Cruz n = 48. Elsewhere n = 61. g After school work. Doesn’t work n = 88; Works n = 8. h Degree wanted. Bachelors n = 5; Masters n = 8; Doctorate n = 82. i English skill level. Basic n = 15; Intermediate n = 46; Advanced n = 34. j Family education. No siblings with a college degree n = 41; At least one sibling with a college degree n = 55; No parent with a college degree n = 19; At least one parent with a college degree n = 77. k Perceived education quality. Satisfied n = 53; Not satisfied n = 43. l Academic success in biology. High achiever n = 65; Low achievement = 11; No previous course n = 20. m Confidence in science performance. Insecure n = 6; Neutral n = 9; Confident n = 81. n Exposure to STEM professionals. Little exposure n = 8; Some exposure n = 47. A lot of exposure n = 41. o Type of learner. Visual learner n = 68; Verbal learner n = 28. Bars represent mean ± SD. Paired t-test within group, unpaired t-test across groups; *p < 0.05; **p < 0.01; ***p < 0.001. C.D. college degree

Full size image

As expected, college students performed better than precollege students in pre-course tests (p < 0.01). No difference was found in post-course test scores, indicating that education level did not impact the extent of learning (Fig. 5d). This observation is in agreement with previously published data showing that students from different backgrounds can achieve the same learning gains upon engagement in project-based biology courses (Newman et al., 2013). Moreover, these data are consistent with the notion that project-based STEM courses preferentially benefit students at an initial disadvantage (Han et al., 2015).

One could argue that previous exposure to similar outreach programs could confer a performance advantage in the tests used in this study. Previous large-scale science outreach programs in Bolivia, such as science olympiads, fundamentally differed in scope from CdeCBo due to their competitive nature, so we accounted only for student participation in previous CdeCBo iterations. Comparison of test scores between first time students and those with previous experience revealed no significant difference in test scores either before or after the course between these two groups (Fig. 5e), suggesting that our approach can be used to educate student populations independently of their previous access to similar opportunities.

Bolivia contains high ethnic diversity, with large disparities in income and access to education between different departments (López-Videla and Machuca, 2014; Nunnenkamp et al., 2007; Punch, 2004). Abundant literature shows strong correlations between socioeconomic status and educational attainment (Caro et al., 2009). Santa Cruz, in particular, is the wealthiest department in Bolivia per capita, and students residing in Santa Cruz are the most represented group in our study (48/109). Nevertheless, our results demonstrate that there is no significant difference in scores either before or after the courses between students residing in Santa Cruz and those coming from other departments (Fig. 5f). However, we did find a modest but statistically significant (p < 0.05, N = 96) decrease in the post-course test scores of students who reported after-school professional employment in comparison with those who did not (Fig. 5g). Work obligations from a young age, prevalent in rural parts of Bolivia, have been shown to result in slower learning speeds, disinterest in academics, and ultimately higher school dropout rates (Punch, 2004). Nonetheless, none of several other possible confounding circumstances impacted test scores after the courses. We found no significant differences after dividing students based on the highest academic degree they desire to acquire, or their English skills, education level of their parents and siblings, perceived education quality, previous academic success in biology classes, confidence in performance in STEM subject tests, previous exposure to STEM professionals, or even preferred mode of acquiring new information (Figs. 5h–o). Altogether, we conclude that our approach was efficient at transmitting complex biological concepts to students independently of their socioeconomic and training background.

Initial interest in the course material does not affect learning

Next, we sought to determine whether the extent of learning depended on previous interest in given course topics. Interest in a topic is well documented to influence motivation and academic achievement in that topic (Harackiewicz and Hulleman, 2010). Conversely, becoming more knowledgeable about a topic increases an individual’s interest in it (Hidi and Renninger, 2006). To investigate the possibility that previous interest in a course altered the extent of learning after participating in that course, we categorized the students based in their expressed interest in the courses., and compared student performance. Students in the ‘No Initial Interest’ group did not perform differently from those enrolled in a course on their list of three top choices (i.e., ‘Initial Interest’ group) across all courses (Fig. 6, Table S7). Students from either group attained significantly higher test scores after taking their respective courses (p < 0.0001), suggesting that our approach was effective even for those students who were not directly interested in the specific subject taught a priori.

Fig. 6
figure 6

Initial interest in the topic does not affect student performance. a Model organisms and disease modeling. Initial interest n = 11; No initial interest n = 6. b Microscopy. Initial interest n = 6; No initial interest n = 11. c Neuroscience. Initial interest n = 18; No initial interest n = 3. d Genome Engineering. Initial interest n = 18; No initial interest n = 2. e Microbiology. Initial interest n = 12; No initial interest n = 5. f Regenerative Biology. Initial interest n = 12; No initial interest n = 5. Bars represent mean ± SD. N = 17–21. Paired t-test; ****p < 0.0001

Full size image

Altogether, the lack of impact of potential confounding factors in student performance strongly suggests that the efficacy we report in participatory learning approaches in life sciences could be broadly applicable in diverse contexts and additional developing countries.

Hands-on workshops positively impact enthusiasm for STEM

Finally, we sought to assess student satisfaction with our program by characterizing differences in their own personal attitudes towards STEM topics. When asked to rate different aspects of the CdeCBo initiative upon its completion, the vast majority rated their experience very highly (4 or 5, in a 1–5 scale, with 5 being most satisfied), with regards to having both a foreign and a local instructor teaching together, course activities and materials, motivation of the instructors, clarity of course objectives, and connection between theory and practice (Fig. 7a–f, Table S8). These are noteworthy findings when considering the stark contrast between teaching methods utilized herein, and those traditionally applied across Bolivian school systems, which alternatively could have caused attrition and dissatisfaction among students in unfamiliar learning environments. Furthermore, the students also reported high levels of excitement for science at the end of the program. When asked how strongly they agreed that science is exciting, enjoyable to participate in, a possible career choice, or whether they enjoy solving scientific problems and that hard work will help them succeed in science, most students responded 5 in a 1-5 scale, with 5 being strongest agreement (Figs. 8a–e, Table S8). Most students stated they would like to participate in similar programs in the future want to keep learning more about science, definitely want to become scientists, and positively rate their prospects for success in science. More importantly, 89 of 91 students stated that our program increased their interest in science (Fig. 8f–j, Table S8).

Fig. 7
figure 7

Students’ evaluation of our initiative. a Having a foreign or a local instructor. n = 109. b Course activities. N = 90. c Course materials and equipment. N = 91. d Motivation of instructors. N = 91. e Clarity of course objectives. N = 91. f Connection between theory and practice. N = 91. Rating 1–5, with 5 being most satisfied

Full size image
Fig. 8
figure 8

Students’ perceptions, attitudes, and prospects after participating in our initiative. a–e Students gave a rating 1–5, with 5 being the strongest agreement with the statement made. a Science is exciting. N = 92. b I enjoy participating in science projects. N = 92. c I would like to pursue a career in science. N = 91. d I enjoy solving scientific problems. N = 92. e Hard work will help me succeed in science. N = 90. f–h Students replied Yes or No to the posed question. f Did our program increase your interest in science? N = 91. g Would you like to participate in similar programs? N = 92. h Do you want to keep learning more about science? N = 92. i-j Students replied No, Maybe, or Definitely to the posed question. i Do you want to become a scientist? N = 92. j Do you think you can succeed in science? N = 92

Full size image

Discussion

Developing countries, and Bolivia in particular, have long been in the backwaters of research and technological innovation, despite a sizable population and GDP (Arauco et al., 2014; Villarroel-Mareno et al., 2012). It is not trivial to devise and implement measures to position developing countries on par with developed nations, where a rich history of prolific research and development has long been established (Kremer et al., 2013; Serpa et al., 2018; Eilks and Hofstein, 2014; Knamiller, 1984). Notwithstanding, we envision that our efforts to transform the landscape of science education in Bolivia may help reverse this trend.

Implementing education programs and performing research in the developing world has several challenges, even when structural and technological limitations are overcome (Serpa et al., 2018; Kremer et al., 2013). For example, implementation of the One Laptop per Child program in Peru did not significantly increase students’ scores in mathematics (Cristia et al., 2012). Similar results were obtained from the Computers for Education program in Colombia (Barrera-Osorio and Linden, 2009), showing that access to technology alone is not sufficient to increase student performance. Using life sciences courses as examples, we demonstrate that project-based instruction can be successfully implemented in a developing country and can adequately relay important biological concepts to local students of diverse backgrounds in a limited time span.

Studies in project-based learning have shown that critical thinking skills of students exposed to this methodology are significantly increased over time (Sepherd and Cosgrif, 1998). Yet, not all students transition equally towards project-based learning, and students lacking experience with this approach can experience difficulties (Frank et al., 2003). Previous studies have shown that this teaching method strongly favors academic ‘low achievers’ (Rosenfeld and Rosenfeld, 1999). ‘High achievers’, on the other hand, tend to have a difficult transition towards project-based learning, with many of them abandoning the projects altogether (Rosenfeld and Rosenfeld, 1999). Our application process recruited academically high-achieving students from across Bolivia (Table S1). The majority of our students have had little, if any, previous experience in project-based learning. Yet, none abandoned their course and 108 of 109 showed improved learning in the post-tests (Fig. 4). These results not only challenged our expectations, but also support the importance of performing ‘on the ground’ work across the globe. To date, most project-based education studies in underrepresented populations have left behind the developing world (Dochy et al., 2003; Stark and Freishtat, 2014; Hrabowski, 2011). Yet, even comparisons of U.S.-born Latinxs to first generation Latinx immigrants in the United States have consistently shown differences in several behaviors (Alegria et al., 2008; Potochnick and Perreira, 2010), suggesting that underrepresented minorities in the developed world are not necessarily representative of their native populations.

In this study, we chose Bolivia for its high population diversity and striking economic disparities, but we foresee the need to perform similar projects in other regions of Latin America and across the developing world. Indeed, other groups are currently replicating this model in Brazil, Colombia, Mexico, Paraguay, Peru, and Spain. A large, multinational, and collaborative effort will be needed to understand the effectiveness of participatory learning approaches across different cultural and political scenarios.

Investment in education at the early stages produces long-term benefits in decision-making (Luo et al., 2018), but must be accompanied by policies that foster innovative approaches at the level of teacher training. An observational study in rural Tarija, the most Southern region of Bolivia, registered several concerning behaviors in teachers that would be considered unacceptable in most countries: favoritism, lack of interest, and unexcused absence from the classroom (Punch, 2004). In addition to obvious considerations for ethical teaching practices, school curricula must also be made amenable to rigorous international standards to ensure accurate performance evaluation and teacher accountability.

Poor academic performance and low student retention in Bolivia could at least in part be attributed to child labor practices. Bolivian children are instilled with a strong sense of family responsibility, and often enter the labor market at an early age (Punch, 2004). In Bolivia, child labor reduces educational attainment by two years (Psacharopoulos et al., 1997). In 2014, the Bolivian government, disregarding international conventions, passed a law reducing the legal working age to 10 years of age if self-employed and 12 years of age for contract work (Coon, 2016). It has been shown that school dropout rate is positively correlated with a decrease in the minimal legal working age (Montmarquette et al., 2007). In line with this observation, Bolivia is the only country in Latin America where the percentage of students that drop out of university is greater than those of still enrolled students and those who completed their degrees (Ferreyra et al., 2017). Nevertheless, we find encouragement in anecdotal accounts of CdeCBo students who invested considerable personal monetary savings to obtain transportation and lodging accommodations to participate in CdeCBo.

We believe that addressing these intricate issues will require not only domestic changes in Bolivia but also collaboration with scientists abroad working in more established and vibrant research environments. Our initiative is unique in that it allows Ph.D. students and postdoctoral fellows carrying out cutting-edge research in the United States and Europe to collaborate with local scientists in designing project-based courses. Moreover, it allows considerable numbers of students from developing countries to meet and interact with scientists in a supportive, informal, and discovery-oriented environment, fostering the transmission of knowledge and experiences and, ultimately, the creation and nurturing of a local pool of talent. Maintaining small course sizes of 17–21 students, leads, in our experience, to a close and lasting mentor relationship between instructors and students. Together, these characteristics drive the efficacious transmission of scientific knowledge and aid in developing and strengthening scientific endeavors in Bolivia in the medium and long term (Hrabowski, 2011).

Research universities sponsoring such initiatives can reciprocally benefit in tangible ways. Having a graduate student teaching a course abroad allows universities to target two unique pools in recruitment: students in the developed world interested in teaching, and students in the developing world interested in training as scientists in the U.S. and Europe.

The current study has some limitations. First, it is not possible to completely account for selection bias in an initiative with eligibility and selection criteria that enrich our sample for academic high achievers interested in STEM. Nevertheless, categorizing students based on their education perspectives, previous academic achievement, and initial interest in their course yielded no difference in post-course test scores (Figs. 5, 6). Comparing students from different education levels also revealed no difference in post-course test scores (Fig. 5d). Because the format of the courses included lecture and laboratory components (Table S2), it is difficult to disentangle the direct contribution of each component to overall learning. Future studies including simultaneous lecture-only and laboratory-only courses are warranted.

One unexpected limitation we encountered was more social in origin. Following CdeCBo 2017, particular segments of national media and university professors cautioned against the adoption of modern science education techniques and the intellectual challenges these impose. One publication lamented CdeCBo’s forward-thinking approach in relation to the Bolivian ‘national patriotic agenda’:

‘The issues raised may be challenging and the future is always a challenge, but one should not lose sight of the present. In a country like Bolivia, where we have not even solved the basic health problems, we do clubs that require one to learn about the post-genome stages. [In] a stretch in which we are not manufacturers of bicycles, we think about principles of flight and aeronautical design...’ (Martinez and Sanabria, 2017).

The article goes on to note CdeCBo’s lack of emphasis on ‘ancestral knowledge’, and ponders whether CdeCBo’s neuroscience course should be ‘reoriented toward primary education, infant health, to students’ cognitive processes, or to potentiating these to contribute to development of the country’. The following year, following CdeCBo 2018, one newspaper stressed the need to consider ‘national reality’ in contrast to CdeCBo’s perceived vision to ‘practice a science that is academic, technocratic, prize-winner of modernity, not originated in the [2010 Bolivian education law] 070’.

While we fundamentally disagree with the assertion that exposing students to knowledge of modern scientific advancements will negatively impact Bolivia’s future, the point is well taken of a need for regional sensitivity and consideration of local contexts. An illustration of CdeCBo’s regional considerations comes in topics chosen during course design. Strikingly, some of the above authors investigated specific educational desires among Bolivian students in two departments and found that the most highly sought topic was HIV/AIDS (Ramirez-Martinez et al., 2017). Indeed, one of CdeCBo’s most popular courses dealt with genome editing in the context of immunological resistance to HIV (Table S2). Moreover, a subsequent publication from the same group stated that ‘science clubs [Bolivia] could encourage that a positive image of science and its consequent practice are not contradictory, but continuous… This activity will create a good basis for medical practice to combine assistance with solution, research with intervention’. (Martinez, 2018). It is our view that many negative perceptions, among limited segments of population, toward modern scientific education approaches by CdeCBo can be alleviated through enhanced transparency and dialogue between these groups and CdeCBo instructors. We are confident that both camps have in mind Bolivia’s best interests, and that open channels of mutually respectful dialogue will identify the most effective and beneficial means of national development.

Future directions include carrying out longitudinal studies to evaluate our students’ future success in academic applications and their career choices in STEM versus other fields. In particular, it will be fascinating to investigate whether students who were allocated to courses that did not initially interest them will be any less likely to follow STEM careers than those who attended their preferred courses. Even though test scores were similar between these two groups, only a long term follow-up study can shed light on whether their experience overall was similarly inspiring. In either scenario, exposure to biological research has been shown to increase the odds of obtaining internships and scholarships to study biology at the college or Ph.D. level (Besecke and Reilly, 2006; Nauta et al. 1998). Ultimately, we aim to provide an experience that empowers students with the self-confidence and motivation necessary to help them realize that they too can pursue successful careers as scientists and thrive alongside scientists of any other nation.