A scalable goal-setting intervention closes both the gender and ethnic minority achievement gap

Abstract

The gender and ethnicity gap in academic achievement constitutes one of today’s key social problems. The current study, therefore, assessed the effects of a brief, evidence-based online intervention aimed at enhancing goal-directed conceptualization and action among first year college students (N=703) at a large European business school. The academic performance of these students was contrasted with that of three pre-intervention control cohorts (N=896, 825 and 720), with particular attention paid to the role of gender and ethnicity. The intervention boosted academic achievement and increased retention rates, particularly for ethnic minority and male students (who had underperformed in previous years). The gap in performance between men and women, and for ethnic minorities versus nationals, became considerably smaller within the intervention cohort. After Year 1, the gender gap closed by 98%, and the ethnicity gap by 38% (rising to 93% after the second year). All groups in the intervention cohort performed significantly better than control cohorts, but the effect was particularly large for males and ethnic minorities. The increase in performance was largest for ethnic minority males: they earned 44% more credits, and their retention rate increased 54%. Overall, the results indicate that a comprehensive goal-setting intervention implemented early in students’ academic careers can significantly and substantially reduce gender and ethnic minority inequalities in achievement.

Introduction

The inequality in performance and life outcomes between groups who differ in gender, ethnicity and socioeconomic status (SES) constitutes one of the most pernicious current social problems. Males, specifically, are not only under-represented in post-secondary institutions, a phenomenon that began to emerge in the 1980s, but complete their education at a lower rate and obtain lower grades (Buchmann and DiPrete, 2008; Conger and Long, 2010). The gender gap first manifests itself in high school, where females earn better grades and rank higher in their classes. Several studies conclude that female students tend to be more motivated when starting academic programmes, spend more time studying than male students and work harder (for example, Buchmann and DiPrete, 2008; Richardson et al., 2012). Immigrant children in the European Union (EU) and the United States perform more poorly in the educational system regardless of grade, school type and age (reviewed in Di Bartolomeo and Bonfanti, 2014), although the data are sometimes equivocal. Finally, students from poorer families obtain worse grades and lower standardized test scores, miss more school, drop out of high school more frequently, and are less likely to attend and/or graduate from post-secondary institutions, although the effects are also dependent on timing, depth and duration of poverty, as well as provision of learning experiences at home (Brooks-Gunn et al., 2005). The existence of ethnic-group-based inequality is often attributed to social factors such as negative stereotyping (Cohen et al., 2009; Miyake et al., 2010) and differences in peer and parental relations (Gore and Aseltine, 2003). However, the relationship between such factors and the gender gap remains unclear and, overall, the causes of both gaps are poorly understood (cf. Buchmann and DiPrete, 2008).

Women reached parity with men in terms of college bachelor degree completion in 1982. Since then, the gap favoring women has grown, although the female advantage among minority students appears to be much weaker (Buchmann and DiPrete, 2008). Many explanations have been offered for the former phenomenon, such as growing gender egalitarianism and declining discrimination. However, the causal factors have not been identified with any real certainty (Buchmann and DiPrete, 2008). Women do enter college with higher high school grades, and this might provide them with a head start (Conger and Long, 2010), but this fact cannot explain the origin of the gender gap already extant in high school.

Despite the fact of this profound societal transformation, research lags behind society, and the literature still often stresses the female disadvantage in terms of education (Jacobs, 1996). Much of it focuses on the increasingly small number of areas where women are still under-represented, attributing this gap to stereotype threat, moderated by gender identification, or the importance placed on gender identity by women (Schmader, 2002). Thus, most interventions address the stereotype threat for women regarding performance on subjects where men often excel (for example, math, engineering), using such techniques as values affirmation (writing about important values; for reviews see Miyake et al., 2010; Cohen and Sherman, 2014; Walton, 2014), and self-affirmation (writing about a valued self-relevant characteristic such as sense of humour or social skills, Martens et al., 2006). Both these particular interventions were effective in enhancing female but not male performance (Cohen and Sherman, 2014). Studies that do acknowledge the current disadvantage of men, often propose that the under-representation of men in teaching is a problem that needs to be addressed by hiring more male teachers (for example, Arnot et al., 1999), although there is no proof that this strategy has any effect (Martin and Marsh, 2005; Carrington et al., 2008).

The achievement gap between European-American majority and ethnic minority students remains large and concerning (Jencks and Phillips, 1998). Ethnic minorities in the present article are defined as the first and second generation population with a non-Western (for example, African, Middle-Eastern, Asian) foreign background (see method section for a more elaborate description and definition; note that the majority in our context are native Dutch students). The existence of this gap has been attributed, in part, to psychological reasons, such as the pervasive effect of negative stereotypes (as in the case of the gender gap). Several studies have discussed the fact that awareness of a negative stereotype about a given group’s intelligence might depress academic engagement and performance (for example, Aronson et al., 2002; Cohen et al., 2006; Cohen et al., 2009). Such stereotype threat has apparently been countered by informing students that intelligence is malleable, rather than fixed (for example, Dweck, 1986; Aronson et al., 2002), and by helping them reappraise the stereotype threat (for example, Walton and Cohen, 2011). However, as Aronson et al. (2002) indicate, many factors other than stereotyping may affect academic performance.

Sociocultural factors, such as persistent economic inequality, may also play a role. According to Reardon (2011), the income achievement gap (academic performance differential between children at the 90th versus 10th percentile of family earnings) is now double that of the black–white gap. Five decades ago, by contrast, the latter was 1.5 times the former. The stress produced by such disparity, discussed in detail by Wilkinson and Pickett (2009), appears to produce scepticism among those lower in the SES hierarchy about the relationship between effort, ability and life outcomes, with an attendant lack of motivation for achievement. It is important to note, however, that neither the psychological nor the sociocultural theories described above account well for the emergence of higher female performance.

Despite the fact that the causes of gender and ethnic performance gaps remain poorly understood, many initiatives have been set in place to reduce these inequalities. One such programme, designed to boost the sense of belonging among marginalized college freshman groups, appears to be particularly effective, halving the academic performance gap characteristic of ethnic minorities (African Americans) over 3 years (Walton and Cohen, 2011). The majority of interventions are merely assumed to be successful, however, and their effectiveness is often not assessed (Morisano et al., 2010). When they are, meta-analyses indicate that their positive effects range from small to moderate (Robbins et al., 2009), and a small minority have a lasting impact on students’ academic performance (Wilson, 2011). Furthermore, few if any of the current interventions are conceptualized or structured so that they might address the growing gender gap favoring women, and they are frequently designed with a single relatively homogeneous minority group (for example, African Americans) in mind.

Finally, practical difficulties in employing most evidence-based interventions are manifold, preventing wide-scale implementation. Many require extensive contextualization and careful introduction, are often designed to target only one specific group of underperformers, and are difficult to scale widely and effectively (Wilson, 2011). Thus, it comes as little surprise that overall college completion rates have remained relatively stable over the past 20 years (Carey, 2004; Tinto, 2010), while the establishment of a widely adopted means of improving academic achievement and decreasing drop-out in post-secondary education has remained a task for the future.

It is for such reasons that our research team has been assessing the broad-scale effects of structured techniques to help students formulate and articulate comprehensive goals. Goal setting, an intervention most frequently employed in business and corporate environments, appears of high potential effectiveness (Schmidt, 2013), and can be simple and direct enough to avoid the pitfalls of excess complexity, specificity and cost.

Goal setting

Goal-setting theory, developed in the mid-1960s by Edwin Locke, provides one of the most influential and practical accounts of motivation, in managerial and academic contexts (for a review see Locke and Latham, 2006). Hundreds of articles have supported the theory’s main claims. The establishment of specific, challenging goals clearly stimulates goal-directed behavior and boosts performance (Locke and Latham, 1990; Zimmerman and Schunk, 2001). Conscious goal setting appears to increase the probability of goal-directed action and goal-related outcomes, while the acts of mentally contrasting positive and negative outcomes and considering how goal-related obstacles might be overcome appears to free up mental resources, improving student self-discipline and performance (Duckworth et al., 2013). Goal setting is clearly related to individual performance improvement (Locke and Latham, 2013) and to overall organizational productivity. Schmidt (2013), for example, used utility analysis during his meta-analysis of studies including more than 20,000 participants, concluding that employers could assume a 10% increase in productivity following goal-setting exercises helping workers establish realistic, but difficult and specific goals. Programmes aimed at helping individuals set and elaborate on long-term personal goals have proved particularly successful (Morisano et al., 2010; Travers et al., 2014).

A well-designed goal-setting programme links specific subgoals to more general, comprehensive goals and, likewise, proximal subgoals to their more distal counterparts (for example, Latham and Brown, 2006; Locke and Latham, 2006). Latham and Brown (2006) demonstrated, for example, that goal setting enhanced academic performance of MBA students who set their own proximal and distal learning goals (compared with students who merely set distal goals, or were urged to do their best). Other researchers have shown that for students with clear goals, goal-conflicting temptations seem to be able to strengthen goal-directed behavior, instead of weaken it, since these temptations activate the “larger” goals this information conflicts with and thus tend to inhibit giving in to temptations. For instance, students who have set clear goals are better able to withstand the temptation to procrastinate or to distract themselves with other activities (for example, watching TV; cf. Kruglanski et al., 2002).

Thus, goal setting seems to enhance what has been broadly defined as self-regulation (Latham and Locke, 1991; Oettingen et al., 2000; Zimmerman and Schunk, 2001). For this to occur, goals must be challenging enough to inspire the quest for their attainment, but not so difficult that failure is probable (Locke and Latham, 2002). Levels of perceived self-efficacy increase, as progress is made, and the sense of accomplishment rises (cf. Latham and Seijts, 1999; Latham and Brown, 2006).

This is all in logical keeping with cognitive and neuropsychologically predicated models of motivation. The establishment of a goal tags behavior relevant to that goal with dopaminergically mediated and rewarding incentive significance, motivating approach behavior (Gray, 1982). Complex hierarchies of such goals must be brought into being, for effective, complex functioning (Powers, 1973; Carver and Scheier, 1998), and the establishment of such a hierarchy tags specific, implementable subgoals with the positive affective significance of the superordinate goals they serve (Peterson, 1999).

The mental structures subsuming goal-directed action focus perception, specify the objects of attention, determine the emotional significance of ongoing events (as these are evaluated in relationship to the current goal-framework; Deci et al., 1991; Deci and Ryan, 2000; Hirsh et al., 2013). They also facilitate employment of task-relevant knowledge and strategies, and direct action towards goal-relevant activities, increasing energy, persistence and motivation (Locke et al., 1981; Locke and Latham, 1990, 2002; see also Smith et al., 1990). Well-organized and articulated personal goal hierarchies also appear to constrain entropy and uncertainty, decreasing the probability that anxiety and avoidance will compromise both health and productivity (Hirsh et al., 2012). Thus, such structures appear both to enhance positive meaning, in the manner described above, as well as to delimit negative meaning, by restricting threat and anxiety (Gray, 1982).

Morisano et al. (2010) put these ideas into practice by contrasting the effects of a detailed, explicit and written goal-setting intervention with those of an extensive written non-goal-oriented intervention among university students on academic probation. Goal setting markedly improved the grade point average in the former group, as well as substantively increasing the proportion of students who stayed enroled full-time. This study indicated that goal setting can work effectively and efficiently to enhance academic performance, and suggested that it might be particularly effective among struggling students.

The online goal-setting intervention

The intervention reported here, an elaborated goal-setting programme (see http://www.selfauthoring.com; future authoring), was designed with all due scientific and practical considerations in mind. It is a package intervention, with elements derived from goal-setting theory, as detailed above, and from the extensive published work on the salutary effects of expressive writing (Pennebaker and Chung, 2011; for reviews see Smyth, 1998; Pennebaker et al., 2003). It also included elements derived from Darwinian-inspired creativity models (Simonton, 1999) such that students were asked, in Stage 1, to formulate, step by step, a loose, inclusive initial vision and counter-vision and then, in Stage 2, to articulate that vision in a detailed, edited and prioritized fashion, and to fortify their commitments with careful arguments, so they could deal with opposition, doubts and practical difficulties. While completing these two stages, participants were encouraged to consider the nature and desired quality of their future experience on a number of important life dimensions—family, intimate relationships, activity outside of work, career, education and so on. In this manner, participants emerged from the process with an implementable, practical, coherent and defensible plan. Later, participating students scheduled a 10 min session with a professional photographer, for a picture combined with a motivational statement. This was Stage 3, and the final step, of the process (see Supplementary Material for a more elaborate description of the intervention).

The intervention helped participants to produce clear and specific goals, instead of more general “do your best” goals (Austin and Vancouver, 1996; Locke and Latham, 2002), helped them to avoid potential goal conflicts (Locke et al., 1994), and encouraged them to assess whether their goals were practical and attainable (Locke and Latham, 2002). It required them to make a mental comparison of the future and the present and to develop “if-then” strategies for dealing with potential obstacles (Duckworth et al., 2013), and enabled them to form plans for assessing and monitoring progress towards their goals (Schunk, 1990). Participants were also asked to produce and make public a single summary goal statement, in an attempt to use social pressure to increase goal commitment.

The programme is (1) cost-effective, fully scalable and available online (Stages 1 and 2), (2) requires little if any potentially expensive and time-consuming contextualizing, (3) is equally applicable, in theory, to all groups of underperforming students, despite the putatively diverse origins of inequality and (4) has previously been shown, in a randomized control study, to be effective, specifically although not necessarily exclusively, for struggling students (Morisano et al., 2010). Other researchers have noted, similarly, that goal reflection may be particularly useful for non-thriving students (Zimmerman, 2002). Underperforming students may be in particular need of clear goals to self-regulate: to increase on-task time and focused action, for example.

The male-skewed tendency towards resistance to authority during childhood and adolescence (Hudziak et al., 2003; López-Romero and Romero, 2010) may also make it more necessary for males to set personally relevant goals to motivate themselves to perform well. The problems of values, ambition and broader context associated with the conflicts inherent in integration may also increase goal-related conflict for visible ethnic minorities attempting to adapt themselves to their new cultural surroundings. In consequence, we believed that the goal-setting exercise might prove particularly useful for disadvantaged groups, and expected reductions in both the post-intervention gender and ethnicity gaps.

We hoped that the short-term effects of success during goal pursuit might result in positive long-term effects, in a “recursive virtuous cycle” of study success (Walton and Cohen, 2011). Given that the intervention specifically targeted personal goal pursuit (including that associated with academic performance), which is logically related to important subsidiary factors such as time on task, assignment completion and exam attendance, we also expected that its effects would manifest themselves sooner than those produced by other less direct interventions relating to social belonging or values affirmation (Miyake et al., 2010). We therefore report here the results of a structured written goal-setting intervention, completed online; this was aimed at enhancing the performance of lower-performing students, and was applied to a large population of students, followed over a 2-year period.

Methods

The study made use of existing data derived from a larger data-gathering effort (see Schippers et al., 2014). Its design was quasi-experimental, comparing the academic performance of a post-intervention cohort with that of three pre-intervention control cohorts. The methods were carried out in accordance with the approved American Psychological Association guidelines and all (quasi-)experimental protocols were approved by the research school’s (ERIM) institutional review board.

“Ethnic minorities”, for the purposes of article, were defined as the first and second generation population with a non-Western (for example, African, Middle-Eastern, Asian) foreign background. This is the definition most commonly used by the Dutch governmental Statistics Body (CBS) to distinguish between ethnic majority (that is, Dutch) and ethnic minorities. It has been widely applied by ministries, local governments and media in the Netherlands (Alders, 2001).

Subjects were recently enrolled first-year students (N=703 for the intervention cohort and N=896, 825 and 720, respectively, for the pre-intervention control cohorts). We could not specify the ethnicity of 44 students (6.3%). Thus, from the 659 students used in the analysis of the intervention cohort sample, 20% of the students belonged to the non-Western ethnic minority group. The final samples for the pre-intervention control cohorts were 841, 723 and 656 from pre-intervention years 1, 2 and 3, with 18, 21 and 19% of the students, respectively, belonging to the ethnic minority.¹ The remaining population in the Netherlands is considered native Dutch and defined here as ethnic majority. As is typical of many business schools, 72%, 69% and 71% of the pre-intervention cohorts at the Rotterdam School of Management (RSM) were male, as well as 71% of the students in the intervention cohort.

Since the study made use of existing data, obtaining prior consent was not feasible. Therefore, and because the intervention was an integral part of the academic programme, students were informed by e-mail of the purposes and preliminary outcomes of the study, and were given the chance to opt out. Five students indicated that they wanted to opt out; these students were removed from the database.

The participating university relies on the European Credit Transfer and Accumulation System (ECTS), a system that awards standardized course credits in accordance with work required to achieve the objectives of a given higher education programme. The system was designed to ease cross-institutional transfer and academic progression across the EU and in certain other countries. ECTS credits are awarded for successfully completed programs. Number of ECTS credits earned is a primary measure of academic achievement, standardized across the EU (Grosges and Barchiesi, 2007), within the ECTS (http://ec.europa.eu/education/tools/ects_en.htm), designed to represent the student workload required to achieve the objectives of a given study programme, and incorporating a standardized ECTS grading scale. Within this system, credits can only be awarded when a pass grade is obtained for a course of a specified time requirement. The full course load in a standardized European academic year corresponds to 60 ECTS credits, equivalent to 1,680 h of study.

Under the “Binding Study Advice” (BSA) system, in effect since 1999, students in the business programme are required to achieve 40 out of 60 ECTS in their first year to continue to the second. Furthermore, by the end of their second year, students must have attained all 60 first-year ECTS otherwise they must leave the programme.

The programme suffers from high drop-out rates, in part because differential selection for programme entrance is strictly limited by Dutch law, and in part because of the stringent nature of the BSA system. Drop-out rates approximate 50% in the first year, with substantial gender and ethnic differences (for example, 62% of the ethnic minority students dropped out in the year preceding the intervention), but declines substantially after the first year. Despite the BSA system, students typically complete the 3-year bachelor programme in 4 years, with outliers requiring up to 7 years.

College academic records

Gender and ethnicity data were gathered using university transcripts. Official university transcripts were collected for all participants in the intervention cohort at the end of the first and second year, providing information on the number of ECTS (credits) obtained and retention rate.

The goal-setting intervention

The goal-setting intervention was delivered as part of the curriculum for a full cohort of students in the first trimester of their first year. The effects of the intervention were assessed by contrasting that cohort’s performance and enrolment status with that of the average performance and enrolment of three previous pre-intervention control cohorts. We chose 3 years of control cohorts for several reasons: (1) the curriculum had been substantively altered and updated precisely 3 years before the intervention cohort, (2) the university rules and the curriculum remained virtually unchanged during the 3-year period after those changes, offering a nearly ideal window in which to compare the different cohorts and (3) it allowed us to control for natural fluctuations in academic performance across different cohorts provided with the same curriculum.

The intervention required two sessions (Stages 1 and 2, described previously, below, and in the Supplementary Material) of about 2 h each. This was followed by a subsequent 10 min visit to a professional photographer (Stage 3) for a photo to be combined with a single goal statement chosen by the student and then made public (part of an “I WILL” motivational initiative already in place at the university). Students were provided with an individual login code and information about how and when to complete the homework assignment. They were explicitly instructed to complete Stage 1 in one uninterrupted session, and Stages 2 and 3 in a second session.

Stage 1 of the online intervention guided students to think in a structured way about habits they would like to improve, what they might like to learn, their social, leisure and family lives, and their future career. Then they were asked to write freely for about 15 min (without worrying unduly about grammatical niceties), about the life that would be good for them 3–5 years in the future, assuming that things were going as well as realistically possible. Following this, they were asked to write for the same length of time about the future they would truly want to avoid, but that could arise if bad habits or lack of discipline took them in the wrong direction. In Stage 2, students were asked to define and describe their overall plan for the future, beginning by specifying, clarifying, defining and prioritizing eight specific goals relating to their initial vision and counter-vision. They were then asked to articulate their motives, to consider in detail the potential personal and social impact of their goals, to describe detailed strategies for goal attainment and to delineate clearly a strategy for monitoring progress towards those goals. Stage 3 comprised the photo and “I WILL” statement with their ambition and goals, as a form of public commitment (Hollenbeck et al., 1989) transmitted through campus posters and via social media such as Facebook. A more detailed description of the intervention is available in the Supplementary Material.

Delivery of intervention (first semester of Year 1)

Three weeks after the start of Year 1, students received a login from their tutors as part of the introductory course on Management Skills. The tutors described the purpose of the intervention to their groups of 20–25 first-year students. Students were required to finish Stage 1 within 2 weeks, and Stage 2 within 4 weeks, so in effect Stage 1 was due 5 weeks and Stage 2 7 weeks after college entry. Stage 3, the “I WILL” initiative was completed 1–2 weeks after that. After completing the online part of the programme, they were asked to print out their work and show it to their tutor, in fulfillment of the course requirement. The tutor made a note of each student’s participation. Participation in Stage 3 was obtained from the university records.

Results

We analysed the progress of students using two primary dependent variables: number of ECTS credits earned (as described previously) and retention rate.

In line with our expectations, analysis of variance (ANOVA) between cohorts revealed (1) a significant increase for the intervention cohort in number of credits earned for male majority students [M_{control cohorts}=33.32 (SD=20.45), M_{intervention cohort}=40.90 (SD=18.04), Cohen’s d=0.39, F(1, 1506)=37.96, P=0.000] and (2) a significant increase in number of credits earned for male minority students [M_{control cohorts}=26.42 (SD=20.01), M_{intervention cohort}=37.95 (SD=20.49), Cohen’s d=0.57, F(1, 336)=19.85, P=0.000].

No significant increase was apparent, however, for either female majority students [M=40.77 (SD=19.09), M_{intervention cohort}=42.87 (SD=20.80), F(1, 544)=1.10, P=0.294] or female minority students [M_{control cohorts}=28.24 (SD=19.78), M_{intervention cohort}=34.06 (SD=22.26), F(1, 216)=3.22, P=0.074].

Within the cohorts, separate univariate ANOVA revealed that the intervention reduced or virtually eliminated many of the academic achievement differences evident in the subgroups within the control cohorts. In the control cohorts, for example, there were significant main effects of gender [M_male=32.07 (SD=20.54), M_female=37.24 (SD=20.08), Cohen’s d=0.25, F(1, 2016)=16.30, P=0.000] and ethnicity [M_majority=35.30 (SD=20.36), M_minority=27.13 (SD=19.92), Cohen’s d=0.41, F(1, 2016)=71.55, P=0.000], as well as a significant interaction between gender and ethnicity [M_{majority male}=33.32 (SD=20.45), M_{majority female}=40.77 (SD=19.09), Cohen’s d=0.38; M_{minority male}=26.42 (SD=20.01), M_{minority female}=28.24 (SD=19.78), Cohen’s d=0.09, F(1, 2016)=6.03, P=0.014].

As predicted, however, the gender effect was no longer significant in the intervention cohort [M_male=40.34 (SD=18.62), M_female=40.25 (SD=21.56), Cohen’s d=0.01, F(1, 592)=0.23, P=0.633]. Furthermore, the effect for ethnicity was much reduced, although it remained significant [M_majority=41.43 (SD=18.88), M_minority=36.40 (SD=21.21), Cohen’s d=0.25, F(1, 592)=8.55, P=0.004]. The gender by ethnicity interaction was also no longer significant [M_{majority male}=40.90 (SD=18.14), M_{majority female}=42.87 (SD=20.80); M_{minority male}=37.95 (SD=20.49), Cohen’s d=0.10, M_{minority female}=34.06 (SD=22.26), Cohen’s d=0.18, F(1, 592)=2.12, P=0.146]. These results indicate that both the gender and ethnicity gap were reduced.

More detailed analyses (see Table 1; Figure 1) revealed that the remaining ethnicity effect in the intervention cohort was significant only between the majority male and females on the one hand, and the female minority students on the other. No significant differences remained between male and female majority students, between male and majority and minority students, between female majority students and male minority students, nor between male and female minority students (Fig. 1; Table 1).

Table 1 Comparison of differences between subgroups in number of credits and retention rate after Year 1 between the control cohorts (combined) and intervention cohorts

Figure 1

Number of credits (ECTS) earned after the first academic year by gender, ethnicity and cohort.

(a) While the three pre-intervention control cohorts show a consistent gender gap, this gap closes almost completely in the intervention cohort, even though all students in the intervention cohort participated. (b) While the ethnicity gap seems to widen rather than close in the pre-intervention control cohorts, in the intervention cohort the gap closes significantly. (c) The interaction between gender and ethnicity shows that while both gaps diminish in the intervention cohort, the largest performance gain is achieved by male minorities.

Between cohorts, χ² analysis with retention (percentage of cohort) as the dependent variable revealed that retention was significantly increased for the male majority students [Retention_{control cohorts}=56.5%, Retention_{intervention cohort}=72.9%, χ² (1, N=1509)=29.57, P=0.000] and for the male minority students [Retention_{control cohorts}=43.6%, Retention_{intervention cohort}=67.1%, χ² (1, N=338)=13.33, P=0.000]. There was no significant increase for the female majority students [Retention_{control cohorts}=70.0%, Retention_{intervention cohort}=77.2%, χ² (1, N=546)=2.47, P=0.116]. Retention rate change among the female minority students was marginally significant [Retention_{control cohorts}=44.6%, Retention_{intervention cohort}=59.6%, χ² (1, N=218)=3.59; P=0.058].

Within the cohorts, whereas retention rate differences between the pre-intervention groups were significant [Retention_{majority male}=56.5%, Retention_{majority female}=70.0%, Retention_{minority male}=43.6%, Retention_{minority female}=44.6%), χ² (3, N=2017)=58.34, P=0.001], the differences in the intervention cohort were reduced below statistical significance [Retention_{majority male}=72.9%, Retention_{majority female}=77.2%, Retention_{minority male}=67.1%, Retention_{minority female}=59.6%, χ² (3, N=593)=6.70, P=0.082]. This indicated that in the intervention cohort, the differences between the subgroups consisting of gender and ethnicity with respect to retention rate were significantly reduced. This was despite the fact that all subgroups in the intervention cohort improved to some extent, with groups that previously performed worst improving most. The increase in percentages academic performance and retention rate were impressive, ranging from 5 to 34% for number of credits earned and from 10 to 54% for retention rate (see Table 1).

Closing the gender and ethnicity gaps

Detailed analyses comparing each two subgroups within the control cohorts as well as in the intervention cohort further indicated that the goal-setting intervention reduced the gender and ethnic gap in retention. In the control cohorts, the differences between every subgroup were highly significant except for the difference between minority males and minority females. There were no longer significant differences between the subgroups In the intervention cohort, with the exception of small differences between majority males and females and minority females. However, even these differences were significantly smaller than in the pre-intervention cohorts (see Table 1; Fig. 2).

Figure 2

Closing of the gender and ethnicity gap as a function of academic year, for number of credits (ECTS) earned and retention rate.

(a) With respect to gender, the gap in number of credits earned closes altogether after the first academic year, and increases slightly after Year 2. For ethnicity, the gap closes considerably after Year 1 and even more after Year 2. (b) With respect to gender, the gap in retention rate closes altogether after the first academic year, and opens slightly after Year 2. For ethnicity, the gap closes considerably after Year 1, and remains stable in Year 2.

After the first year, there was a marked reduction in the gender gap in performance with respect to the number of credits earned after 1 year. In the control cohorts, there was a difference of 5.17 ECTS between female (M=37.24 ECTS, SD=20.08) and male students (M=32.07 ECTS, SD=20.54), Cohen’s d=0.26. The intervention cohort difference shrank to a mere 0.09 ECTS between female (M=40.25 ECTS, SD=21.56) and male students (M=40.34 ECTS, SD=18.62), Cohen’s d=0.01, for a reduction of 98.25% (Fig. 2a). To check if this effect was lasting—that student performance did not decline during Year 2—we calculated the difference in the number of credits earned in Year 2. Analyses showed that this was essentially equivalent for the control cohorts (the difference was 3.68 ECTS) and the intervention cohort (the difference was 3.49 ECTS); a reduction of 5%, meaning that the gap did not widen in Year 2.

Furthermore, there was a marked reduction in the gender gap with respect to retention after 1 year. In the control cohorts, there was a difference of 8.6 percentage points between female students (Retention=62.8%) and male students (Retention=54.2%). The post-intervention retention difference shrank to 0.2 percentage points in the intervention cohort (Retention_{female students}=72.0%; Retention_{male students}=71.8%), a gender gap reduction of 97.67%. Thus, the goal-setting intervention virtually eliminated the gender gap in retention and number of credits earned (a reduction of approximately 98%). The effect on retention also seemed to be lasting. In the control cohorts the difference between female (59.3%) and male students (48.6%) was 10.7 percentage points after 2 years. In the intervention cohort, by contrast, the difference with regard to retention rate between female (68.0%) and male students (66.8%) was 1.2 percentage points (a gender retention difference of 1.8%). This is a reduction of 88.78% (Fig. 2b).

The ethnic performance gap in number of credits earned and retention was also reduced significantly in the intervention cohort. In the control cohorts, after 1 year, there was a difference of 8.17 ECTS between the majority (M=35.30, SD=20.36) and minority students (M=27.13, SD=19.92), Cohen’s d=0.41. In the intervention cohort, by contrast, there was a difference of 5.03 ECTS (Credits _majority M=41.43, SD=18.88; Credits _minority M=36.40, SD=21.21), Cohen’s d=0.25, a gap reduction of 38.43%. After 2 years, in the control cohorts, there was a difference of 3.74 ECTS between the majority (M=47.84) and minority students (M=44.10). In the intervention cohort, the majority students underperformed slightly in comparison with the minority students (by 0.27 ECTS) (Credits _majority M=48.62; Credits _minority M=48.88). This was a change of 107.2% (Fig. 2a).

The retention in the control cohorts, after 1 year, differed by 16.1 percentage points (Retention_{majority students}=60.1%; Retention_{minority students}=44.0%). In the intervention cohort, by contrast, the difference was 10.0 percentage points (Retention_{majority students}=74.1%; Retention_{minority students}=64.1%), a reduction of the ethnicity gap by 37.88%. After 2 years, in the control cohorts, retention rates differed by 16.4 percentage points (Retention_{majority students}=55.2%; Retention_{minority students}=38.8%). After 2 years, retention in the intervention cohort differed by 10.7 percentage points (Retention_{majority students}=69.5%; Retention_{minority students}=58.8%), a reduction of 34.75% (Fig. 2b). Overall, the difference regarding gender and ethnicity seems to diminish significantly and disappear altogether with respect to gender after the first academic year (Fig. 3).

Figure 3

Retention rate after the first academic year by gender, ethnicity and cohort.

(a) While the three pre-intervention control cohorts show a consistent gender gap, this gap closes almost completely in the intervention cohort, despite the participation of all students in the intervention cohort. (b) While the ethnic minority gap seems to widen, rather than close, in the control cohorts, in the intervention cohort the ethnicity gap closes significantly. (c) The combination of gender and ethnicity shows that both gaps diminish in the intervention cohort, while the largest gain in retention is evident in the male ethnic minorities.

The goal-setting intervention therefore appears to have closed the ethnicity gap by approximately 38% in both retention and number of credits earned after 1 year. The ethnicity gap took somewhat longer to close than the gender gap, taking 2 years instead of 1 to close almost completely, by 93%.

Additional analyses

There is often a gap between important goals that people have set and their actual goal attainment (Webb and Sheeran, 2007). A significant body of research has shown that the relationship between goal setting and performance is mediated by factors such as attention to goal-relevant activities, persistence and the discovery of task strategies to facilitate goal achievement (Zimmerman and Schunk, 2001). Locke and Kristof (1996) showed, for example, that students who achieved higher grades tended to use well-specified study methods and often completed all their assigned work. These students seem to have established the more specific achievement goals that typically leading to better performance than vague or general goals, such as try to “do your best” (Locke and Latham, 2002).

The current goal-setting intervention was aimed at getting students to reflect on their general, higher-order life goals, prioritize them, form implementation intentions and monitor goal attainment. Presumably, self-regulation becomes easier when the specifics of course work were viewed by students in the context of such globally important, broader life goals. Indeed, the study of Morisano et al. (2010) showed that it was participation in the goal-setting programme, per se, rather than the number of academic goals specified, that was important in relation to academic achievement. We also attempted to determine what more specific factors might have produced these changes in number of credits attained and rate of retention.

University rules governing our participant students allowed them to skip regular exams and wait for later, programmed exam re-sits, instead. In the first year, students can take a maximum of 12 regular exams, but are allowed to defer these until later, if necessary. These exam-related rules allow for flexibility with regards to sudden, unexpected life events, but also enable counter-productive avoidance behavior and procrastination. Since diploma completion is an important goal for most students, they must stay on track, instead of procrastinating (for example, waiting for the re-sits instead of taking part in regular exams; cf. Bayer et al., 2010). Thus, we hypothesized that any decrease in the number of re-sits taken by the students after completing the intervention might be a marker for increased commitment to achievement and career (reflected in a decrease in procrastination). We also hypothesized that struggling students, in particular—males and ethnic minorities—would be better able to prioritize their goals, after completing the intervention, and would therefore be inclined to participate more regularly in the study programme, avoiding re-sits or other means of postponing exams.2

Univariate ANOVA indeed revealed that in the control cohorts there was a significant gender effect [M_male=10.32 (SD=2.79), M_female=10.84 (SD=2.30), Cohen’s d=0.20, F(1, 1740)=11.99, P=0.001] and a significant ethnicity effect [M_majority=10.67 (SD=2.45), M_minority=9.74 (SD=3.25), Cohen’s d=0.32, F(1, 1740)=41.05, P=0.000] in relation to number of exams taken. There was no significant interaction effect between gender and ethnicity [M_{majority male}=10.49 (SD=2.62), M_{majority female}=11.16 (SD=1.85), M_{minority male}=9.56 (SD=3.37), M_{minority female}=10.01 (SD=3.05), F(1,1740)=0.45, P=0.503].

In the intervention cohort, however, neither the gender effect nor the ethnicity effect remained significant: [M_male=10.44 (SD=3.01), M_female=10.05 (SD=4.03), Cohen’s d=0.11, F(1, 574)=1.64, P=0.201] and [M_majority=10.47 (SD=3.10), M_minority=9.83 (SD=4.06), Cohen’s d=0.18, F(1, 574)=3.66, P=0.056]. Similar to the pre-intervention cohorts, the gender ethnicity interaction effect in the intervention cohort was not significant [M_{majority male}=10.52 (SD=2.77), M_{majority female}=10.34 (SD=3.88), Cohen’s d=0.05, M_{minority male}=10.12 (SD=3.88), M_{minority female}=9.40 (SD=4.33), Cohen’s d=0.18, F(1574)=0.58, P=0.445].

Detailed analyses (see t-tests in Table 2) showed that in the control cohorts there were significant effects between all subgroups, except between majority males and minority females and between minority males and minority females. In the intervention cohort no significant differences remained between any of the subgroups. This may help explain, practically, why the gender and ethnicity gap closes after the intervention: the groups of students that performed worse in previous, pre-intervention cohorts now take exams at a rate equivalent to the previously higher-performing groups. This suggests that these groups are now characterized by enhanced self-regulation. As a result, their academic integration increased (Rienties et al., 2012).

Table 2 Comparison of differences in number of regular exams between the control cohorts (combined) and intervention cohorts

Discussion

In this article, we assessed the effects of an online goal-setting programme on the academic performance of a full cohort of undergraduate management students. We specifically concentrated on and contrasted the performance of primary gender and ethnicity subgroups in the intervention cohort with those of previous cohorts. In our research, we took a novel approach to goal-setting theory and stated that formulating life goals will help students enhance their academic performance. We hypothesized that the performance enhancement produced by this programme would be especially pronounced for previously poor-performing students, and that it might help redress both the gender and ethnicity gap. The results indicated that these hypotheses were well-founded: substantive performance gaps can be closed, apparently regardless of their origin, with a generic, scalable online intervention. Furthermore, the effects of the intervention manifest themselves within a single academic year. In addition, although participants benefited, generally, in terms of academic performance, the (comparatively underperforming) male and ethnic minority students showed the greatest improvement. This speaks well to the generalizability of the process, of which the effectiveness has now been demonstrated in Canada (Morisano et al., 2010)—as well as in the Netherlands, and in the present study with males and females and an ethnically diverse student population. After Year 1, the gender gap closed by 98%, while the ethnicity gap closed by 38%, rising to 93% after Year 2. Ethnic minority males earned 44% more credits, and their retention rate increased 54%. Whereas the meta-analysis of Robbins et al. (2009) concluded that most interventions show relatively modest effects, we found that our intervention cohorts showed substantial increases, ranging from 5 to 44% for various subgroups in terms of academic performance and from 10 to 54% in terms of retention rates, compared with the pre-intervention cohorts.

It thus appears that the goal-setting intervention assessed in this study has a strong salutary effect, improving academic performance and decreasing drop-out, particularly among male students, generally, and among ethnic minority male and female students, more specifically. Given that students had to obtain a minimum of 40 ECTS credits in Year 1 to continue to Year 2, the performance improvement was of particular consequence. Interestingly, the positive effect of the intervention continued to increase in Year 2 among ethnic minority students.

Most previous studies reporting performance improvement addressed either the gender or ethnicity gap (Miyake et al., 2010; Walton and Cohen, 2011), but not both. The goal-setting intervention appears to have addressed both simultaneously, and relatively comprehensively. Furthermore, the current intervention addressed a relatively heterogeneous minority group, as opposed to a relatively homogeneous minority group (for example, African-American students; Cohen et al., 2006; Cohen et al., 2009; Walton and Cohen, 2011). Furthermore, the effects of the intervention manifested themselves within a relatively short period of time. Within a single year, the gender inequality gap was almost closed, while the ethnicity gap was significantly reduced after 1 year and virtually eliminated after 2.

The current results thus highlight the potential importance of detailed, written goal setting for reducing performance inequalities in higher educational settings. They also appear to indicate that modifying students’ conceptions of their futures (and, simultaneously, teaching them that such planned modifications are possible) producing effects powerful enough to override gender and structural socioeconomic impediments (Walton and Cohen, 2011; Walton, 2014). The question of why poorer-performing students hypothetically had less effective future conceptions to begin with still remains. Future research investigating other psychological and sociocultural processing, theoretically underpinning underachievement, should begin to address this more specific and potentially mediating factor. Perhaps an impoverished past, for example, produces an impoverished view of the future—vague, lacking in detail and pessimistic (cf. Walpole, 2003). Such a view would do very little to increase motivation or control counter-productive anxiety.

It appears that the intervention succeeded partly because it specifically improved exam-taking behavior, most likely because of enhanced self-regulation. Indeed, self-regulation does appear to be enhanced when specific goals are set (Latham and Locke, 1991), particularly when people contrast a fantasized ideal future with present reality (Oettingen et al., 2001), as was required during the current intervention. Guided reflection on goals as it was used in the current intervention may help elicit meta-cognitive awareness, help reflect on detailed goal-setting and goal-monitoring strategies, and as such enhance self-regulation (Lyke, 2009; Stein and Grant, 2014). Indeed, reflection can be helpful in making sense of prior experiences and may improve future functioning (for a review see Schippers et al., 2013; Ellis et al., 2014).

As a result, students might attain internal state awareness of their preferred goals and future and may therefore be better able to self-regulate and direct energy towards their goals (cf. Grant et al., 2002). Cybernetic/behavioural models of motivation (Gray and McNaughton, 2000) strongly indicate that positive emotion and approach motivation occur primarily when cues indicate progress towards a desired goal occur (with satiation, by contrast, occurring when the goal is achieved). Furthermore, adopting clear goals and belief structures helps to constrain the experience of uncertainty, and writing about goals can clarify and strengthen self-regulation and cognitive integration (Hirsh et al., 2012, 2013). If people experience positive affect in relationship to desired goals, it stands to reason that articulating and delineating those goals will sharpen the connection between specific on-task actions in the achievement of present and long-term goals, and that people will therefore experience a heightened sense of accomplishment when moving towards or achieving their goals.

Such cybernetic/behavioural models also suggest that uncertainty about goals itself can have a powerful inhibiting effect on current effort-intensive behavior, and can also impair overall physical and psychological health, as Pennebaker and his colleagues have demonstrated (for a review see Pennebaker et al., 2003). Thus, clarifying goals, and carefully articulating the relationship between action and outcome, should increase the positive felt consequences of productive behavior in the present, as well as decreasing uncertainty-related negative emotions such as anxiety, disappointment and frustration. However, full differentiation of these two potential motivating effects (enhancement of positive and decrease of negative emotion) is not yet possible, and comprehensive understanding of the effects of expressive writing and goal setting, has not yet been attained (cf. Pennebaker, 2004).

As well as documenting the utility of the programme, particularly to underachieving students, it is worthwhile bringing attention to its ease-of-use, inexpensive nature and minimal disruption during implementation. It was made available online, at low per-student cost. Its implementation required very little additional contextual or explanatory information to be provided, except for that offered in text form online during the process itself. Its demands on teaching staff were therefore low, as was intended during its design. Finally, this intervention is fully scalable, and could be expanded in its present form to serve thousands or tens of thousands of users.

We do strongly believe, however, that the probability of students participating in the intervention and completing it was increased by the fact that it was a required component of the first-year curriculum itself, complete with deadlines. The goal-setting intervention, although not particularly time-consuming, can be sufficiently demanding to potentially dissuade students—particularly those who are likely to perform poorly—from completing it as a mere extracurricular activity.

Conclusion

Overall, the results demonstrate that an inexpensive, scalable, written online goal-setting programme can be used effectively and efficiently to increase educational quality and equality, by promoting improvement in academic performance and retention among students struggling in comparison to their peers, particularly if those students are male and/or from a visible ethnic minority.

Data Availability

The datasets generated during and/or analysed during the current study are not publicly available due to the privacy of these data, but are available from the corresponding author upon reasonable request.

Author Contributions

MS wrote the main manuscript text; AS performed and reported the analyses; AS and MS prepared all figures; JP designed the intervention; all authors discussed the results, commented on and extensively reviewed and edited the manuscript.