Leveling up citizen science

To the Editor — Over the past decade, online science discovery games (SDGs) have become an increasingly popular practice for engaging the public in research activities. This methodology has had the most impact in molecular and cell biology, where participants are recruited for not only data collection and classification tasks, but also solving complex computational problems that require human supervision or insight. As the life sciences move toward increasingly large and ambitious projects, however, there is a need to complement existing models that try to attract users to yet another third-party website. Here we describe our experience with Borderlands, a massively multiplayer online game that embeds citizen-science problems into its virtual universe and engages a new user base, with implications for future citizen science initiatives seeking to meet the growing demands of biology.

Citizen science in biology

Since 2008, the Foldit team has recruited online participants to predict protein 3D structure via a 3D puzzle game distributed as a standalone program for personal computers¹. This project helped refine a retroviral protease structure² and later discovered novel protein folds³. Two years later, Phylo⁴ pushed the gamification concept to reach a broader public by crowdsourcing comparative genomics tasks via tile-matching puzzle games. Other successful projects include web-based games for RNA structural modeling (for example, EteRNA⁵) and neuron segmentation (for example, EyeWire⁶).

The efficiency and effectiveness of SDGs is necessarily determined by the number of contributors and volume of data collected, regardless of citizen-science strategy. Games like Foldit take advantage of large groups of participants for identifying skilled users and promising solutions to a hard problem. In contrast, Phylo and related SDGs aim to harness the wisdom of the crowd by detecting collective behaviors of the gamers while solving small instances of the problem, and using this knowledge to build a global solution.

Having sustainable and large crowds of participants is necessary for opening this technology as a service to the whole scientific community⁷. Popular SDGs represent the best option for solving on-demand problems for many groups of independent scientists. Although few such platforms have yet to be deployed, this approach constitutes a natural and promising evolution of SDGs that is exemplified by recent initiatives like OpenVaccine or OpenCRISPR from the EteRNA project.

Recruiting large groups of volunteers should also serve long-term objectives and not be confined to solving a specific problem without a larger context. Projects like ENCODE⁸ or the Earth Microbiome Project⁹ involve hundreds of scientists worldwide and require assembly and coordination of many diverse areas of expertise. At all stages of the research pipeline and at various degrees of involvement, collection, curation and analysis of the data need the supervision of human experts. But as the size and complexity of the project teams expand, so too does the cost of human intervention and logistics, which may not be sustainable. To expand the boundaries of the system, we need to refactor the organization of large scientific projects.

The state of affairs

The most successful citizen science initiatives have reported several hundreds of thousands of volunteers, and eventually more than a million if we aggregate statistics from distinct projects using the same infrastructure over years of operation. Notably, following a pattern commonly observed in user participation, the majority of the contribution is completed by the participants during the first days or weeks following the launch of the project. To prevent this drop-off, projects like Stall Catchers or EyeWire organize various events and competitions to renew participant interest and maintain engagement. Foldit and EteRNA also regularly unveil new challenges and datasets designed to support research in different areas, such as COVID, tuberculosis or synthetic biology. Nevertheless, in the best case, the volume of work generated reaches a couple of hundreds of hours of work per day¹⁰. Although such a number is useful, it hardly meets the demand of large-scale science projects.

Therefore, despite undeniable successes not only in scientific impact but also in science outreach to the general public, the vision of online citizen science initiatives is still out of range of current methodologies. A model based on distributing scientific tasks through a web page or mobile application managed by academic researchers appears to have reached its limits. Even when a project benefits from extensive media coverage, the core of its audience remains confined to participants already naturally interested in the scientific subjects, which is unfortunately only a small fraction of the population.

Reach out to your public

Fundamentally, the goal of citizen science is to organize science at population scale. The internet has been a key technology for gathering crowds of participants worldwide in a short time, and the development of SDGs has been equally important for enabling citizen scientists to participate in the resolution of complex problems. But the next frontier is to bring SDGs to the general public. In 2016, Massively Multiplayer Online Science¹¹ (MMOS) proposed an original solution to this problem. The first implementation was a collaboration with the video game company CCP to integrate citizen science activities in the science fiction massively multiplayer online role-playing game Eve Online. In less than a year, Project Discovery enrolled more than 300,000 online gamers to classify fluorescence microscopy images for the Human Protein Atlas¹². Remarkably, engagement was also very high, with an average of 100 tasks solved per participant. Overall, these statistics represent a considerable improvement over those of previous projects. The rationale for the approach taken by MMOS is both simple and efficient; instead of trying to attract users to yet another third-party website, bring citizen science to the virtual universes that users already inhabit. This strategy is even more powerful given that gamers are already expert at developing problem-solving skills.

With the introduction of Project Discovery, citizen science entered the AAA video game scene. (AAA is an informal classification used for video games produced and distributed by a mid-sized or major publisher, typically having a high development and marketing budget.) But another crucial step is needed to make this approach broadly accepted by both researchers and game developers. Eve Online is a very special game. The player base is more mature and attuned to science than other player communities. The game is also slower paced and more complex, making it an almost too perfect match for citizen-science activities.

The deployment of SDGs within fast-paced games with a wide and diverse audience is the next frontier. It has the potential to bring millions of contributors to the most ambitious scientific endeavors, but comes with new challenges. The integration of these new activities must preserve the immersion experience and seamlessly integrate into the ecosystem of the host platform. Whereas a game like Foldit requires extensive training and substantial investment of time from the participants, this may be incompatible with action games.

Choosing the right game

The sophistication and gamification of SDGs vary considerably. However, there is a common motif in the gamification of classic citizen science projects: the game is formed around the scientific problem. When integrating citizen science activities with AAA games, the opposite approach must be taken. To be appealing to an established player base of millions, we must first understand their world. This means that we cannot approach game developers with a fixed agenda, but rather work as intermediaries to find the best-fitting scientific problem: one that resonates with the player base, one that fits aesthetically and thematically and, most importantly, one that doesn’t risk breaking immersion. If such a match can be found, we can expect the same level of enthusiasm and activity toward the citizen-science feature in the game as toward any other part. Therein lies the true power and challenge of this approach.

Enter Borderlands Science: an intersectoral collaboration between scientists, engineers and game developers that strives to demonstrate the formidable potential of an alliance of citizen science and the highest profile AAA games. Borderlands Science is a casual tile-matching game (inspired from the previous SDG Phylo) that is distributed within Borderlands 3 — a fast action role-playing first-person video game. Each puzzle in Borderlands Science is made of fragments of microbial 16S ribosomal RNA gene sequences collected by the American Gut Project¹³, and the results are aggregated to improve a reference alignment of millions of sequences.

At first glance, the match between user base and scientific problem seems unlikely. The fast-paced, first person shooter–looter game filled with dark humor is primarily designed for gamers seeking adventure and action. But what could be a dead end turns out to be the strength of this initiative. A deployment in the Borderlands universe offers an opportunity to reach a public not particularly exposed to science, while at the same time it opens the door to a large and strong online community of players who can carry and amplify the impact of this initiative. To complete this picture, the partnership with the Microsetta Initiative (the parent organization of the American Gut Project) provides another central piece of this project. Only a long-term scientific endeavor with apparent benefits for the public can trigger the interest of a large audience and justify the scale of such an enterprise.

The challenges of design

A tight collaboration between the game design and scientific team is essential to guarantee the success of the integration. Since its inception, Phylo aimed to be a casual game. Still, its adaptation to Borderlands required a complete redesign. The scoring system had to be simplified, while preserving its relevance to science. Intuitive rules and swift-moving mechanisms were also essential to accelerate the pace of the game. In particular, entry-level puzzles had to be quick to complete to promote engagement. Finally, the presentation of the activity had to match the host universe. These factors led us to embed Borderlands Science into a retro-style arcade video game that suits well the postapocalyptic atmosphere of the main game (Fig. 1). But this work on adaptation also provides access to powerful mechanisms. For instance, rewards in the form of boosters or skins can be used to engage the gamers. (In video games, boosters are items that gamers collect to temporarily improve attributes of their character, while skins are cosmetic elements used to personalize it.)

Fig. 1

Arcade booth allowing the gamers to access Borderlands Science in Borderlands 3.

The adaptation of Borderlands Science from Phylo enables us to directly quantify the impact of the more recent implementation and calibrate our expectation for this methodology. In fewer than three months, Borderlands Science reached more than 1 million active participants who solved at least one open task, whereas Phylo had an estimated 350,000 participants over ten years. During those three months, we collected 50 million puzzle solutions, resulting in an average of slightly under 50 puzzles solved per player. This is one order of magnitude higher than that of Phylo, which has a mean number of 5 solved tasks per volunteer, and is consistent with previous observations made in Project Discovery⁹. Strikingly, the volume of work generated by Borderlands Science is oscillating between 10,000 and 15,000 hours of work per day, which can be compared to the couple of hundreds of hours per day reported in other projects⁸. The fact that a mainstream shooter–looter action videogame managed to translate a scientific problem to a wide audience of millions of players with such unprecedented success is a clear proof of the validity and viability of the original proposition.

Although it is too early to fully comprehend all the implications of this project, it already demonstrates the formidable potential of partnerships with video game companies to engage large communities of citizen scientists in addressing complex scientific tasks. The reach of video games grows by the day and will very likely continue to do so. In 2019, 65% of adults in the United States played games (https://www.theesa.com/esa-research/2019-essential-facts-about-the-computer-and-video-game-industry/). As players spend more and more time in these virtual worlds¹⁴, it is imperative for us as a society that we find all possible ways to extract value from this time. Tapping into this virtually limitless resource might well be one step to advance toward the dream of building a science-oriented society.

Future opportunities

Inevitably, developing such a platform has a substantial cost in time and resources, as the final product must also meet industry standards and consumer expectations. To justify this investment, any citizen-science game must be sustainable and provide guarantees for longevity. Borderlands Science was released with one of the first downloadable contents of Borderlands 3, which offers not only high visibility but also the opportunity for introduction early in the life cycle of the video game. Yet, if the SDG is expected to run for several years, its purpose must also be maintained over the same period.

We designed Borderlands Science to help us align microbial 16S rRNA sequences from the human microbiome. The same framework can be applied to work with data from different origins, such as soil samples from the Earth Microbiome Project, or obtained with different techniques, such as shotgun metagenomic sequencing. Recently, the Microsetta Initiative started to collect data to study the spread of SARS-CoV-2 and factors influencing susceptibility to COVID-19, which are a natural match for this project. Applications such as aligning coronavirus genome sequences highlight the impact that partnerships with the video game industry may have during emergency situations when the scientific community could benefit from a platform that quickly mobilizes large numbers of participants to complete urgent tasks. Finally, minimal modifications of the game play could authorize new types of alignment, including protein sequences.

Adhering to the principle of sustainability requires applying this strategy to problems with high relevance to multiple areas of science and a broad community of researchers. At the same time, the ability to easily adapt the presentation of the data is another strong asset that facilitates SDG integration. A sequence-based problem like Borderlands Science fulfils these requirements, but graph-based models, such as biochemical networks or COVID-19 contact tracing data, also seem perfectly suited. Mass spectrometry and flow cytometry data are also promising candidates.

Overall, the evolution of our project from Phylo to Borderlands Science suggests a pattern for others to follow for successful integration. Once a scientific problem is identified, the scientific team should develop a prototype to explore the potential for gamification and collect feedback from citizen scientists. Equipped with this information, they can begin the collaboration with the game designers. Our experience suggests that multiple rounds of discussion are necessary and that each party must have an open mind about the objectives and constraints of the other. Finally, the enthusiastic participation of the scientific team at all stages of the promotional campaign is essential for reinforcing the credibility of the project and satisfying the curiosity of the gamers.

If properly executed, the potential impact of the uncanny partnership of researcher and public is huge. After all, as Turing awardee Donald Knuth put it: “Biology easily has 500 years of exciting problems to work on; it’s at that level”¹⁵.