Every year, many hundreds of thousands of peer-reviewed scientific papers are published, enabling the scientific community to read and assess the reported work. Unfortunately, in some cases, the claimed advance does not bear up to repeated scrutiny, and the paper must be retracted. This can happen both in low-profile and high-profile journals, with recent high-profile retractions in different scientific fields, from condensed-matter physics to genomics studies. While in some cases this is due to a mistake made during the analysis, in other cases the data can simply not be reproduced.

Image reproduced with permission from the Feature by Adams and colleague, Springer Nature Ltd. Scale bar, 2 cm.

There is a growing focus on the reproducibility of scientific studies. Although initially highlighted by the social sciences1, concerns about reproducibility are also raised in the life and physical sciences2. Data may not be reproducible for several reasons, ranging from honest errors, such as those in complicated analyses needed to extract results3,4, to shameful cases of data manipulation5. In this Focus issue, we curate a selection of pieces on reproducibility in different areas of materials science. These overview fields of research, highlight issues that may arise during measurements and provide recommendations for best practice.

In contentious fields with reproducibility issues, it is wise to see what results are on solid ground. In a Feature, by Mikhail Eremets and colleagues, this is done for the field of high-pressure superconductivity, where superhydrides formed at high pressures (of order 100 GPa) can exhibit superconductivity at temperatures up to 250 K. As they note, compared with other reports of high pressure and temperature superconductivity, in these systems, superconductivity follows the conventional mechanism, enabling established complementary approaches to be used. Moreover, given the crystal structures are well characterized, measurements of different samples can be reliably compared across different research laboratories.

Reproducibility of materials synthesis is key. In a Feature, Dave Adams and colleague discuss how to enable this in supramolecular gels. As they note, rheological properties are essential to stem cell differentiation, and this in turn is affected by subtle differences in synthesis. To illustrate, six different people perform the blind synthesis of a dipeptide gel following the same literature method, but given differences in sonication bath temperature or mixing process, quite different gels are synthesized (pictured) and different storage moduli reported. Clear communication of protocols is key. Similarly, Lane Martin and colleagues in a Feature note that even minor differences in ferroelectrics synthesis have substantial impact, all details should be published. If possible, thermodynamics should be used to control stoichiometry, and multimodal approaches used to characterize ferroelectricity, while knowing the limitations of any single technique is important. Jing Kong and colleagues also emphasize the importance of synthesis in a Feature on 2D transition metal dichalcogenides for devices. Intrinsic defects can form during the chemical vapour deposition that forms these materials at wafer scale, while wafer transfer and device fabrication can also form extrinsic defects. It is important to characterize structure during processing, and when devices are prepared, their variability of performance should be reported, not that of a single best ‘champion’ device.

Knowing how to make reproducible measurements is also essential. In a Comment by Alexandra Paterson and colleagues, the figure of merit for organic mixed ionic–electronic conductors μC is discussed, where μ is the charge carrier mobility and C is the volumetric capacitance. They conclude that due to non-idealities in transistor performance it is most accurate to measure these quantities separately. Similarly, a Feature by Joseph Heremans and colleague discusses best practices for thermoelectric property measurements; moreover, they note from interlaboratory studies that uncertainties in the thermoelectric figure of merit are of the order of 20%. Reference materials can aid in determining measurement uncertainty. Daniel Caillard and colleagues in a Feature discuss using transmission electron microscopy to observe deformation in situ. As they note, issues such as electron-beam effects, damages during sample preparation, surface effects, and types of holders and measurements should all be addressed.

A more general issue is to ensure best practices that enable reproducible materials science. Two Features discuss this. Zachary Smith and colleague overview two different classes of gas-separation material, polymers of intrinsic porosity and zeolitic imidazolate frameworks. The former are metastable, whose separation properties evolve with time, while the literature reports on the latter show a considerable degree of variation in their gas selectivity and permeance properties. Multiple material batches should be synthesized, while multi-lab studies are important to understand reproducibility. Aron Walsh discusses how to enable more open and repeatable computational science, as he states, given the complexity of workflows these also should be recorded, perhaps in an electronic workbook. He also notes that open research is an additional workload, which should be formally recognized by institutions.

Scientific research has evolved, and journals have evolved accordingly. We have enabled checklists to aid reproducibility in the life sciences, and in specific fields such as the laser or photovoltaics communities6, while making data available can also help this quest7. Ensuring reproducibility is time-consuming, but it is essential to ensure good scientific practice.