Keeping up standards

    Scientific claims, particularly those related to devices, are often backed up by standardized characterization. In fields without accepted standards, is it beneficial to strive to establish them?

    As hard as it is to believe, almost 60 years after the invention of the laser, at conference lunches and laboratory meetings alike, debate over which devices are and are not lasers is rife. In 2017, we announced1 a laser ‘checklist’ that should be completed by the authors of relevant papers submitted to Nature Photonics (if their paper proceeds to peer review). The checklist aims to encourage authors to standardize characterization as well as to provide information to aid referees . Likewise, in 2015, we announced a checklist for solar devices — another area that also benefits from standardized characterization2.

    Not all items on those checklists are entirely relevant for some new types of laser or solar device, in which case we ask authors to let us know such circumstances and provide details on how they have otherwise suitably characterized their devices. But therein lies a potential problem; in areas where standardized characterization has not yet been established researchers may characterize their devices in different ways . Arguably, a burden is placed on authors and readers alike to comprehend the different characterization approaches, and this may cause difficulty in comparing and verifying claims made in manuscripts.

    In the emerging field of highly transparent photovoltaic devices — which absorb mostly ultraviolet or infrared light but have varying degrees of visible transparency for deployment as windows, for example — there is a relevant review that discusses figures of merit3. Other solar cell-relevant reading includes commentaries on solar cell characterization4, perovskite solar cells5 and mischaracterization of organic and hybrid photovoltaic devices6.

    Imaging and microscopy techniques are also particularly relevant to this discussion. Just think about the number of parameters or procedures involved in determining imaging resolution. If researchers use different characterization approaches, or even different imaging targets, there can be confusion. For example, some imaging approaches are ‘indirect’, leading to questions about reconstruction algorithms.

    Speaking about the complexity of these issues, Thomas Huser — a professor whose group works in areas such as super-resolution microscopy and live-cell microscopy — mentioned that, for example, no two high-power microscopy objective lenses are identical; each objective consists of several lenses and materials and while we use general parameters and point spread functions to characterize them, none of this fully characterizes all various aberrations and differences between objective lenses. “There are no absolute standards. Even though images taken with one specific objective lens can be compared with images obtained by the same lens, an absolute comparison between different microscopes is missing, and it is virtually impossible to accomplish,” Huser told Nature Photonics. “This would require such an extensive characterization of a multitude of parameters in terms of, for example, wavelength-specific throughput (or absorption), wavelength-specific aberrations, long-term stability, characterization at different temperatures, mechanical stress, polarization, and so on. Nobody has ever even attempted to embark on such a challenge.”

    Efforts to calibrate these will only enable a rough, relative comparison with other images taken with the same system, explained Huser. And that is possible, if somehow, all of the variables are more or less unchanged. “There are a number of so-called standards that are often used to at least provide an impression of standardization when it comes to characterizing the optical resolution of microscopes, for example the US Air Force test targets,” said Huser. “Their tolerances are, however, still above the resolution limit of objective lenses with high numerical aperture. Other targets, such as Gattaquant’s DNA origami-based calibration patterns are significantly more reliable in this regard, but these will also only provide a rough calibration for a single colour channel, and for the specific conditions in which the microscope was being used when these calibration images were acquired (such as temperature, humidity, and so on).”

    Roarke Horstmeyer and colleagues recently published a Commentary on standardizing resolution claims for coherent microscopy7. Changhuei Yang, the corresponding author, thinks that creating standards is one thing, but steering the community to employ the standards is another. We spoke to Yang recently and he explained that the current peer-review process is not optimized for this and that the process is individualized and lacks consistency and ‘memory’ to apply standards in a consistent fashion.

    “Right now, we rely on time and ‘osmosis’; a well-designed standard should eventually win out, but waiting for it to happen just seems so wasteful and inefficient,” Yang told Nature Photonics.

    Computational methods are an increasingly relevant issue and Yang explained that in imaging there is a growing need for a standardized way to characterize any given computational method’s trade-off in resolution, noise and information content. “Most computational methods, including phase retrieval, ptychography and compressive sensing techniques, operate in a way that no longer allows a direct analytical way to understand the relationships between those three factors,” said Yang.

    As we push the boundaries of optics, reliable standards might not yet exist, they may be difficult to determine and the community takes time to adopt them. However, for the sake of clarity and rigor, let’s strive to establish them.

    References

    1. 1.

      Nat. Photon. 11, 139 (2017).

    2. 2.

      Nat. Photon. 9, 703 (2015).

    3. 3.

      Traverse, C. J., Pandey, R., Barr, M. C. & Lunt, R. R. Nat. Energy 2, 849–860 (2017).

      ADS  Article  Google Scholar 

    4. 4.

      Snaith, H. J. Nat. Photon. 6, 337–340 (2012).

    5. 5.

      Grätzel, M. Nat. Mater. 13, 838–842 (2014).

      Article  Google Scholar 

    6. 6.

      Zimmermann, E. Nat. Photon. 8, 669–672 (2014).

      ADS  Article  Google Scholar 

    7. 7.

      Horstmeyer, R. Nat. Photon. 10, 68–71 (2016).

      ADS  Article  Google Scholar 

    Download references

    Rights and permissions

    Reprints and Permissions

    About this article

    Verify currency and authenticity via CrossMark

    Cite this article

    Keeping up standards. Nature Photon 12, 117 (2018). https://doi.org/10.1038/s41566-018-0131-6

    Download citation

    Further reading