A commentary on evaluation of the evenness score in next-generation sequencing

The development and adoption of appropriate performance measures is critical to advancing science and engineering, because such measures allow the community to rationally choose the (current) best solution to a problem among numerous alternatives. In that sense, measures which help us make decisions by providing a single number criterion are especially useful. For example, the unfortunately named ‘area under the receiver operator characteristic’ (AUROC) is a classic measure that provides a well-principled way to rank the performance of binary classification; interpretable as the probability that a randomly chosen positive example achieves a higher score than a randomly chosen negative example.¹ Before the widespread adoption of AUROC it was often difficult to compare methods from different publications—how does one decide if a method reporting 50% sensitivity with 80% specificity is better or worse than one reporting 30% sensitivity with 95% specificity? Another popular measure which reduces a tradeoff into a single number is the so-called h-index,² which measures the scientific output of a researcher as the number of their published papers with at least h citations.

In this issue, Oexle³ presents a mathematical treatment of the ‘evenness’ score introduced by Mokry et al.⁴ as a single number measure of the uniformity of DNA sequencing coverage. This measure provides an attractive alternative to simply listing the percentage of sites with coverage above an arbitrary threshold (for example, 30 ×), and has been used in some recent publications.^{5, 6} However, to be widely adopted, a measure should not only be informative, but like the AUROC and h-index measures mentioned above also easy to compute and interpret. Oexle’s³ exposition goes a long way toward making this the case for the evenness score by (1) developing a simple and efficient formula for its computation and (2) giving mathematical insight regarding its behavior vis-à-vis the coefficient of variation, a more conventional statistical measure of the width of a distribution. Although his manuscript is by nature mathematical, Oexle’s³ main conclusions are easily understandable without diving deep into the derivations. Thus, regardless of whether the sequencing community ultimately chooses to widely adopt the evenness score or perhaps switches to the coefficient of variation or related measures; the choice can now be a well-informed one.