We read with interest the recent report by Reimann et al. [1], in which authors from Inform Diagnostics™ compare their histopathologic diagnoses of melanocytic neoplasms to the results of single-nucleotide polymorphism (SNP) array (performed at Memorial Sloan-Kettering Cancer Center, New York City, NY), melanoma fluorescence in situ hybridization (FISH) (performed by the authors at Inform Diagnostics, Irving, TX), and the myPath Melanoma test (Myriad Genetic Laboratories, Salt Lake City, UT). We submit that several methodological weaknesses compromise the validity of the study’s results and conclusions, including (1) use of a suboptimal reference standard and (2) adoption of an alternative, author-devised classification system that is not consistent with the clinically-validated classification system provided with the myPath test results.
The scarcity of melanocytic neoplasms with known outcomes often necessitates use of histopathology-based reference standards, but such standards are limited by the inherent inter-observer variability and diagnostic discordance that occurs even among experts [2,3,4,5,6]. The diagnoses for 70% (187/268) of the Reimann et al. [1] cohort were assigned by just a few individuals representing a single commercial laboratory (Inform Diagnostics™) and therefore might reflect the particular diagnostic predisposition of this group. This, in combination with the tendency for melanoma ‘over-diagnosis’ noted by the authors themselves and many others [7,8,9], may have contributed to the apparent low sensitivity of the myPath test observed here (50–72%) vs. that observed previously (92%) in a larger cohort (n = 736) where diagnoses were assigned by a panel of dermatopathologists from a variety of different institutions [10]. When subjective reference standards such as diagnosis by histopathology generate two very different estimates of a test’s performance, an objective standard such as clinical outcome may help determine which estimate is more accurate. There is no objective standard for non-metastasizing melanocytic neoplasms, but in a large cohort of cases proven to be malignant melanomas by the development of distant metastases, the sensitivity of the myPath test was 94% (Table 3) [11].
The authors’ conclusions are also influenced by their decision to re-classify all cases with indeterminate myPath results as ‘benign’ in their calculations of sensitivity and specificity. The authors claim this was done to 'avoid exclusion of indeterminate results in statistical analysis,' yet their own indeterminate results are, in fact, excluded from statistical analyses. Cases that they declared ‘diagnostically ambiguous’ are eliminated from some calculations (see Table 3 in Reimann et al. [1]), and lesions for which they cannot reach consensus diagnoses are excluded from others (see Table 5 in Reimann et al. [1]). In addition, the authors classify indeterminate myPath results differently from one calculation to the next. Specifically, they are considered negative when assessing sensitivity and specificity and when comparing myPath to FISH interpretation, but for agreement calculations the ‘indeterminate’ designation is reinstated and they are counted as discordant. Interestingly, in these agreement calculations, re-assigning myPath indeterminate results to either category—positive or negative—would have substantially improved concordance. In fact, in every circumstance, the classification strategy selected by Reimann et al. [1] is the one that produces the least favorable estimate of myPath performance. In clinical use, indeterminate myPath results are not dismissed as benign; on the contrary, they are considered carefully and on a case-by-case basis within the context of clinical findings, histopathologic features, and all other relevant data. The re-interpretation of indeterminate results as benign in the clinical setting represents a potential danger to tested patients and we strongly discourage this practice, as do several publications cited by the authors themselves [12, 13].
Re-analysis of the data generated by Reimann et al. [1] using the clinically-validated myPath system improves every calculation of the test’s performance (Tables 1–3), and while sensitivity remains lower than that reported in the clinical validation studies, the small number of cases studied by Reimann et al. [1] produce wide confidence intervals that make such comparisons questionable (Fig. 1). It is notable that, when calculated according to the validated classification system, the myPath test’s sensitivity and specificity for cases considered 'unequivocal' and 'ambiguous' by Reimann et al. [1] do not differ, supporting prior observations that performance is consistent regardless of whether lesions are deemed 'ambiguous' or 'straightforward' (Table 3 and Fig. 1) [11, 14]. This suggests that the negative/benign myPath results for 5 of the 15 'ambiguous' melanomas may not reflect a low test sensitivity, but rather a tendency for 'over-classification of benign lesions with atypical features' noted by the authors themselves.
The accuracy of the myPath melanoma test has been evaluated in three separate peer-reviewed clinical validation studies [10, 11, 14] and two clinical utility studies [15, 16] performed in collaboration with more than 16 dermatopathologists from 10 academic institutions. We believe the preponderance of the data demonstrate the myPath melanoma test to be a useful ancillary diagnostic method. However, no test is perfect, nor is any study without flaw, and knowledge of the molecular pathogenesis of specific subtypes and melanocytic neoplasms in general continues to evolve. All diagnostic methods for melanocytic neoplasms, including histopathology, warrant continued investigation.
References
Reimann JDR, Salim S, Velazquez EF, Wang L, Williams KM, Flejter WJ, et al. Comparison of melanoma gene expression score with histopathology, fluorescence in situ hybridization, and SNP array for the classification of melanocytic neoplasms. Mod Pathol. 2018;31:1733–43.
Elmore JG, Barnhill RL, Elder DE, Longton GM, Pepe MS, Reisch LM, et al. Pathologists’ diagnosis of invasive melanoma and melanocytic proliferations: observer accuracy and reproducibility study. BMJ. 2017;357:j2813.
Hawryluk EB, Sober AJ, Piris A, Nazarian RM, Hoang MP, Tsao H, et al. Histologically challenging melanocytic tumors referred to a tertiary care pigmented lesion clinic. J Am Acad Dermatol. 2012;67:727–35.
Veenhuizen KC, De Wit PE, Mooi WJ, Scheffer E, Verbeek AL, and Ruiter DJ. Quality assessment by expert opinion in melanoma pathology: experience of the pathology panel of the Dutch Melanoma Working Party. J Pathol. 1997;182:266–72.
Farmer ER, Gonin R, Hanna MP. Discordance in the histopathologic diagnosis of melanoma and melanocytic nevi between expert pathologists. Hum Pathol. 1996;27:528–31.
McGinnis KS, Lessin SR, Elder DE, Guerry D 4th, Schuchter L, Ming M, et al. Pathology review of cases presenting to a multidisciplinary pigmented lesion clinic. Arch Dermatol. 2002;138:617–21.
Glusac EJ. The melanoma ‘epidemic’, a dermatopathologist’s perspective. J Cutan Pathol. 2011;38:264–7.
Weyers W. The ‘epidemic’ of melanoma between under- and overdiagnosis. J Cutan Pathol. 2012;39:9–16.
Welch HG, Woloshin S, Schwartz LM. Skin biopsy rates and incidence of melanoma: population based ecological study. BMJ. 2005;331:481.
Clarke LE, Flake DD 2nd, Busam K, Cockerell C, Helm K, McNiff J, et al. An independent validation of a gene expression signature to differentiate malignant melanoma from benign melanocytic nevi. Cancer. 2017;124:617–28.
Ko JS, Matharoo-Ball B, Billings SD, Thomson BJ, Tang JY, Sarin KY, et al. Diagnostic distinction of malignant melanoma and benign nevi by a gene expression signature and correlation to clinical outcomes. Cancer Epidemiol Biomark Prev. 2017;26:1107–13.
Shinkins B, Thompson M, Mallett S, Perera R. Diagnostic accuracy studies: how to report and analyse inconclusive test results. BMJ. 2013;346:f2778
U.S. Food and Drug Administration. Guidance for industry and FDA staff: statistical guidance on reporting results from studies evaluating diagnostic tests. U.S. Food and Drug Administration; 2007.
Clarke LE, Warf BM, Flake DD 2nd, Hartman AR, Tahan S, Shea CR, et al. Clinical validation of a gene expression signature that differentiates benign nevi from malignant melanoma. J Cutan Pathol. 2015;42:244–52.
Cockerell CJ, Tschen J, Evans B, Bess E, Kidd J, Kolquist KA, et al. The influence of a gene expression signature on the diagnosis and recommended treatment of melanocytic tumors by dermatopathologists. Medicine. 2016;95:e4887.
Cockerell C, Tschen J, Billings SD, Evans B, Brown K, Rock, C, et al. The influence of a gene-expression signature on the treatment of diagnostically challenging melanocytic lesions. Per Med. 2017;14:123–30.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
LEC is employed by Myriad Genetic Laboratories, Inc., and SAL has participated in research supported by Myriad Genetic Laboratories, Inc.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Clarke, L.E., Leachman, S.A. Reply to Reimann et al.. Mod Pathol 32, 725–727 (2019). https://doi.org/10.1038/s41379-018-0197-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41379-018-0197-1