Reproducibility of histopathological findings in experimental pathology of the mouse: a sorry tail

Abstract

Reproducibility of in vivo research using the mouse as a model organism depends on many factors, including experimental design, strain or stock, experimental protocols, and methods of data evaluation. Gross and histopathology are often the endpoints of such research and there is increasing concern about the accuracy and reproducibility of diagnoses in the literature. To reproduce histopathological results, the pathology protocol, including necropsy methods and slide preparation, should be followed by interpretation of the slides by a pathologist familiar with reading mouse slides and familiar with the consensus medical nomenclature used in mouse pathology. Likewise, it is important that pathologists are consulted as reviewers of manuscripts where histopathology is a key part of the investigation. The absence of pathology expertise in planning, executing and reviewing in vivo research using mice leads to questionable pathology-based findings and conclusions from studies, even in high-impact journals. We discuss the various aspects of this problem, give some examples from the literature and suggest solutions.

Main

Histopathological descriptions of the frequency and nature of lesions and disease entities are very often the endpoints in biomedical research conducted in model organisms such as the mouse. In contrast with clinical pathology, where endpoints are usually assessed using biochemical and molecular assays, histopathological assessment, despite using molecular markers and imaging as adjunct qualitative and quantitative techniques, is highly dependent on the individual expertise of trained expert pathologists. Pathologists must not only recognize lesions, but also have knowledge of the background diseases of the mice and understand the meaning of the pattern of disease in the whole mouse^1,2,3,4. Reproducibility of histopathological endpoints therefore depends on the implementation of a common standardized vocabulary, competent work-up and an in-depth knowledge of the mouse strains under investigation so that, for example, background lesions are not mistaken for those that are experimentally induced. Such knowledge is critical in the design of experiments, as well as in understanding the effects of husbandry, the microbiome and diet on the interpretation of results^2,5.

In recent years, funding agencies and scientific communities alike have expressed increasing concern about the lack of reproducibility of experiments in the biomedical domain. Attention was initially drawn to this issue by pharmaceutical companies, which rely on preclinical, precompetitive research for drug development pipelines^6,7,8. Identification of this problem has been followed by an outpouring of concern from funding agencies such as the US National Institutes of Health^9,10,11 and, to an extent, journals and professional bodies^{12,13,14,15,16,17}.

Although much attention has been paid to the reproducibility of molecular assays, in vitro (cell culture) assays and the inappropriate application of statistical methods, only recently have the issues surrounding reproducibility in animal experimentation been discussed in depth¹³. Many of these discussions have concerned husbandry and the effect of diet and microbiome on experimental outcomes^18,19,20, particularly in neuroscience¹⁴. However, recent papers have addressed the problem of what a sound histopathological investigation should look like, how to use knowledge of pathology in experimental design, based on the ARRIVE and related guidelines, and the confounding effect of the environment and the gut and skin microbiomes⁵.

Here we address some of the issues that affect reproducibility of histopathological findings: a lack of pathology expertise, in author lists and in peer review; poor standards of reporting, as illustrated by common errors seen in papers, and inconsistent pathology nomenclature; and availability of primary data, without which it is impossible to assess a paper without attempting a complete experimental replication^21,22,23. Notably, we emphasize that if pathologists are not involved in designing mouse experiments and interpreting lesions, the accuracy of the diagnoses reported and the conclusions drawn may be questionable.

The importance of pathologists

Pathology is a medical specialty that requires years of training, experience and board certification as a minimum. Although pathology has many sub-disciplines, such as mouse pathology, a general pathologist is much more of an expert than a non-pathologist, and is often sufficient to provide substantial benefit to an animal research study²⁴. However, investigators often do not have enough funds to pay for research pathology services and/or believe that they can perform histopathology interpretations themselves. A lack of pathology expertise by investigators leads to inaccurate histopathological descriptions of lesions and often missed, or spurious reporting of, pathological findings in publications. Absence of a pathologist may be noticed in the figure legends, which often do not describe the lesions displayed, or, in some rare cases, in images that are replicated in various orientations for different lesions of mice²⁵. In some cases, a pathologist was not involved in late of final edits or did not review the galley proofs of an accepted manuscript²⁶, leading to a substantial error in reporting.

In addition to accurate interpretation of data, pathologists are important for ensuring that proper nomenclature is used when reporting on results. The use of generally accepted pathology nomenclature for unexpected and new findings leads to publications that can be interpreted by readers, including other pathologists. Rodent pathology terminology often mirrors that used for humans, but there are differences. The pathology of genetically engineered mice often requires interpretation of new findings, as each mouse may have unique lesions that have not been previously reported, especially when the study is the first for a new gene knockout or treatment. A classic example is the relatively common lesion in mouse hearts that pathologists diagnose as epicardial and myocardial mineralization and fibrosis, but non-pathologists often call dystrophic cardiac calcinosis or a variety of other names^{27,28,29,30,31}. Investigators without pathology backgrounds often over-interpret their research findings, with the temptation being to fit results to their hypotheses. Over-diagnosis of lesions as malignant when they may be, in fact, benign, hyperplastic or even normal is a common problem. This latter point emphasizes the value of knowing the anatomical differences between the species.

Some examples of errors seen in reported histopathological diagnoses

In addition to incomplete reporting of the experimental design, including the pathology protocol, there are common questionable diagnoses that can be found in published results (Table 1)^32,33,34. Often, the figure legends do not describe what is illustrated by the figure, normal tissues are misidentified as lesions, non-neoplastic lesions are reported as cancer or benign lesions are diagnosed as malignant neoplasms. In addition, inflammatory lesions may be described as neoplasms, benign or malignant.

Table 1 Evidence of questionable pathology interpretation in publications

Our interpretations of histopathology figures in published reports, which we will give as examples, are based solely on our interpretation of what was present in the published figures and are not based on microscopic slide review, which may reveal different findings than what is in the published figures. Often, the published histopathology figures are small, and they can lose resolution when enlarged, to the point of being uninterpretable. One of many approaches to this problem is to post additional digital images at a variety of magnifications or whole-slide images as supplemental data. Images could also be posted on public websites such the Mouse Tumor Biology Database^35,36, Gene Expression Database³⁷, Pathbase^38,39 and many others⁵.

To evaluate any organs, a clear understanding of the normal anatomy is absolutely necessary to recognize any type of change, be it disease or just subtle changes in normal physiology. When evaluating the skin, the normal hair cycle is a commonly reported source of misinterpretation. All hair follicles regularly go through anagen, the normal growth phase; catagen, the transition stage to telogen, the long-term resting stage; to exogen, when the old hair shaft is lost. This process then starts over again and is repeated throughout life. The cycle varies by hair type (vibrissae cycle is different compared with body hair) and species (mice cycle in waves, whereas humans cycle in a mosaic pattern)¹. The hypodermal fat layer in the skin changes thickness through the cycle. When thinnest, during the telogen phase⁴⁰, this is often reported as an abnormal phenotype. Sebaceous gland size also changes through the hair cycle, making estimation of the size of this gland an unpredictable feature that is also commonly misinterpreted⁴¹. Changes in the numbers of hair follicles can be misinterpreted owing to artifacts of section orientation (Fig. 1)⁴².

Figure 1: Skin of mouse in anagen with abundant hair follicles.

This normal stage of the cell cycle has been reported to be a marked increase in hair follicles, when in fact it is an orientation artifact. Hematoxylin and eosin stain, 10× magnification.

Male mice have modified sebaceous glands (with a large excretory duct along the penis) known as preputial glands; these are also known as clitoral glands in the inguinal area of females (Fig. 2). These tissues have been diagnosed as teratomas or skin tumors^33,43,44, and an erratum has been published for one of the publications in question⁴³. Mouse accessory sex glands include various prostate lobes, seminal vesicles and other structures, the architecture of which differs from that of humans. Tissue artifacts have been diagnosed as early stage prostate cancer⁴⁵ (Fig. 3).

Figure 2: Normal mouse preputial gland showing glandular tissue with central ducts.

Two publications reported normal glands as teratomas or carcinomas. Hematoxylin and eosin stain, 4× magnification.

Figure 3: Normal prostate of young mouse with artifactual folds of the acinar epithelium that were misdiagnosed as early stage prostate cancer in a publication.

Hematoxylin and eosin stain, 40× magnification.

Immunohistochemistry findings can also be problematic in publications with pathology results. Owing to omission of proper controls, authors often report positive labeling (often with a brown chromogen) of cells and tissues that appear to represent nonspecific background staining⁴⁶. A good example was reported in mouse prostate epithelial cells and connective tissue⁴⁷ using an anti-human antibody that has never been shown (even by the company selling the antibody) to work in mouse tissues. Mast cells are often nonspecifically positive in mouse tissues using peroxidase-based reagents⁴⁸.

When is a neoplasm not a neoplasm?

Knowledge of appropriate nomenclature is also important for accurate reporting. Neoplasms and their preneoplastic/precancerous lesions are commonly found in mouse experiments involving chemical carcinogens and/or in genetically engineered mice. Many papers involving mice with these induced or spontaneous lesions do not refer to the publications that focus on standardized nomenclature for the organ or disease under investigation, such as those noted above and in our reference list. Although many journals list in their 'instructions for authors' that they require authors to use standardized nomenclature, this requirement is often not enforced by editors. This policy holds not only for diagnostic terms, but also for mouse strain and allele designations, as mouse genetic nomenclature is very uniformly standardized^49,50. Some examples of questionable diagnoses of preneoplastic and neoplastic lesions of mice are given below.

The most widely used prostate cancer mouse model, commonly called TRAMP, is an example of the misuse of standardized nomenclature. A search of Mouse Genome Informatics (http://www.informatics.jax.org/genes.shtml) yielded four matches (Table 2), only one of which is the transgenic line used for prostate cancer research: Tg(TRAMP)08247Ng⁵¹. With over 600 publications, this transgenic line is often considered to be the best mouse model of human prostate cancer. The Tg(TRAMP)08247Ng line was noted to have a high incidence of prostate adenocarcinoma metastases, suggesting that this was a good model for human prostate cancer. However, these were later shown to be of neuroendocrine origin rather than epithelial, with non-neuroendocrine epithelial being the most common in humans (Fig. 4)^51,52,53. Phylloides prostate carcinomas, initially diagnosed in the first Tg(TRAMP)08247Ng publication, were later shown to be adenomas or benign epithelial-stromal tumors of the seminal vesicles (Fig. 5)^51,54. Metastatic prostate carcinoma to bone marrow was described in a new genetically engineered mouse model, but a pathology nomenclature consensus committee determined that the cases were merely direct invasion from a large prostate mass^51,55.

Table 2 Partial table from Mouse Genome Informatics (Accessed 19 Sept. 2016) to illustrate three genes and one transgene identified by the term TRAMP, three of which are unrelated genes

Figure 4: Neuroendocrine carcinoma in the prostate of a TRAMP (Tg(TRAMP)8247Ng) mouse.

These mice were reported to develop highly metastatic prostate adenocarcinoma. Hematoxylin and eosin stain, 40× magnification.

Figure 5: Benign (epithelial-stromal) tumor in the seminal vesicles of a TRAMP mouse.

Note neoplastic growth into the lumen and no invasion. These lesions were reported as phylloides prostate carcinomas. Hematoxylin and eosin stain, 4× magnification.

Inflammatory lesions caused by bacteria have sometimes been reported as tumors (neoplasms)⁵⁶, as have been other types of inflammation⁵⁷. This may be technically correct, as some textbooks define tumor literally as any type of swelling and one of the five cardinal signs of inflammation, but inflammation should and can be easily differentiated from neoplasia. Lymphomas and leukemias are often difficult to diagnose accurately. Tail tumors in transgenic mice were reported to be large granular lymphocytic (LGL) leukemia⁵⁸, but other investigators working with the same mice found sarcomas of various types, including those arising in tendons and nerves in the tail (Figs. 6, 7, 8). These tail tumors may have developed accompanying inflammatory responses that included LGLs. Large spleens have been diagnosed as myeloproliferative disorders and leukemias, especially in mice with ulcerative skin lesions that cause reactive myeloid hyperplasia in the spleen⁵⁹. Using a Helicobacter felis mouse gastric model, a research group developed a model of chronic gastritis that eventually was reported to develop gastric lymphomas. These changes appeared histologically unconvincing as described in the initial publication. In this case, however, a subsequent publication did provide molecular proof that these were indeed lymphomas⁶⁰.

Figure 6: Tail of a HTLV-I tax transgenic mouse with early neoplasms (on left and right side) of tendon origin.

This mouse was reported to develop leukemia and not tendon neoplasms. Hematoxylin and eosin stain, 4× magnification.

Figure 7: Tail of a HTLV-I tax transgenic mouse with an early tumor of tail tendon origin (darker tumor in the lower portion of the figure beneath the skin).

This mouse was reported to develop leukemia. Hematoxylin and eosin stain, 4× magnification.

Figure 8: Early tail tendon sarcoma showing tumor infiltration by myeloperoxidase-positive neutrophils.

Immunoperoxidase, 40× magnification.

How to increase reproducibility by improving nomenclature usage

Problems in pathology evaluation may occur at various stages of the study, leading to questionable pathology interpretation in the submitted or published manuscript. To increase reproducibility in mouse studies, a trained pathologist knowledgeable in rodent pathology nomenclature should be involved, in study design, manuscript writing and/or chosen by the editors during the peer-review process. Pathology nomenclature in the paper should follow general guidelines for mouse pathology, as published by international committees and experts.

As discussed above, the ability of a pathologist to accurately diagnose lesions in laboratory animals, especially rodents, depends on training and experience. Experience includes the use of widely acceptable veterinary pathology and species-specific nomenclature that is often provided through publications by expert groups of pathologists and by international or national committees^{33,61,62,63,64,65} and in books with multiple authors^{61,66,67,68,69}. Others have proposed formal ontologies for data capture and analysis^39,70,71, which are also based on international nomenclatures and informatics standards.

There are numerous publications on neoplastic diseases in mice, especially in genetically engineered mice. The National Cancer Institute Mouse Models of Human Cancer Consortium has established pathology committees over the past 15 years to develop nomenclatures for several important organs^33,51,61. The publications on neoplastic diseases in mammary gland, prostate, lung, intestine, brain, skin and pancreas provide important guidelines for investigators. The INHAND pathology nomenclatures have similarly created detailed terminological recommendations for both proliferative and non-proliferative lesions under the auspices of the committees established by a consortium of societies of toxicologic pathology⁶³. These, together with detailed publications on individual classes of lesion, make up a substantial terminological corpus with which pathologists making diagnoses should be familiar.

Conclusion

Compared with factors confounding the reproducibility of mouse biological experiments originating in experimental design, husbandry and microbiome, the problem of reliable histopathological interpretation of experimental animals is perhaps one of the most tractable sources of error. Enrollment of an experienced pathologist onto a study early in its inception and planning stages and then its subsequent analysis is clearly highly desirable, as many of the problems we discussed above are unlikely to arise under the guidance of appropriately trained personnel. The issue of finding experienced mouse pathologists has been discussed at length elsewhere^2,24,34, although the authors feel that the mouse pathology community is sufficiently interactive that good advice can easily be sought out by motivated investigators.

Changes in priorities at journals and funding agencies are also needed to substantially improve the reliability of pathology in mouse model studies. The availability of the primary images on which experimental conclusions are based should be mandatory at journals and consistent with the FAIR guidelines^5,72. Similarly, funding agencies need to pay more attention to the intended use of histopathology in grant applications and insist on provision of appropriate expertise with an appropriate budget. Increased stringency surrounding the processes of funding and publishing studies might represent more effort for researchers, reviewers and journal editors, but will reduce the instances of flawed histopathology that we see in many journals today⁷³.