Probabilistic cell typing enables fine mapping of closely related cell types in situ


Understanding the function of a tissue requires knowing the spatial organization of its constituent cell types. In the cerebral cortex, single-cell RNA sequencing (scRNA-seq) has revealed the genome-wide expression patterns that define its many, closely related neuronal types, but cannot reveal their spatial arrangement. Here we introduce probabilistic cell typing by in situ sequencing (pciSeq), an approach that leverages previous scRNA-seq classification to identify cell types using multiplexed in situ RNA detection. We applied this method by mapping the inhibitory neurons of mouse hippocampal area CA1, for which ground truth is available from extensive previous work identifying their laminar organization. Our method identified these neuronal classes in a spatial arrangement matching ground truth, and further identified multiple classes of isocortical pyramidal cell in a pattern matching their known organization. This method will allow identifying the spatial organization of closely related cell types across the brain and other tissues.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Fig. 1: Detection of 99 genes in a mouse brain coronal section.
Fig. 2: Cell-type map of CA1 from an example experiment (experiment 4-3, right hemisphere).
Fig. 3: Validation of cell calling.

Data availability

Analysis files are available at and an interactive online viewer is at The raw image files are available from corresponding authors upon reasonable request. Source data for Figs. 1–3 are presented with the paper.

Code availability

Code of the ProMMT algorithm for gene selection is available at Code for probe design is available at MATLAB code for image analysis and cell typing is available at A Python version of the cell-calling algorithm, designed to work with StarFISH data standards, is available at All custom code is freely accessible.


  1. 1.

    Lein, E., Borm, L. E. & Linnarsson, S. The promise of spatial transcriptomics for neuroscience in the era of molecular cell typing. Science 358, 64–69 (2017).

  2. 2.

    Zeisel, A. et al. Brain structure. Cell types in the mouse cortex and hippocampus revealed by single-cell RNA-seq. Science 347, 1138–1142 (2015).

  3. 3.

    Tasic, B. et al. Adult mouse cortical cell taxonomy revealed by single cell transcriptomics. Nat. Neurosci. 19, 335–346 (2016).

  4. 4.

    Cembrowski, M. S., Wang, L., Sugino, K., Shields, B. C. & Spruston, N. Hipposeq: a comprehensive RNA-seq database of gene expression in hippocampal principal neurons. eLife 5, e14997 (2016).

  5. 5.

    Paul, A. et al. Transcriptional architecture of synaptic communication delineates GABAergic neuron identity. Cell 171, 522–539 (2017).

  6. 6.

    Harris, K. D. et al. Classes and continua of hippocampal CA1 inhibitory neurons revealed by single-cell transcriptomics. PLoS Biol. 16, e2006387 (2018).

  7. 7.

    Tasic, B. et al. Shared and distinct transcriptomic cell types across neocortical areas. Nature 563, 72–78 (2018).

  8. 8.

    Zeisel, A. et al. Molecular architecture of the mouse nervous system. Cell 174, 999–1014 (2018).

  9. 9.

    Shah, S., Lubeck, E., Zhou, W. & Cai, L. In situ transcription profiling of single cells reveals spatial organization of cells in the mouse hippocampus. Neuron 92, 342–357 (2016).

  10. 10.

    Cembrowski, M. S. & Spruston, N. Integrating results across methodologies is essential for producing robust neuronal taxonomies. Neuron 94, 747–751 (2017).

  11. 11.

    Shah, S., Lubeck, E., Zhou, W. & Cai, L. seqFISH accurately detects transcripts in single cells and reveals robust spatial organization in the hippocampus. Neuron 94, 752–758 (2017).

  12. 12.

    Freund, T. F. & Buzsaki, G. Interneurons of the hippocampus. Hippocampus 6, 347–470 (1996).

  13. 13.

    Pelkey, K. A. et al. Hippocampal GABAergic inhibitory interneurons. Physiol. Rev. 97, 1619–1747 (2017).

  14. 14.

    Somogyi, P. Hippocampus: intrinsic organization. in Handbook of Brain Microcircuits (Eds. Shepherd, G. M. & Grillner, S.) (Oxford Univ., 2010).

  15. 15.

    Wang, X et al. Three-dimensional intact-tissue sequencing of single-cell transcriptional states. Science 361, eaat5691 (2018).

  16. 16.

    Moffitt, J. R et al. Molecular, spatial and functional single-cell profiling of the hypothalamic preoptic region. Science 362, eaau5324 (2018).

  17. 17.

    Codeluppi, S. et al. Spatial organization of the somatosensory cortex revealed by osmFISH. Nat. Methods 15, 932–935 (2018).

  18. 18.

    Ke, R. et al. In situ sequencing for RNA analysis in preserved tissue and cells. Nat. Methods 10, 857–860 (2013).

  19. 19.

    Lein, E. S. et al. Genome-wide atlas of gene expression in the adult mouse brain. Nature 445, 168–176 (2007).

  20. 20.

    Eng, C.-H. L. et al. Transcriptome-scale super-resolved imaging in tissues by RNA seqFISH. Nature 568, 235–239 (2019).

  21. 21.

    Chen, K. H., Boettiger, A. N., Moffitt, J. R., Wang, S. & Zhuang, X. Spatially resolved, highly multiplexed RNA profiling in single cells. Science 348, aaa6090 (2015).

  22. 22.

    Pertuz, S., Puig, D., Garcia, M. A. & Fusiello, A. Generation of all-in-focus images by noise-robust selective fusion of limited depth-of-field images. IEEE Trans. Image Process 22, 1242–1251 (2013).

  23. 23.

    Hörl, D. et al. BigStitcher: Reconstructing high-resolution image datasets of cleared and expanded samples. Nat. Methods 16, 870–874 (2019).

  24. 24.

    Preibisch, S., Saalfeld, S. & Tomancak, P. Globally optimal stitching of tiled 3D microscopic image acquisitions. Bioinformatics 25, 1463–1465 (2009).

  25. 25.

    Elad, M. Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing (Springer-Verlag, 2010).

  26. 26.

    Robinson, M. D. & Smyth, G. K. Small-sample estimation of negative binomial dispersion, with applications to SAGE data. Biostatistics 9, 321–332 (2008).

  27. 27.

    Lu, J., Tomfohr, J. K. & Kepler, T. B. Identifying differential expression in multiple SAGE libraries: an overdispersed log-linear model approach. BMC Bioinformatics 6, 165 (2005).

  28. 28.

    Baddeley, A., Rubak, E. & Turner, R. Spatial Point Patterns: Methodology and Applications with R (CRC Press, 2015).

  29. 29.

    Bishop, C. M. Pattern Recognition and Machine Learning (Springer, 2006).

Download references


We thank P. Somogyi, M. Carandini, S. Linnarsson, M. Hilscher, N. Kessaris and L. Magno for valuable discussions. We thank K. Karlsson for providing scRNA-seq reads for Cxcl14 gene. This work was supported by grants from the Wellcome Trust (108726, to K.D.H., J.H.L. and M.N.), Chan–Zuckerberg Initiative (182811 to K.D.H.), the Swedish Research Council (2016-03645 to M.N.), Knut och Alice Wallenbergs Stiftelse (to M.N.) and Familjen Erling-Perssons Stiftelse (to M.N.).

Author information

X.Q. wrote the DNA probe design software, performed experiments, analyzed data, designed the in situ sequencing protocol, prepared figures and wrote the manuscript. K.D.H. conceived the study, designed and wrote analysis software and wrote the manuscript. T.H. designed the in situ sequencing protocol. D.N. designed and wrote the online web viewer, performed simulations and wrote a Python translation of the cell-calling code. A.B.M.-M. designed tissue preparation protocols and provided samples. N.S. contributed to gene panel selection. J.H.-L. conceived the study and supervised tissue sample preparation and collection. M.N. conceived the study, designed the in situ sequencing protocol, supervised experiments and wrote the manuscript.

Correspondence to Kenneth D. Harris or Mats Nilsson.

Ethics declarations

Competing interests

X.Q., T.H. and M.N. hold shares in Cartana AB, a company that commercializes in situ sequencing reagents.

Additional information

Peer review information Rita Strack was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–19, Supplementary Methods, Supplementary Discussion and Supplementary Results.

Reporting Summary

Supplementary Table 1

Gene selection for pciSeq.

Supplementary Table 2

Probe sequences.

Supplementary Table 3

Number of reads detected in each coronal section.

Supplementary Table 4

All scRNA-seq cell types and their prior probability, and superclasses used in pciSeq.

Supplementary Table 5

Number of cells of each superclass in individual hippocampus

Source data

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Qian, X., Harris, K.D., Hauling, T. et al. Probabilistic cell typing enables fine mapping of closely related cell types in situ. Nat Methods 17, 101–106 (2020).

Download citation

Further reading

  • A Spatiotemporal Organ-Wide Gene Expression and Cell Atlas of the Developing Human Heart

    • Michaela Asp
    • , Stefania Giacomello
    • , Ludvig Larsson
    • , Chenglin Wu
    • , Daniel Fürth
    • , Xiaoyan Qian
    • , Eva Wärdell
    • , Joaquin Custodio
    • , Johan ReimegÃ¥rd
    • , Fredrik Salmén
    • , Cecilia Österholm
    • , Patrik L. StÃ¥hl
    • , Erik Sundström
    • , Elisabet Åkesson
    • , Olaf Bergmann
    • , Magda Bienko
    • , Agneta MÃ¥nsson-Broberg
    • , Mats Nilsson
    • , Christer Sylvén
    •  & Joakim Lundeberg

    Cell (2019)