# Biology and Bias in Cell Type-Specific RNAseq of Nucleus Accumbens Medium Spiny Neurons

## Abstract

Subcellular RNAseq promises to dissect transcriptional dynamics but is not well characterized. Furthermore, FACS may introduce bias but has not been benchmarked genome-wide. Finally, D1 and D2 dopamine receptor-expressing medium spiny neurons (MSNs) of the nucleus accumbens (NAc) are fundamental to neuropsychiatric traits but have only a short list of canonical surface markers. We address these gaps by systematically comparing nuclear-FACS, whole cell-FACS, and RiboTag affinity purification from D1- and D2-MSNs. Using differential expression, variance partitioning, and co-expression, we identify the following trade-offs for each method. RiboTag-seq best distinguishes D1- and D2-MSNs but has the lowest transcriptome coverage. Nuclear-FACS-seq generates the most differentially expressed genes and overlaps significantly with neuropsychiatric genetic risk loci, but un-annotated genes hamper interpretation. Whole cell-FACS is more similar to nuclear-FACS than RiboTag, but captures aspects of both. Using pan-method approaches, we discover that transcriptional regulation is predominant in D1-MSNs, while D2-MSNs tend towards cytosolic regulation. We are also the first to find evidence for moderate sexual dimorphism in these cell types at baseline. As these results are from 49 mice (nmale = 39, nfemale = 10), they represent generalizable ground-truths. Together, these results guide RNAseq methods selection, define MSN transcriptomes, highlight neuronal sex differences, and provide a baseline for D1- and D2-MSNs.

## Introduction

The ability to isolate individual populations of cells with a homogeneous molecular profile has allowed biological inquiry to address new, more refined regulatory functions. Studies are parsing biological signals from complex, heterogeneous tissues and thereby revealing effects masked by cell type variability1,2,3,4. This work brings us one step closer to understanding the quantum nature of biology and to developing therapeutics that leverage its heterogeneity, rather than ones that lose efficacy as a consequence of it. We report here a comprehensive, technical comparison of cell isolation methodologies for RNAseq, and leverage the biological context of our study to provide new insight into the function of D1- and D2-type medium spiny neurons (MSNs) of the nucleus accumbens (NAc), part of the ventral striatum.

Several techniques exist that allow isolation of cell type-specific RNA. Some depend on FACS, using fluorescent labels to sort cell populations into discrete tubes. These FACS-based techniques can operate on either whole cells or on purified nuclei. On the other hand, methods like RiboTag affinity purification depend on cell type-specific genetically tagged ribosomal subunits, which can be immunoprecipitated in order to obtain samples of cell type-purified RNA. Importantly, the RNA isolated by such ribosomal affinity purification methods represents mRNA undergoing active translation at the ribosome.

While these techniques have been widely used in various tissues, they have not been systematically compared in a well-controlled framework for brain. There are substantial differences between methods; for example, whole cell-FACS and RiboTag extract cells from fresh tissue, while nuclear-FACS does so typically from frozen tissue; RiboTag and nuclear-FACS use a mechanical dissociation protocol, while whole cell-FACS uses an enzymatic one; and each recovers a different subpopulation of RNA species. Based on the literature and on procedural documentation for these methods, we hypothesize that they uncover related but only partially overlapping gene sets and biological functions.

Unlike library preparation methods for whole-genome and whole-transcriptome analysis, which have been carefully compared and assessed for biasing effects5,6,7,8, cell isolation protocols have been inconsistently compared and have yielded conflicting results. Several studies have analyzed differences generated by isolation of either whole cells or nuclei, but these studies are often performed in cell culture6,9, which lacks the heterogeneity and variability of in vivo tissues, and they draw limited conclusions about the biological types of transcripts isolated by each technique. Comparative studies on more complex tissues confine their interpretation to effects of each technique on a subset of differentially expressed genes, transcript length or RNA biotype10.

Few RNAseq studies have included a comparison between FACS-isolated cells/nuclei and ribosomal purification techniques like TRAP or RiboTag, and therefore ignore the important question about which RNA species are isolated by the latter methods that capture active translation11,12,13. Those studies that include such a comparison have relatively simple biological endpoints like method repeatability and contamination, and lack the comprehensiveness of a whole-genome analysis6,10,14.

The absence of a methodical comparison of whole cell-FACS, nuclear-FACS, and RiboTag affinity purification is becoming increasingly problematic as an increasing number of studies using these techniques are published and their results taken at face value. These techniques capture different cellular processes while simultaneously defining the same cellular identity. Only a head-to-head comparison for the same cell types can demonstrate if they predominantly capture differences or similarities, and identify the nature of those biases. The present study was designed to address this deficiency in the field.

Our two cell types of choice are both GABAergic MSNs of the NAc, a forebrain region implicated in reward and motivation. Both MSN subtypes respond to dopamine, but do so through the activity of different dopamine receptors15,16, display different physiology in response to reward-related stimuli17,18,19, and generate different behavioral outcomes20,21,22,23,24,25,26,27. Whole cell-FACS15 and TRAP11 have been used to distinguish between D1- and D2-MSNs, but the two methods have not been compared directly and these prior studies focused on the entire striatal complex, of which the NAc represents a small sub-region. Here, we use all three RNA isolation methods – whole cell-FACS, nuclear-FACS, and RiboTag – to provide a deeper characterization of these behaviorally relevant NAc cell types than ever before. Most importantly, this study presents the first genome-wide, biological network-focused analysis of the contribution of these three methods to RNA characterization.

## Results

### Comparison of library complexity and distribution

As noted in the Introduction, whole cell-FACS, nuclear-FACS, and RiboTag affinity purification have methodological differences and retrieve different subcellularly-located RNAs. We provide a schematic of the important technological differences in sample preparation in Fig. 1a. Each method has unique elements: nuclear-FACS can be performed on frozen tissue, whole cell-FACS uses enzymatic – not mechanical – dissociation, and RiboTag affinity purification uses immunoprecipitation as opposed to FAC sorting. To compare the ability of these methods to distinguish D1- and D2-MSN populations from the NAc, we generated RNAseq libraries using ribo-depleted, total RNA isolated from the NAc of individual D1- or D2-Cre mice (see Methods). The libraries were prepared using the same kit and sequenced on the same platform using the same parameters. Thirty-nine samples were used for downstream analyses, constituting 16 whole cell-FACS (D1 n = 9, D2 n = 7), 11 nuclear-FACS (D1 n = 6, D2 n = 5), and 12 RiboTag (D1 n = 6, D2 n = 6) samples. We first confirmed that nuclear sequencing produces a larger percentage of intronic reads (Supplementary Fig. 1a), consistent with previous findings5,28.

Differential RNA expression between D1 and D2 MSNs was performed within each method, and signatures were compared across methods. With typical relaxed filtering parameters (≥1 fragments per million (FPM) in ≥2 samples and differential expression padjusted ≤ 0.05, 30% change, top 75% of mean size factor normalized count), overlap in differentially expressed genes (DEGs) between conditions was not very high (Supplementary Fig. 1b), and the nuclear DEG pool was large and non-specific. Based on this nuclear DEG observation, we implemented a strict filter to remove genes with very low and very high value fragments per kilobase per million (FPKM; keeping genes with FPKM ≥1 in at least one sample and ≤5 × 104 in all samples, Supplementary Fig. 1c). These filters reduced intronic and intergenic noise29 and eliminated artifacts of sample-specific overamplification of short-length genes. In contrast to nuclear and whole cell libraries, the RiboTag method had more lowly-expressed genes (Fig. 1b), necessitating different filtering cut-offs. This is consistent with previous literature30. We therefore implemented a less strict low-FPKM filter for the RiboTag dataset balanced by requiring observations in multiple samples (FPKM ≥0.1 in ≥2 samples). This filter dramatically increased RiboTag overlap compared to that found using the stringent FPKM filter for all methods (Supplementary Fig. 1d). These empirically-determined FPKM filtering parameters optimized comparison between the conditions, generating similar transcriptome-wide FPKM distributions across methods (Fig. 1c).

Nuclear DEGs appeared to still be somewhat non-specific, and we ran a logistic regression to determine parameters contributing to nuclear DEG overlap vs. non-overlap (Supplementary Table 1). Perhaps due to increased variance in the set of nuclear DEGs (Fig. 1g), StandardError(log2foldchange) was significantly higher in non-overlapping nuclear DEGs compared to overlapping ones (logistic regression β = 5.4, ppost-hoc = 2.0 × 10−2). We therefore removed nuclear DEGs within the top quartile of StandardError(log2foldchange), greatly improving the specificity of this pool (Fig. 1d) and generating a robust population of cross-method D1-MSN- and D2-MSN-enriched genes, whose fold change followed similar patterns across methods (Fig. 1e).

More genes were differentially expressed between the two cell types in the nuclear (2,361) compared to the whole cell (1,416) and RiboTag (133) datasets (Fig. 1d, Supplementary Table 2). These nuclear DEGs included genes of a wide variety of coding and non-coding biotypes (Fig. 1g), and the percentage of protein-coding DEGs increased steadily from nuclear (61.59%) to whole cell (69.52%) to RiboTag (82.71%). This bias towards protein-coding genes in the RiboTag dataset manifests before the implementation of differential expression, as demonstrated by the method’s relatively lower capture of short-length genes across the transcriptome, an effect which disappears when analyzing only protein-coding genes (Supplementary Fig. 1e).

Density plots of whole cell and RiboTag DEG average FPKMs within D1-MSNs and D2-MSNs did not show a detectable difference in distribution between cell types (Fig. 1e, Supplementary Table 3). However, in the nuclear dataset, there was a notable difference between cell types, with D2-enriched DEGs having a lower average FPKM than D1-enriched DEGs. Given its lower expression level and small overlap with RiboTag D2-DEGs, the D2 nuclear profile may represent stochastic transcriptional noise in D2-MSNs which, at baseline, does not correspond to active translation, but which poises cells to respond to future stimuli.

Nuclear DEGs had a higher median log(FPKM + 1) (0.76) than whole cell (0.61) or RiboTag (0.24) DEGs, as well as a higher median variance (nuclear = 0.24, whole cell = 0.13, RiboTag = 0.02) (Fig. 1f). This is largely driven by the higher median variance of nuclear D2-DEGs (0.31) than nuclear D1-DEGs (0.21). Importantly, within genes passing the FPKM filter in all conditions, FPKM is significantly correlated across method, as is variance (Supplementary Table 4).

Finally, we demonstrate that these data can be used to model RNA regulatory dynamics across cellular compartments, extending previous work with ribosomal and whole cell data31. Using a negative binomial generalized linear model (Supplementary Fig. 2a), effects of transcriptional, cytosolic and translational regulation were deduced. Genes undergoing one or more of these levels of regulation were overlapped (Supplementary Fig. 2b). The largest overlap was seen for genes transcriptionally enriched in D1 nuclei, but cytosolically enriched in D2-MSNs. Upstream miRNA analysis of this list using miRTarBase32 generated 27 mouse miRNAs (Fisher’s exact test two-sided padjusted < 0.05), and 4 of these were differentially enriched in D2-nuclei but not in D2-whole cells (Supplementary Fig. 2c), pointing to cell type-specific mechanisms of subcellular transcriptional regulation. Numbers of genes falling into each category of regulation suggest that D1-MSNs undergo predominantly transcriptional and translational regulation, while D2-MSNs undergo predominantly cytosolic regulation.

### Variance partitioning of the combined datasets

Having assessed the transcriptome-coverage and basic library composition for each method, we analyzed the major sources of variance between and within the three datasets. A principal component analysis revealed strong separation of samples by method (Fig. 2a), with RiboTag samples differing the most from the other two methods, and clustering together most tightly. The Euclidean distances between method cluster centroids were higher for RiboTag (d (WholeCell, Nuclear) = 41, d (RiboTag, Nuclear) = 75, d (RiboTag, WholeCell) = 69) and the mean Euclidean distances within method to each cluster centroid were significantly smaller for RiboTag ($${\bar{d}}_{{\rm{RiboTag}}}$$ = 1.7, $${\bar{d}}_{{\rm{WholeCell}}}$$ = 5.9, $${\bar{d}}_{{\rm{Nuclear}}}$$ = 10.9, two-sided t-test p < 5 × 10−5 for both RiboTag comparisons, p = 0.005 for WholeCell vs Nuclear). Samples from each method separated clearly by cell type (Fig. 2a, insets), and this was also seen in PC5/PC6 of the combined data (Supplementary Fig. 3a). Method accounted for more variability than cell type, though this is likely due to the similarity of the two cell types studied. When comparing our datasets to published RiboTag RNAseq from liver cells30, most of the variability in the combined data (PC1, 50.5%) was due to tissue, while less variability (PC2, 30.6%) separated RiboTag datasets from whole cell and nuclear (Supplementary Fig. 3b)

Variation within the total data was further explored using variance partitioning analysis (Fig. 2b). Variance in expression of a given gene that was not explained by method of separation or by cell type was categorized as ‘residual’. On average, 65.3% of transcriptome-wide variance is explained by residual factors, which could include parameters such as the duration and method of trituration7,33,34. This number goes down to 53.6% when only analyzing genes found in all datasets (Supplementary Fig. 3c, Supplementary Table 5). Method accounted for 32.0% of transcriptional variance, and such method-variable genes enriched for expected GO terms relating to cellular localization, which likely has to do with the range of biotypes represented in this gene list (Supplementary Table 6). Our biological variable of interest (cell type) exerted a small effect on transcriptional variance across samples, with only 2.7% of variance explained by cell type (Fig. 2b). This is consistent with the fact that D1- and D2-MSNs are highly similar neurons as noted earlier, which in fact cannot be readily distinguished by PCA on single cell RNAseq of whole striatum35. Interestingly, Gene Ontology (GO) analysis of this small list of cell type-variable genes using g:Profiler36,37 showed enrichment for expected functions relating to dopamine signaling (Fig. 2b, Supplementary Table 7). This striking – though unsurprising – result highlights the importance of normalizing the method of cellular separation in order to reduce noise in a given dataset and magnify the focus on biological variability.

The top 20 method-variable genes are displayed along with the z-score of their library-normalized expression across method (Fig. 2c). It is interesting to note that many of these top method-variable genes are uncharacterized (8/20, 40%). The percentage of annotated genes steadily increased as the list length increased, demonstrating a bias towards unannotated genes among the most highly method-variable transcripts.

Whole cell- and nuclear-variable genes largely cluster together (Fig. 2c). RiboTag-variable genes comprise a largely non-overlapping pool, though some are shared with whole cell. Future experiments using only one of the methods included here, but seeking to discover consistent biology across cellular compartments, can reduce the technological bias in their data by removing method-enriched genes identified in this study (Supplementary Table 8).

Using RNAlocate, a database of subcellular RNA transcript localization generated across tissues and biological timepoints38, we investigated cellular distribution of method-variable genes (Fig. 2d). Our results show the expected subcellular distribution of these genes based on the compartments for which each method respectively enriches. We demonstrate the bias of nuclear isolation towards nuclear-located transcripts, and RiboTag purification towards ribosomal transcripts, with whole cell showing an intermediate profile. Interestingly, the nuclear-variable list showed the highest percentage of exosomally-located transcripts.

### Method-associated topological networks

Variance partitioning analysis revealed genes whose transcriptional variability was determined most by method and cell type. Because many of the method-variable genes were unannotated, potential for further biological interpretation of them was limited. Therefore, in order to generate more functional conclusions about method-related genes, we employed Weighted Gene Co-expression Network Analysis (WGCNA)39, which generates transcriptional networks based on the co-expression of genes across a population of samples. For every network module calculated with WGCNA, the eigengene (i.e., first principal component) was used to capture the relative expression changes observed in each module across samples. These eigengenes were further clustered and correlated with each other as well as cell type, method, and/or sex.

Network modules were generated by collapsing across all methods, which resulted in 9 unique modules. Of these 9, correlation analysis discovered one RiboTag-associated module, one nuclear-associated module, and one whole cell-associated module. The top 20 hub genes and their first level connections were extracted in order to generate “hub networks” for modules of interest (see Methods), and these hub networks were probed for functional and biological enrichments using g:Profiler (Supplementary Table 7).

The RiboTag-associated module enriched for terms relevant to protein-coding activity (GO:0005515, padjusted = 1.48 × 10−14; GO:0006412, padjusted = 9.02 × 10−4; KEGG:03010, padjusted = 2.21 × 10−3; REAC:R-MMU-72702, padjusted = 1.29 × 10−2) (Supplementary Table 7). Many of the top twenty hub genes in this network are involved in crucial neuronal functions: organization of DNA (Hist2h2aa2, Zbtb7a, Hcfc1, Hnrnpa3, Chd3), regulation of cell growth (Celsr2, Clstn1), microtubular organization (Pcdhgc4, Gphn), and protein quality control (Vcp, Bag6). Their centrality in the RiboTag-associated hub network is congruent with their high levels of expression (all above the 70th percentile of expression by FPKM across method) and their importance in baseline cellular functioning.

The whole cell module enriched for terms relating to cellular energetics (GO:0022900, padjusted = 3.52 × 10−6; GO:0006119, padjusted = 1.14 × 10−5; GO:0044265, padjusted = 3.61 × 10−3) and to RNA processing (GO:0003723, padjusted = 6.04 × 10−5; REAC:R-MMU-72163, padjusted = 1.12 × 10−2). In comparison, the nuclear module enriched for behaviorally relevant terms (GO:0007610, padjusted = 8.17 × 10−4; GO:0050890, padjusted = 4.57 × 10−3) and “Voltage-gated potassium channels” (REAC:R-MMU-1296072, padjusted = 1.26 × 10−2), highlighting the importance of transcriptional control as a means of regulating K+ channel activity40,41,42, as well as perhaps the function of nuclear K+ channels in transducing signals of neuronal activity to the nucleus43,44.

Using data from the NHGRI-EBI genome-wide association study (GWAS) Catalog45, we overlapped hub networks with known risk loci for neuropsychiatric diseases (Fig. 3 and Supplementary Table 9). In the nuclear hub network, we found a significant overall enrichment (Fig. 3d, OR = 5.36, Fisher’s exact test two-sided p = 3.23 × 10−3, 5/146 term genes). Neither of the other two method-associated hub networks showed overall enrichment for neuropsychiatric GWAS hits (Fig. 3b,c). The nuclear hub network enrichment was driven by genes implicated in Alzheimer’s disease (APOE), non-alcoholic drug-related phenotypes (RAB4B-EGLN2), and cognitive ability (LRRN2 and MAPT). This demonstrates an ability of the nuclear method, over and above the other two, to detect disease-relevant genes.

### Cell type-associated topological networks

To investigate the ability of each method to characterize these biologically similar cell types, we performed WGCNA separately on samples from each method (Figs 4a and 5a). Within each method, hub networks were derived, as above, for modules most correlated to cell type eigengenes. These networks are displayed in Figs 4b and 5b, with node size scaled for relative, within-network degree of connectivity, and hub genes labeled.

Despite a small overlap in hub network genes across methods for D1-MSNs (73/1622, 4.5%) and D2-MSNs (22/1713, 1.3%) (Supplementary Fig. 4a), convergent conclusions can be made from the separate techniques. These hub networks were characterized using g:Profiler (Supplementary Table 7) and significantly enriched GO, KEGG and Reactome pathway terms are displayed (Figs 4c and 5c). Overlap of genes predicting the same term in separate hub networks is much higher than overall overlap of network genes. For example, both nuclear and RiboTag D1 modules enrich for “anterograde trans-synaptic signaling”, and 12.2% (12/98) of the predictive genes overlap across the two methods.

The top 100 transcription factor binding sites (TFBSs) by enrichment p-value were selected and overlapped for each method. 46 of the top TFBSs for D1 hub networks overlapped, and these corresponded to 33 transcription factors, 21 of which were unique to D1 modules (i.e., not also predicted in D2 modules) (Fig. 4d). DESeq2 p-values were extracted for those transcription factors that met expression filtering cutoffs (11 out of 21 passed filtering in at least one method) and corrected for the multiple comparisons. We found that two of the transcription factors were significantly enriched in D1-MSNs (Gabpa, padjusted_wholecell = 0.0003, Krox, padjusted_nuclear = 0.006), while Creb1 trended towards D1-enrichment (padjusted_wholecell = 0.08). The fact that not all predicted transcription factors are enriched in the expected cell type – in fact, Bteb3 is surprisingly trending towards enrichment in D2 nuclei despite our upstream analysis indicating it as D1-unique (padjusted_nuclear = 0.08) – can likely be explained either by the inability of upstream analyses to differentiate between multiple members of a transcription factor family or by genome architectural differences between D1- and D2-MSNs, which must be mapped out in order to fully understand the transcriptional dynamics of these cell types.

D1 hub networks generated several terms relating to synaptic function (nuclear, GO:0098916, padjusted = 1.03 × 10−10, GO:0060291, padjusted = 2.39 × 10−5; RiboTag, GO:0098916, padjusted = 4.36 × 10−16; GO:0099504, padjusted = 1.67 × 10−13; GO:0007269, padjusted = 3.83 × 10−10) (Fig. 4c, Supplementary Table 7). The combined data implicate a D1-enriched function in regulation of synaptic dynamics involving second messenger signaling from the synapse and regulation of vesicle formation and release.

24 of the top 20 TFBSs for D2 whole cell and RiboTag hub networks overlapped, corresponding to 18 overall and 6 D2-unique transcription factors, though none overlapped with the one site predicted from the nuclear dataset (Fig. 5d). However, 6 of the 30 predicted TFBSs for nuclear D2-DEGs did overlap with whole cell and RiboTag module TFBSs (Supplementary Fig. 4b). This suggests that upstream analysis of nuclear sequencing datasets with low average expression levels (Fig. 1f) should be performed on differential expression rather than on network analyses, given the high level of gene co-expression that the nuclear method uncovers. In fact, upstream TFBS analysis on nuclear D1-DEGs also improves overlap of predicted sites (Supplementary Fig. 4c). Of the 6 D2-unique transcription factors, only 2 met filtering criteria for our differential expression analysis. Using DESeq2 p-values corrected for these 2 comparisons, we found significant D2-enrichment for one of the transcription factors, Maz (padjusted_RiboTag = 0.04).

Nuclear and RiboTag D2 hub networks showed enrichment for functional terms relating to RNA processing (Fig. 5c, Supplementary Table 7), implicating post-transcriptional mechanisms in the maintenance of D2-MSN identity (nuclear, GO:0035195, padjusted = 5.72 × 10−25, GO:1903231, padjusted = 4.27 × 10−27; RiboTag, GO:0003723, padjusted = 6.16 × 10−13; REAC:R-MMU-8953854, padjusted = 2.12 × 10−6). The importance of post-transcriptional regulation in D2-MSNs helps to explain the higher variance observed in D2 compared to D1 MSN nuclei (Supplementary Fig. 5).

### Analyzing sex differences using RiboTag-RNAseq

The RiboTag-RNAseq dataset was considerably less complex than those generated by the other two methods, and therefore showed the most robust separation of cell types on PCA. We thus expected it to most powerfully separate two biological types – namely, MSNs from male and female mice – whose difference has not yet been characterized. We compared the aforementioned male mice (n = 12) to RiboTag-RNAseq samples from female mice (n = 10). WGCNA performed on RiboTag-purified male and female D1- and D2-MSN RNA generated modules (n = 22, Fig. 6b,c) which correlated with the eigengenes for male and female sex. Notably, the correlation p-values for the sex-enriched modules did not meet statistical significance (male, p = 0.2; female, p = 0.008; p-value significance threshold = 0.005). This suggests that MSN differences by sex are less pronounced than differences between cell types within a given sex, and future experiments may require increased sample size to fully characterize differences by sex. Nevertheless, male- and female-correlated modules contained genes in terms relating to metabolism of genetic material (padjusted < 9.6 × 10−2), a result mirrored in GO analysis of previous whole tissue RNAseq46 (Supplementary Table 7). The female hub network predicted Tel1, a regulator of telomere maintenance, as an upstream transcription factor (Fig. 6c). This prediction is in line with evidence of sexual dimorphism in telomeric structure47,48, and demonstrates that sex differences in the brain can be investigated at the cell type-specific level. This is possible even when cell type-enriched DEGs are highly overlapping across sex (Fig. 6d).

## Discussion

RNAseq is a powerful method for examining the baseline, homeostatic mechanisms that determine cellular identity, but its output is highly dependent on the method by which libraries are prepared. We examined, in particular, the effect of cell separation technique on RNAseq of biologically similar populations of dopamine receptor-expressing D1- and D2-MSNs in the NAc. Our results highlight, foremost, the importance of reducing technique-induced noise by implementing well-controlled experiments and using strict FPKM minimum and maximum cutoffs to eliminate sequencing artifacts.

This work also provides several gene lists that will inform future work. Our modeling of transcriptional control mechanisms can be used to make further predictions about the subcellular transcriptional regulation of various transcripts. Our variance partitioning data present a resource of technique-variable genes with which to normalize when comparing datasets generated using different cell separation techniques. As well, our findings expand on existing RNA localization datasets by demonstrating a robust list of genes expressed only in the nuclear and whole cell datasets, including genes whose transcripts are restricted to the nuclear compartment.

We confirm the bias of RiboTag affinity purification towards isolation of protein-coding genes, a fact which affords both advantages and disadvantages. The disadvantage is, of course, the failure of this technique to recover large sets of non-coding regulatory genes which differ dramatically between cell types (as indicated by nuclear and whole cell sequencing) and are known to play an important role in biological regulation. Its advantage is the superior separation of cell types as compared to the other methods. The RiboTag dataset showed the clearest separation of D1- and D2-MSNs, although it also showed the lowest fold-change in differential expression of cell type-enriched genes.

Nuclear- and whole cell-FACS-based isolation similarly have advantages and disadvantages. One challenge is their extreme complexity, especially when using the nuclear method, requiring strict cut-offs or large sample sizes for more robust results. This complexity is furthered by the presence of immature mRNAs, which may or may not be destined for translation. For cell types whose nucleus contains lowly expressed, high variance genes, nuclear sequencing generates noisier results than either whole cell or RiboTag sequencing, and upstream analysis of differential expression, as opposed to co-expression data, generates more reliable conclusions.

However, the nuclear method does offer advantages. A methodological advantage of nuclear isolation is the fact that it can be performed on frozen tissue, making implementation of large experiments and/or use of banked human tissues much more manageable. A conceptual advantage of the nuclear method is demonstrated by our GWAS hit enrichment analysis, which reveals that this method captures more disease-relevant loci than either of the other two, making it an appealing option for those studying the genetic basis of neuropsychiatric disease.

Despite these differences between methods, and the relatively low overlap of differentially expressed genes, convergent conclusions can be made about cell type-enriched transcription. We were able to robustly identify regulatory transcription factors, central biological functions, and enriched protein complexes across methods. Orthogonal, low-throughput methods such as qPCR, RNAscope, or confocal microscopy, could be used to validate the cell type-specificity of the D1- and D2-enriched transcripts we identify. The precise role of these transcripts –specifically, whether they contribute to cell identity – should be examined using cell type-specific viral-mediated gene transfer followed by morphological and electrophysiological analyses. This work will provide important insight into the transcriptionally-dependent functional differences between D1- and D2-MSNs.

An important future question concerns the ability of these techniques to capture transcriptional responses of cell populations to various stimuli. We would expect nuclear and RiboTag-generated datasets to represent different stages of response, and to be differentially valuable during given post-stimulus time windows. This is a question which requires empirical investigation, and to which baseline normalization based on our results can be applied.

## Methods

### Transgenic animal lines

All animal protocols were approved by the Institutional Animal Care and Use Committee at the Icahn School of Medicine at Mount Sinai and carried out in accordance with their guidelines. All mice were bred on a C57BL/6J background. Mice used for nuclear isolation heterozygously expressed a nuclear GFP label under the promoter of either Drd1 or Drd2 (Drd1-cre × Thy1-loxPSTOPloxP-EGFP-F; Drd2-cre × Thy1-loxPSTOPloxP-EGFP-F). Mice used for whole cell isolation heterozygously expressed a cytoplasmic fluorophore under the promoter of either Drd1 or Drd2. Mice used for RiboTag affinity purification expressed an HA-tagged ribosomal subunit under the control of Drd1 and Drd2 (Drd1-cre × Rpl22-loxPexon4loxP-exon4-HA; Drd2-cre × Rpl22-loxPexon4loxP-exon4-HA).

### Cell isolation

Each sample represents tissue from a unique mouse. Nuclear samples were obtained from frozen tissue. This tissue was mechanically dissociated and nuclei lysed using a glass douncer in ice-cold lysis buffer (10.94% w/v sucrose, 5 mM CaCl2, 3 mM Mg(CH3COO)2, 0.1 mM EDTA, 10 mM Tris-HCl pH 8, 1 mM DTT, in H2O). Samples were centrifuged and supernatant removed and placed on top of a 60% sucrose solution (60% w/v sucrose, 3 mM Mg(CH3COO)2, 10 mM Tris-HCl pH 8, 1 mM DTT, in H2O). The sucrose gradient was centrifuged at 24,400 rcf for an hour, and nuclei at the bottom of the gradient were resuspended in PBS. DAPI was added at a concentration of 0.5 μg/mL. Whole cell samples were obtained from fresh tissue, which was rotated at 37 °C for 45 minutes in 1 mg/mL papain suspended in digestion buffer (5% w/v D-trehalose, 0.05 mM APV, 0.0125 mg/mL DNAse, in HibernateTM-A (Thermofisher, A1247501)). Tissue was then placed in FACS buffer (0.58 mg/mL albumin inhibitor (Worthington Biochemical, LK003182), 5% w/v D-trehalose, 0.05 mM APV, 0.0125 mg/mL DNAse, in HibernateTM-A) and triturated using progressively smaller pipette tips. Samples were passed through a 70 µm filter and placed on top of a layer of 10 mg/mL ovomucoid-albumin in FACS buffer. The pellet was resuspended in FACS buffer and DAPI was added at a concentration of 0.5 μg/mL. RiboTag samples were obtained from fresh tissue as previously described13,49.

### Library preparation and sequencing

All isolated RNA was prepared using the Clontech SMARTer® Stranded library preparation kit (Cat. No. 634838). Briefly, samples were ribo-depleted, and total RNA was reversed transcribed, classified using Illumina indices and amplified with 8 cycles of PCR. Samples were sequenced to a minimum depth of 30 million reads using paired-end reads with V4 chemistry on an Illumina Hi-Seq machine.

### RNAseq alignment preprocessing

Reads were aligned to GRCm38 with HISAT250. All aligned samples were reviewed for quality control with FASTQC (see URLs). Reads were counted for Mus_musculus.GRCm38.90 using featurecounts with the settings strandSpecific = 0 allowMultiOverlap = T, countMultiMappingReads = F, and isPairedEnd = T51. Gene level counts were generated from both only exons and all features. The percent of introns per gene was obtained by subtracting gene-level exon counts from all counts. Gene-level counts from all features were used unless otherwise specified.

Data were processed in three tranches: (1) male samples from all methods pooled, (2) male samples within each method, and (3) male and female RiboTag samples pooled. Genes with >1 fragments per million in ≥2 samples were kept for initial analyses. Final filters were ≥1 fragments per million per kilobase (FPKM) in at least one sample for nuclear and whole cell, and ≥0.1 FPKM in at least two samples for RiboTag. Counts were normalized to effective library size, and variance stabilized with DESeq2. Design matrix covariates, depending on the tranche, method (whole cell, RiboTag, or nuclear), cell type (D1 or D2) and/or sex (male or female). Variance stabilized transcripts (from the DESeq2 vst function) were used for both principal component analyses (PCA) and weighted gene co-expression network analysis (WGCNA)39. Two RiboTag samples were excluded because they were PCA outliers and had >80% of reads unmapped.

### Differential expression

Differential expression was assessed between D1 and D2 MSNs within each method using DESeq252. To determine cell type-enriched genes, lists were filtered for |log2foldchange| >0.38 corresponding to a 30% change (past work has demonstrated that at least a 15% change is needed to replicate with qPCR53), padjusted ≤ 0.05, mean normalized counts above the 25th percentile within method. Nuclear data were also filtered for log2foldchange standard error below the 75th percentile. Length and biotype data for each gene were obtained using the R package biomaRt54.

### Variance partitioning

Euclidean distance was calculated from PC1 and PC2. Centroids were determined from the mean PC values within each method. Distance was calculated with Euclidean distance from the cluster centroid, and pairwise t-tests between all groups were performed for Euclidean distances.

To calculate the percent variance explained by every covariate for every gene, VariancePartition was run on all genes and the intersection of genes passing minimum detection thresholds within each method55.

Based on the percent variance explained, genes were assigned to one of three covariate categories (method, cell type, residual). Genes within each category were analyzed using g:Profiler, and five of the top ten most significantly enriched terms are displayed. Method-variable genes were further divided by enrichment within method. Genes were assigned to the method in which they had the highest FPKM and overlapped with the RNAlocate database. Genes in our analysis that were not represented in the RNAlocate database were removed from the subcellular localization analysis.

### High confidence co-expression networks

Networks were generated in three tranches specified previously, all male samples, separately for each method, and for RiboTag with both male and female. DESeq2 variance stabilized transcript expression matrices were used as input.

For WGCNA, soft power thresholds for signed correlations were chosen to achieve approximate scale-free topology (R2 > 0.8): βAllMale = 18, βNuclear = 12, βWholeCell = 18, βRiboTAG = 9, βRiboTAGwithFemale = 6. The WGCNA function blockwiseModules was run with the appropriate power and the parameters networkType = “signed”, TOMType = “signed”, detectCutHeight = 0.99, minModuleSize = 100, reassignThreshold = 0, minKMEtoStay = 0.1, mergeCutHeight = 0.2, corType = “bicor”, numericLabels = TRUE, and pamStage = FALSE. Networks were then plotted and modules were exported as edge and node files from the topological overlap matrix.

ARACNE networks were generated using the minet R package by first building a mutual information matrix (build.mim function with estimator = “mi.empirical” and disc = “equalfreq”)56,57.

Edges detected independently with both WGCNA and ARACNE were kept, and these edges were used to determine the top 20 hubs per network (by degree) as well as their first-degree neighbors.

Genes within each network were analyzed using g:Profiler, and representative terms from those most significantly enriched are displayed.

### GWAS gene set enrichment

The GWAS catalog v1.0.2 (release 2018-08-14) was downloaded from the NHGRI-EBI catalog (see URLs)45. Associations were kept if the lead variants had p < 5 × 10−8 and mapped within a gene’s start and stop coordinates (i.e., was not intergenic). The gene associated with the lead variant was then mapped to its ENSEMBL mouse orthologue (see URLs)58. The remaining 1843 traits were manually curated for neuropsychiatric relevance, and categorized into 20 superseding traits, resulting in a final set of 554 genes from 733 gene-trait pairs.

Gene set enrichment between GWAS genes and network hubs was calculated from a 2 × 2 contingency table with a Fisher’s Exact Test. The contingency table comprised all genes used as input for WGCNA and ARACNE for a given tranche, with rows labeling a gene as GWAS or not and columns labeling genes as belonging to a given hub network.

### Statistical procedures

We corrected for multiple comparisons within each family of hypotheses, which are recapitulated here. Significance was set at FDR or padjusted < 0.05, limiting the risk of false positives. Differentially expressed gene (DEG) p-values were FDR-adjusted with default DESeq2 methods. Logistic regression was used to assess if any DEG features predicted DEG overlap, followed by post-hoc determination of the most pertinent feature(s). Variance partitioning significance was set at 0.05/3 (Bonferroni adjustment). WGCNA eigengene-trait correlation significance thresholds were Bonferroni-adjusted for the number of modules in combined (pthreshold = 0.006), nuclear (pthreshold = 0.003), RiboTag (pthreshold = 0.001), whole cell (pthreshold = 0.002), and female-inclusive RiboTag (pthreshold = 0.005) coexppression analyses. For g:Profiler enrichment, p-values were adjusted with the g:SCS method. The Bonferroni-adjusted significance threshold for GWAS hit overlaps with method modules was 0.017. A Bonferroni correction was applied to test if transcription factors ascertained with binding site enrichment had increased expression D1- vs D2-MSNs (unadjusted DESeq2 p-values multiplied by the number of TFs). In addition to appropriate p-value adjustments, we also considered the rank of any results, effect sizes, and biological plausibility in order to ensure robust inference and generalizability.

All custom code is available under the MIT license (see URLs).

Custom code: https://github.com/frichter/d1_d2_rnaseq/.

Human-mouse orthologues: http://useast.ensembl.org/biomart/martview/.

## Data Availability

All data are available on NCBI’s GEO database, and can be found under the Accession Number GSE121199.

## References

1. 1.

Cruz, F. C. et al. New technologies for examining the role of neuronal ensembles in drug addiction and fear. Nat. Rev. Neurosci. 14, 743–754 (2013).

2. 2.

Orecchioni, S. & Bertolini, F. In Methods in molecular biology (Clifton, N. J.) 1464, 49–62 (2016).

3. 3.

Guez-Barber, D. et al. FACS purification of immunolabeled cell types from adult rat brain. J. Neurosci. Methods 203, 10–18 (2012).

4. 4.

Schwarz, J. M., Smith, S. H. & Bilbo, S. D. FACS analysis of neuronal–glial interactions in the nucleus accumbens following morphine administration. Psychopharmacology (Berl). 230, 525–535 (2013).

5. 5.

Sultan, M. et al. Influence of RNA extraction methods and library selection schemes on RNA-seq data. BMC Genomics 15, 675 (2014).

6. 6.

Slane, D., Kong, J., Schmid, M., Jürgens, G. & Bayer, M. Profiling of embryonic nuclear vs. cellular RNA in Arabidopsis thaliana. Genomics data 4, 96–8 (2015).

7. 7.

Richardson, G. M., Lannigan, J. & Macara, I. G. Does FACS perturb gene expression? Cytometry. A 87, 166–75 (2015).

8. 8.

Hines, W. C., Su, Y., Kuhn, I., Polyak, K. & Bissell, M. J. Sorting out the FACS: a devil in the details. Cell Rep. 6, 779–81 (2014).

9. 9.

Pastro, L. et al. Nuclear Compartmentalization Contributes to Stage-Specific Gene Expression Control in Trypanosoma cruzi. Frontiers in Cell and Developmental Biology 5, 8 (2017).

10. 10.

Zaghlool, A. et al. Efficient cellular fractionation improves RNA sequencing analysis of mature and nascent transcripts from human tissues. BMC Biotechnol. 13, 99 (2013).

11. 11.

Heiman, M. et al. A Translational Profiling Approach for the Molecular Characterization of CNS Cell Types. Cell 135, 738–748 (2008).

12. 12.

Sanz, E. et al. Cell-type-specific isolation of ribosome-associated mRNA from complex tissues. Proc. Natl. Acad. Sci. USA 106, 13939–44 (2009).

13. 13.

Chandra, R. et al. Opposing role for Egr3 in nucleus accumbens cell subtypes in cocaine action. J. Neurosci. 35, 7927–37 (2015).

14. 14.

Okaty, B. W., Sugino, K. & Nelson, S. B. A quantitative comparison of cell-type-specific microarray gene expression profiling methods in the mouse brain. PLoS One 6, e16493 (2011).

15. 15.

Lobo, M. K., Karsten, S. L., Gray, M., Geschwind, D. H. & Yang, X. W. FACS-array profiling of striatal projection neuron subtypes in juvenile and adult mouse brains. Nat. Neurosci. 9, 443–52 (2006).

16. 16.

Gerfen, C. R. et al. D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science 250, 1429–32 (1990).

17. 17.

Schotanus, S. M. & Chergui, K. Dopamine D1 receptors and group I metabotropic glutamate receptors contribute to the induction of long-term potentiation in the nucleus accumbens. Neuropharmacology 54, 837–844 (2008).

18. 18.

Calabresi, P., Picconi, B., Tozzi, A. & Di Filippo, M. Dopamine-mediated regulation of corticostriatal synaptic plasticity. Trends Neurosci. 30, 211–219 (2007).

19. 19.

Kreitzer, A. C. & Malenka, R. C. Endocannabinoid-mediated rescue of striatal LTD and motor deficits in Parkinson’s disease models. Nature 445, 643–647 (2007).

20. 20.

Lee, K.-W. et al. Cocaine-induced dendritic spine formation in D1 and D2 dopamine receptor-containing medium spiny neurons in nucleus accumbens. Proc. Natl. Acad. Sci. 103, 3399–3404 (2006).

21. 21.

Lobo, M. K. et al. Cell Type-Specific Loss of BDNF Signaling Mimics Optogenetic Control of Cocaine Reward. Science (80-). 330, 385–390 (2010).

22. 22.

Lim, B. K., Huang, K. W., Grueter, B. A., Rothwell, P. E. & Malenka, R. C. Anhedonia requires MC4R-mediated synaptic adaptations in nucleus accumbens. Nature 487, 183–189 (2012).

23. 23.

Kravitz, A. V., Tye, L. D. & Kreitzer, A. C. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat. Neurosci. 15, 816–818 (2012).

24. 24.

Chandra, R. et al. Optogenetic inhibition of D1R containing nucleus accumbens neurons alters cocaine-mediated regulation of Tiam1. Front. Mol. Neurosci. 6, 13 (2013).

25. 25.

Khibnik, L. A. et al. Stress and Cocaine Trigger Divergent and Cell Type–Specific Regulation of Synaptic Transmission at Single Spines in Nucleus Accumbens. Biol. Psychiatry 79, 898–905 (2016).

26. 26.

Calipari, E. S. et al. In vivo imaging identifies temporal signature of D1 and D2 medium spiny neurons in cocaine reward. Proc. Natl. Acad. Sci. USA 113, 2726–31 (2016).

27. 27.

Francis, T. C. et al. Molecular basis of dendritic atrophy and activity in stress susceptibility. Mol. Psychiatry 22, 1512–1519 (2017).

28. 28.

Cheng, J. et al. Transcriptional Maps of 10 Human Chromosomes at 5-Nucleotide Resolution. Science (80-.). 308, 1149–1154 (2005).

29. 29.

Grindberg, R. V. et al. RNA-sequencing from single nuclei. Proc. Natl. Acad. Sci. USA 110, 19802–7 (2013).

30. 30.

Song, Y. et al. A comparative analysis of library prep approaches for sequencing low input translatome samples. BMC Genomics 19, 696 (2018).

31. 31.

Chothani, S. et al. Reliable detection of translational regulation with Ribo-seq. bioRxiv 234344. https://doi.org/10.1101/234344 (2017).

32. 32.

Hsu, S.-D. et al. miRTarBase: a database curates experimentally validated microRNA–target interactions. Nucleic Acids Res. 39, D163–D169 (2011).

33. 33.

Chen, E. A. et al. Effect of RNA integrity on uniquely mapped reads in RNA-Seq. BMC Res. Notes 7, 753 (2014).

34. 34.

Tyakht, A. V. et al. RNA-Seq gene expression profiling of HepG2 cells: the influence of experimental factors and comparison with liver tissue. BMC Genomics 15, 1108 (2014).

35. 35.

Gokce, O. et al. Cellular Taxonomy of the Mouse Striatum as Revealed by Single-Cell RNA-Seq HHS Public Access. Cell Rep 16, 1126–1137 (2016).

36. 36.

Reimand, J. et al. g:Profiler—a web server for functional interpretation of gene lists (2016 update). Nucleic Acids Res. 44, W83–W89 (2016).

37. 37.

Gandal, M. J. et al. Shared molecular neuropathology across major psychiatric disorders parallels polygenic overlap. Science 359, 693–697 (2018).

38. 38.

Zhang, T. et al. RNALocate: a resource for RNA subcellular localizations. Nucleic Acids Res. 45, D135–D138 (2017).

39. 39.

Langfelder, P. & Horvath, S. WGCNA: an R package for weighted correlation network analysis. BMC Bioinformatics 9, 559 (2008).

40. 40.

Sutherland, M. L. et al. Overexpression of a Shaker-type potassium channel in mammalian central nervous system dysregulates native potassium channel gene expression. Proc. Natl. Acad. Sci. USA 96, 2451–5 (1999).

41. 41.

Tsaur, M. L., Sheng, M., Lowenstein, D. H., Jan, Y. N. & Jan, L. Y. Differential expression of K+ channel mRNAs in the rat brain and down-regulation in the hippocampus following seizures. Neuron 8, 1055–67 (1992).

42. 42.

Obara-Michlewska, M., Ruszkiewicz, J., Zielińska, M., Verkhratsky, A. & Albrecht, J. Astroglial NMDA receptors inhibit expression of Kir4.1 channels in glutamate-overexposed astrocytes in vitro and in the brain of rats with acute liver failure. Neurochem. Int. 88, 20–25 (2015).

43. 43.

Jang, S. H. et al. Nuclear Localization and Functional Characteristics of Voltage-gated Potassium Channel Kv1.3. J. Biol. Chem. 290, 12547–12557 (2015).

44. 44.

Li, B. et al. Nuclear BK channels regulate gene expression via the control of nuclear calcium signaling. Nat. Neurosci. 17, 1055–1063 (2014).

45. 45.

MacArthur, J. et al. The new NHGRI-EBI Catalog of published genome-wide association studies (GWAS Catalog). Nucleic Acids Res. 45, D896–D901 (2017).

46. 46.

Hodes, G. E. et al. Sex Differences in Nucleus Accumbens Transcriptome Profiles Associated with Susceptibility versus Resilience to Subchronic Variable Stress. J. Neurosci. 35, 16362–16376 (2015).

47. 47.

Aviv, A., Shay, J., Christensen, K. & Wright, W. The Longevity Gender Gap: Are Telomeres the Explanation? Sci. Aging Knowl. Environ. 2005, pe16–pe16 (2005).

48. 48.

Barrett, E. L. B. & Richardson, D. S. Sex differences in telomeres and lifespan. Aging Cell 10, 913–921 (2011).

49. 49.

Chandra, R. et al. Drp1 Mitochondrial Fission in D1 Neurons Mediates Behavioral and Cellular Plasticity during Early Cocaine Abstinence. Neuron 96, 1327–1341.e6 (2017).

50. 50.

Kim, D., Langmead, B. & Salzberg, S. L. HISAT: a fast spliced aligner with low memory requirements. Nat. Methods 12, 357–360 (2015).

51. 51.

Liao, Y., Smyth, G. K. & Shi, W. featureCounts: an efficient general purpose program for assigning sequence reads to genomic features. Bioinformatics 30, 923–930 (2014).

52. 52.

Love, M. I., Huber, W. & Anders, S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol. 15, 550 (2014).

53. 53.

Walker, D. M. et al. Cocaine Self-administration Alters Transcriptome-wide Responses in the Brain’s Reward Circuitry. Biol. Psychiatry, https://doi.org/10.1016/J.BIOPSYCH.2018.04.009 (2018).

54. 54.

Smedley, D. et al. BioMart – biological queries made easy. BMC Genomics 10, 22 (2009).

55. 55.

Hoffman, G. E. & Schadt, E. E. variancePartition: interpreting drivers of variation in complex gene expression studies. BMC Bioinformatics 17, 483 (2016).

56. 56.

Margolin, A. A. et al. ARACNE: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context. BMC Bioinformatics 7(Suppl 1), S7 (2006).

57. 57.

Meyer, P. E., Lafitte, F. & Bontempi, G. minet: A R/Bioconductor package for inferring large transcriptional networks using mutual information. BMC Bioinformatics 9, 461 (2008).

58. 58.

Zerbino, D. R. et al. Ensembl 2018. Nucleic Acids Res. 46, D754–D761 (2018).

## Acknowledgements

This work was supported in part through the computational resources and staff expertise provided by Scientific Computing at the Icahn School of Medicine at Mount Sinai. Also at the Icahn School of Medicine at Mount Sinai, this work was supported by the facilities of and guidance from the Flow Cytometry core.

## Author information

F.R. and H.K. analyzed and interpreted results and wrote the paper. H.K., B.L. and R.C. generated the data. All authors discussed the analysis and results and commented on the manuscript.

Correspondence to Eric J. Nestler.

## Ethics declarations

### Competing Interests

The authors declare no competing interests.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.