Introduction

Compulsions—maladaptive patterns of repetitive, inflexible cognition, and behavior—are a key feature of numerous mental health conditions, including obsessive-compulsive disorder (OCD), trichotillomania, skin picking disorder, eating disorders, addiction, anxiety, and depression1,2,3,4. Mental disorders associated with compulsive behaviors cause significant distress and, because they include various forms and degrees of manifestation, they still are difficult to diagnose and treat3,5,6,7. Thus, better understanding the pathophysiology of compulsive behavior could optimize therapeutic approaches for multiple disorders.

Compulsivity is increasingly understood as resulting from failures of information flow in the neural circuits that govern self-regulation, motivation, and learning2,8,9,10. Clinical and preclinical studies suggest that compulsivity is governed by the frontal cortex in a complex interaction with subcortical systems that include midline thalamic nuclei, striatal regions, and the mesocorticolimbic dopamine system11,12,13,14. Consequently, abnormalities in the function of this cortico-striatal circuitry can contribute to compulsive/inflexible behavior, especially in complex and changing environments. For instance, accumulating evidence points to dysregulation of the cortico-striato-thalamo-cortical (CTSC) circuitry as a cause of OCD, one of the most common diseases of compulsive behavior10,15,16,17. Orbitofrontal cortex–ventromedial striatum (OFC–VMS) hyperconnectivity is particularly thought to drive compulsivity10,18,19,20. However, it is uncertain whether OFC–VMS dysfunction is sufficient to produce compulsivity on its own.

Nonhuman animal studies can contribute to the identification of relevant circuits, such as the OFC–VMS pathway. The main barrier to these studies, however, has been the lack of a clear link between human disease and animal behavior models, particularly in those models that respond to manipulations of the CTSC circuits. Self-grooming has been proposed as an animal model of inappropriate compulsive behavior21,22. Grooming is a ritualized sequence of coordinated movements that include licking, scratching, and wiping, aimed at cleaning fur and skin23,24,25. Self-grooming has clear adaptive functions in certain situations26,27, but it is displaced and maladaptive in others. It may appear, for example, when the animal is placed in atypical or stressful situations, such as when exposed to the elevated plus-maze test26,28.

In mice, hyper-grooming is specifically linked to the OFC–VMS circuit. Ahmari et al.20 used an optogenetic strategy to hyperactivate the OFC–VMS circuit. Repeated optogenetic stimulation of OFC–VMS projections in inbred mice produced excessive self-grooming behavior. This grooming was consistent with induced plasticity: it appeared only after multiple days of stimulation, persisted in the absence of stimulation, and was reversed with chronic administration of fluoxetine, a drug considered a first‐line pharmacotherapy agent for OCD. The change was specific to compulsivity: the model showed no differences from unstimulated mice in sensorimotor filtering (pre-pulse inhibition) or anxiety-related activity (elevated plus-maze and open-field tests).

A major open question is whether the link between induced OFC–VMS hyperconnectivity and compulsivity can translate across species. Most of the literature on induced compulsivity is in mice, given the large genetic toolbox, e.g., the HoxB8 (ref. 29), Slitrk5 (ref. 30), and SAPAP3 mutants31. However, mice have a simpler behavioral repertoire and less flexibility in dealing with novel situations, and thus are not as able to participate in complex cognitive tasks, compared to larger rodents, such as rats32,33. Further, although mice and rats appear to be equally phylogenetically distant from humans, there may be greater similarities between rats and humans in clinically relevant functions. Behaviorally34, genetically35, and in central nervous system receptor biology36, rat brain and behavioral function is better-matched to humans compared to mice. This may favor the use of rats when modeling aspects of mental illness. Rats present a larger brain and more skull area, facilitating complex electrophysiological and imaging studies. The larger anatomy may also allow better modeling of clinical interventions, such as deep brain stimulation (DBS), which depends on fine-grained steering of stimulation to specific circuit elements37,38. It is also important to verify that results generalize across species, as part of understanding their robustness.

We tested whether the OFC–VMS hyperstimulation approach could translate from mice to rats. Since rats might express compulsivity differently compared to mice, and since grooming is only one task-free method for assessing compulsivity, we explored this translation across a range of compulsive behaviors. Other repetitive behaviors considered compulsive-like in rodents include hiding objects and shredding materials recurrently22,39,40. Thus, we tested the same approach across grooming, marble burying, and nestlet shredding tests, which evaluate repetitive behaviors that may become compulsive41,42.

An important aspect of human compulsivity that has not been well-modeled in animals is the impairment of behavioral flexibility, meaning deficits in behavior shifting in response to changing environmental contingencies43,44,45,46. Behavioral flexibility often involves withholding a prepotent (default) response in favor of a more adaptive choice, which in the context of compulsivity could mean avoidance of a previously emitted behavior. As with the broader construct of compulsivity, flexibility depends on a network that includes OFC, prefrontal cortex, and dorsal anterior cingulate cortex47,48,49,50. Flexibility can be tested across species with set-shifting tasks17, and these tasks have revealed cortico-striatal deficits in patients with OCD51,52. Similarly, DBS of the ventral capsule/ventral striatum, an advanced surgical therapy for compulsivity, can increase behavioral flexibility53. Therefore, in addition to the task-free behaviors above, we evaluated the effects of OFC–VMS stimulation on a set-shifting task that tests behavioral flexibility.

We show that, although repeated optogenetic stimulation induced physiological changes in the OFC–VMS circuitry consistent with prior reports in mice, we were not able to induce repetitive behaviors of any type in rats. We also could not disrupt behavioral flexibility in rats by manipulating OFC–VMS circuitry. These findings contrast with prior mouse results and suggest that there are fundamental differences in the outcomes of striatal plasticity even between model species.

Materials and methods

Animals

Twenty-seven male Long-Evans rats (250–300 g) were obtained from Charles River Laboratories (Wilmington, MA) and housed in the McGuire Translational Research Facility of the University of Minnesota, Minneapolis, MN, under a 10:14 dark/light cycle (lights on at 07:00). We selected our animal counts for the behavioral evaluation to be adequately powered to replicate the primary behavioral outcome of the original report of OFC–VMS hyperstimulation20. Sample size calculations were conducted using GLIMMPSE 2.0.0 (ref. 54), with the effect size as observed in Ahmari et al.20. With the desired power set at 95% and the type I error rate at 5%, to detect a main effect of group, the analysis revealed that a minimum sample size of 16 rats (8 per group) was required (power = 0.956). We first targeted above this (21 rats) to account for potential animal exclusions (e.g., due to misplaced injections) and/or a smaller effect size in rats. An additional study focused on measuring OFC–VMS plasticity used a further six rats. The sample size for this second study was chosen ad hoc to be equivalent to that of the comparable experiment in Ahmari et al.20. After euthanasia and histology (see below), five rats showing no enhanced yellow fluorescent protein (eYFP) expression in OFC and/or showing fiber placements outside VMS were excluded from the analysis. This represents one rat from the control group and four rats from the ChR2 group. Of the ChR2 rats—three were from our behavioral experiment and one from the evoked response potential (ERP) experiment. From the 27 rats initially purchased, this yielded n = 17 for the primary behavioral experiment and n = 5 for the follow-on ERP experiment.

Rats were first acclimated for 5–7 days in the animal colony room and, subsequently, were handled for three consecutive days for 5 min/day to familiarize them with the experimenter before the beginning of the procedures. At the end of each of these handling sessions, the rats received five reward pellets (45 mg grain-based pellets; Bioserv, Flemington, NJ) in their home cages to forestall neophobic reactions in the set-shifting operant chambers. Rats had free access to food and water, except during the behavioral training and testing of the set-shifting protocol, when rats were food restricted. For this, animals were restricted to 10 g of standard laboratory rat chow per day until body weight was reduced to 85–90% of the original (after 5 days of food restriction, approximately). At this time food was increased to 15 g per day and their weights maintained at this level without further reduction until completion of the set-shifting sessions. All experiments were approved by the University of Minnesota Institutional Animal Care and Use Committee (protocol number 1806-35990A) and comply with the National Institutes of Health guidelines.

Surgical procedures

Each rat was deeply anesthetized with 3–4% isoflurane in an induction chamber and was mounted in a stereotaxic frame (maintained on 0.5–2% isoflurane for the duration of the surgery). For opsin delivery, a small opening was made over the left ventromedial orbitofrontal cortex—OFC: VO and MO, as in Ahmari et al.20—for the injection of adeno-associated virus (AAV). Injected AAV carried the gene encoding channel-rhodopsin (ChR2) fused to eYFP under the calcium- and calmodulin-dependent protein kinase II (CaMKII) promoter (AAV5-CaMKIIa-hChR2(H134R)-eYFP, University of North Carolina Viral Vector Core Facility, Chapel Hill, NC). Rats that received virus lacking the ChR2 sequence were used to control for any nonspecific effects of the viral transfection or light stimulation (AAV5-CaMKIIa-eYFP). Rats were randomly assigned to control or ChR2 groups. As in the original study20, viruses were injected into the left OFC (4.8 mm AP, 0.8 mm ML, 4.5 mm DV) using a 33-gauge Hamilton syringe and a microinjector. Injections (1.0 μl from a 4.1 × 1012 vg/ml) occurred over 10 min (0.1 μl/min) followed by an additional 5 min to allow diffusion of viral particles. The use of a CaMKIIa promoter enables transgene expression favoring pyramidal neurons when injected into the neocortex, with a level of transduction efficacy and expression of ChR2 of ~90% (ref. 55,56,57,58). We increased both the volume of injection and the viral titer to compensate for anatomic differences between mice and rats.

For the chronic optical fiber implant, a small skull opening was made over the left VMS, as in the original study20. We implanted an optical fiber over the VMS (1.28 mm AP, 2.6 mm ML, 5.8 mm DV) that permitted optical stimulation. Optical fibers were constructed in-house by interfacing a 20 mm piece of 200 μm, 0.66 numerical aperture optical fiber with a 10 mm ceramic ferrule (fiber extending 10 mm beyond the end of the ferrule). This is double the numeric aperture of the fiber used in the mouse study, implying an up to fourfold greater ability to transduce the light source into the brain. Fibers were attached with epoxy resin into ferrules, then cut with a diamond pen and polished. After construction, all fibers were calibrated to determine the percentage of light transmission at the fiber tip that interfaces with the brain; all fibers had >75% efficiency of light transmission. Several small burr holes were drilled around the perimeter of the exposed skull surface to accept small anchor screws. The fiber was then fixed to the skull and screws with dental cement.

To evaluate the effects of optogenetic stimulation on OFC–VMS plasticity, a second study added recording electrodes. Along with the opsin delivery at OFC and optical fiber implant at VMS, during stereotaxic surgery, twisted bipolar platinum electrodes with a ground wire (MS333/8-BIU/SPC, Plastics One, Roanoke, VA) were implanted in both regions for stimulation and recording. For the OFC implant, the electrode was lowered 15 min after the end of the AAV injection; for the VMS implant, the electrode was attached to the optical fiber, being set 200 μm below the tip of the fiber. Once the electrodes were placed, ground wires were wrapped to a nearby screw. After surgery, rats were returned to their home cages and allowed to recover for at least 7 days before any behavioral experimentation. A minimum of 5 weeks (for stable viral expression) elapsed before the beginning of the optical stimulation (Fig. 1A–C).

Fig. 1: Experimental design for behavioral and electrophysiological evaluation and OFC–VMS projection targeting.
figure 1

A Timeline used for repeated stimulation of OFC–VMS projections for the behavioral experiments. H habituation, S stimulation, GR grooming, MB marble burying, NS nestlet shredding. B Timeline showing timepoints for evoked potential (ERP) measurement. C Schematic diagram indicating localization of ChR2-eYFP injections into OFC, optical fiber implant in the VMS, and electrical stimulation and recording in both OFC and VMS. This restricts activation to OFC terminals in VMS. D Schematic representation of eYFP expression in the OFC. Each ellipse represents the expression region of a single animal. Red dashed line depicts the virus spread in the original study20. E Example DAPI stained coronal section showing eYFP expression (green) in the OFC. LO lateral orbitofrontal cortex, VO ventral orbitofrontal cortex, MO medial orbitofrontal cortex. F Localization of the optical fibers in the VMS. Red cross represents the optical fiber’s approximate location in the original study20. G eYFP expression in the VMS projections directly below the optical fiber track (arrow), after virus injection into the OFC. AcbC accumbens nucleus core, LV lateral ventricle. H Light-sheet microscope imaging of transfected brain hemisphere cleared using the PEGASOS technique. Maximum projection reconstruction dataset of selected sagittal/coronal/horizontal planes with virus injection in the OFC (black arrowhead) and projections to the VMS (white arrowheads). A schematic diagram indicates localization of the orthogonal views. cc corpus callosum, ac anterior commissure, LV lateral ventricle, OB olfactory bulb. Scale bar: 500 µm. Modified from Paxinos and Watson96.

Full size image

Optogenetic stimulation

The protocol used for repeated cortico-striatal stimulation followed that described by Ahmari et al.20. After waiting 5–6 weeks for viral expression, the chronic optical fiber implant was connected to an armored optical patch cable, which in turn was connected to a 1 × 1 rotary joint optical commutator via FC adaptors and coupled to a 465 nm blue LED. Optical stimulation was controlled by a pulse generator (Pulse Pal, Sanworks, Stony Brook, NY) connected to the LED driver (Plexon, Dallas, TX). For four consecutive days before the beginning of optogenetic stimulation, rats were connected to the patch cord for 15 min per day for habituation (no stimulation was given). The rats’ behaviors were recorded by video cameras positioned above and laterally to the cage. On the following 6 days, we repeatedly delivered steady optical stimulation, activating the viral optical payload by bright blue light (5 min/day, six consecutive days, 10 ms pulses, 10 Hz, 7 mW), to rats placed in the test chamber. This delivered ~55 mW/mm2 at the fiber tip, well above the ChR2 opening intensity. Self-grooming behavior was evaluated before, during, immediately after, and 1 h after each stimulation, and at 1 and 2 weeks after the 6-day stimulation protocol. Marble burying tests, nestlet shredding tests, and operant attentional set-shifting task sessions were performed before and after the 6-day repeated opto-stimulation.

Self-grooming behavior

Self-grooming behavior was recorded with digital video cameras and scored by a single rater who was blind to the group assignment. Self-grooming was evaluated for 5-min periods: before (pre), during (stim), immediately after (post), and 1 h after each stimulation. One and 2 weeks after the 6-day stimulation protocol, we reevaluated grooming. The total time the animals spent self-grooming was scored from video recordings using ANY-maze software (version 6.0; Stoelting, Wood Dale, IL). To assess locomotor effects, the total distance traveled during the 5-min opto-stimulation (stim) for the 6-day stimulation protocol was scored automatically from video recordings using ANY-maze.

Marble burying test

The experimental protocol for the marble burying test was based on that used previously24. The apparatus used in this experiment consisted of a polypropylene cage (40 × 28 × 20 cm) with the floor filled ~5 cm deep with bedding, lightly tamped down to make a flat, even surface. A regular pattern of 20 1.5 cm diameter translucent light-blue glass marbles was placed on the surface, evenly spaced. Each animal was placed individually in the center of the cage and left for 15 min. At the end of the test, the rat was removed, and the number of marbles buried with bedding (to at least two-thirds their depth) during this interval was counted by three raters, who were blind to the treatment group assignment.

Nestlet shredding test

The experimental protocol for the nestlet shredding test was based on that described by Angoa-Pérez et al.39. The apparatus used in this experiment consisted of a polypropylene cage with the floor filled ~5 cm deep with bedding, and containing a cotton square of known weight that was placed on top of the bedding. The animal was placed individually in the center of the cage and left for 15 min. At the end of the test, the rat was removed, and the largest remaining intact portion of the cotton square was removed and allowed to dry. After 48 h, this portion of the cotton was weighed to calculate the percentage of nestlet shredded.

Operant attentional set-shifting task

All set-shifting sessions were performed in standard automated operant chambers (25 × 29 × 25 cm) with metal sidewalls, a transparent Plexiglas rear wall and front door, and stainless-steel rod floor (Coulbourn Instruments, Holliston, MA), enclosed inside sound-attenuating boxes (Med Associates, Chicago, IL). A 10-W light bulb provided light throughout the sessions. Each behavioral chamber was equipped with three nosepoke holes at one of the sides of the chamber, housing infrared sensors to detect head entries, and white LEDs to provide visual cues. A pellet dispenser that delivers food reward (45 mg grain-based pellets) was placed on the opposite wall. Software and an appropriate interface (GraphicState 4.0, Coulbourn Instruments) controlled the presentation and sequencing of stimuli. Behavior was recorded by video cameras mounted on the top of each unit.

The set-shifting task requires the rats to shift their response patterns between two distinct perceptual discrimination rules, or dimensions: light or side (right or left). On all trials, one of the two peripheral nosepoke holes was illuminated. Performance according to the light discrimination rule required the rats to poke the illuminated nosepoke hole, regardless of its spatial location. Performance according to the side rule required that the animals respond only at the nosepoke hole at a designated spatial location (either the left or the right) across trials, regardless of which one was illuminated.

The set-shifting protocol was modified from Darrah et al.59. Rats were submitted to a shaping period of 5 days before training and test trials. On the first day of shaping, they were placed in the operant chambers, ten reward pellets were delivered in the food tray at the beginning of the session and the animals remained in the chamber for 20 min. On the following day, they remained in the chamber for 20 min and during this period, reward pellets were delivered at 30-s intervals, if the rat had consumed the prior pellet. On days 3–5, rats were submitted to a single session in one of the two discrimination rules to be used in the set-shifting task, light or side (right or left). During these dimension shaping sessions, the rats were reinforced with a single reward pellet for each correct response according to the current discrimination rule. The sessions lasted until a performance criterion of ten consecutive correct responses was reached. After shaping, in which rats learned the light and side rules individually, rats received 4 days of training sessions on the full set-shifting task. Each trial began with illumination of the central nosepoke hole; poking this central hole caused one of the two side nosepoke holes to illuminate. A correct response according to the current rule (side or light) generated a food reward; incorrect responses were not rewarded. Rewarded and non-rewarded responses were followed by a 7-s interval and the beginning of a subsequent trial. When a rat reached the performance criterion of five consecutive correct choices in the first rule, the rewarded dimension shifted immediately to the alternative rule (extradimensional shift), requiring the rat to shift its behavior to receive a reward. No additional cue, besides the absence of reward after a wrong response, indicated the change in rules. During each training day, the task required the rats to reach the performance criterion eight consecutive times, resulting in seven consecutive extradimensional shifts. After training, animals were submitted to six baseline testing sessions to establish a baseline performance level for each individual rat. The test session followed the same protocol as the training sessions. After the baseline testing phase, the rats were removed from the food deprivation regimen and, when the original weight was reached, approximately within a week, animals were submitted to the opto-stimulation. After the 6-day opto-stimulation protocol, rats were again food restricted and, after reaching target weight, underwent set-shift testing during six consecutive days with the same parameters, as those described for the baseline testing.

Measuring OFC–VMS connectivity via evoked potentials

Repeated optogenetic stimulation of OFC terminals in VMS is believed to induce synaptic plasticity in mice. To determine whether this effect holds in rats, we evaluated the bidirectional OFC–VMS ERP before and after the 6-day opto-stimulation protocol (Fig. 1B). For this, animals with active virus in the OFC–VMS pathway and electrodes in both regions (Fig. 1C) were submitted to the same optogenetic stimulation protocol previously described. Before running this experiment, rats were connected to the patch cable and stimulation/recording wires for 4 days, while in the test chamber (15 min per day) for habituation, without receiving stimulation. To measure OFC–VMS effective connectivity, a PC running a custom-made LabVIEW program (National Instruments, Austin, TX) was connected to a NI USB-6343 BNC analog/digital interface unit (National Instruments), and an analog stimulus isolator (model 2200, A-M Systems, Sequim, WA). The software delivered constant-current, single pulses through the stimulus isolator. From the 5th to the 11th day, after animals were connected to patch cable and wires, 30–50 pulses (200 µA, 0.1 ms, with an interpulse interval varying between 3 and 4 s) were delivered to OFC or VMS. The distant response to these pulses (in VMS to OFC stimulation, or in OFC to VMS stimulation) is a measure of synaptic strength between the two regions. We selected this field response, rather than single-unit responses (which can be either excitatory or inhibitory), because we wished to directly measure changes in OFC–VMS excitatory drive. An increase in net excitation would be consistent with the dominant model that specifies hyperconnectivity in CSTC loops during compulsivity17,60. Next, the same optogenetic stimulation protocol followed (5-min pre-stim grooming observation, 5-min stim, 5-min post-stim grooming observation; 10 ms pulses, 10 Hz, 7 mW). After the post-stimulation period, a second measurement sequence of 30–50 pulses to each area was administered. LFP signals were recorded continuously at 30 kHz with an Open Ephys acquisition board61 and an Intan 32-channel headstage (RHD2132, Intan Technologies, Los Angeles, CA). Broadband activity was recorded, stored for offline analysis, filtered (band pass between 0 and 20 Hz by the amplifier), and post-processed in Matlab (Mathworks, Natick, MA). To measure interregional ERPs, we calculated the absolute value of the area underneath the curve (AUC) after each pulse, beginning at the offset of the stimulation artifact and measured out to 1.5 s post-stimulation. For analysis of the change within a day, AUC values were corrected by subtracting the mean baseline voltage (0.5 s period that preceded each pulse). For the analysis of ERP change across days, AUC values were corrected by dividing by the mean pre-stimulation AUC of day 1.

Histology and Immunostaining

At the end of the experiments, optical stimulation was performed for 5 min to induce immediate early gene activation consistent with that induced by the main intervention. Ninety minutes later, rats were deeply anesthetized (pre-anesthesia with isoflurane followed by Beuthanasia-D Special, 150 mg/kg), and then transcardially perfused with cold phosphate-buffered saline (PBS) followed by 4% paraformaldehyde (PFA) in 0.1 M PBS solution. Brains were extracted and kept in PFA for 24 h, transferred to 30% sucrose PBS for 48–72 h, flash frozen in −75 °C isopentane, and then 40 μm sections were obtained using a cryostat. To verify fiber placement, sections were Nissl-stained. Expression of ChR2-eYFP or eYFP was confirmed using fluorescence microscopy (Keyence, Osaka, Japan). Animals were excluded as reported above if they did not show eYFP or fiber placements in the target regions.

For c-Fos immunostaining, sections were washed three times in PBST (PBS with 0.2% Triton-X) and incubated in PBST + NGS (5% normal goat serum) for 1 h. Rabbit anti-c-Fos (1:5000; part number 5348, Cell Signaling Technology, Danvers, MA) was incubated overnight at 4 °C. The next day, sections were washed three times in PBST, followed by 2-h room temperature incubation in Alexa 647 donkey anti-rabbit antibody (part number 711-606-152, Jackson ImmunoResearch, West Grove, PA). Sections were then washed and processed using 4′,6-diamidino-2-phenylindole (DAPI, 1:10000) for 20 min, followed by another PBS wash. Sections were mounted on slides, using VectaShield mounting medium, and coverslipped. Histological counting of cells expressing c-Fos protein in a 720 × 540 µm area of regions of interest was performed using FIJI62.

To further verify transfection of OFC–VMS fibers specifically, we submitted one brain to a PEGASOS tissue clearing protocol. After the brain was removed and kept in PFA for 24 h, a 1 cm3 brain section was cut. The tissue was removed from the fixative, rinsed with PBS several times, and then incubated at 4 °C for 24 h. Afterward, the tissue was processed in the University Imaging Centers, University of Minnesota (https://med.umn.edu/uic), using the PEGASOS passive immersion protocol63. Briefly, tissue was delipidated, dehydrated, and then incubated in the clearing medium consisting of 75% benzyl benzoate/22% polyethylene glycol monomethacrylate (PEGMMA)/3% Quadrol mounting solution (BB-PEG, RI = 1.543). Samples were then imaged with a cleared tissue light-sheet (Intelligent Imaging Innovations, Denver, CO) microscope using a 488 nm laser, with the 488/561 dual band-pass emission filter. Images (2048 × 2048 px) were captured with a Hammatsu ORCA-Flash 4 v3 sCMOS camera (Hamamatsu Photonics, Hamamatsu City, Japan) with an exposure of 250 ms and 1 μm pixel resolution. Images were constructed using Slidebook software (Intelligent Imaging Innovations), and manipulated in Imaris v9.0 (Bitplane, Zurich, Switzerland) and Nikon ElementsV 5.301 (Nikon, Tokyo, Japan).

Analysis of results

Statistical analysis was performed using R (version 3.6.0, R Core Team, 2019) with the package lme4 (version 1.1-23; ref. 64). For each test, generalized linear mixed models (GLMMs) were used to account for the repeated‐measures design and non-Gaussian distribution of the data. For the countable data (marble burying test, number of errors in the set-shifting task, and positive c-Fos cells), GLMMs with log link function and Poisson distribution were used; for the other measurements, GLMMs with identity link function (locomotion, nestlet shredding test, reaction time in the set-shifting task, and ERP measurements), or log link function (grooming) and Gamma distribution were used. For each model, we used the rat’s identity as a random factor and treatment (Control and ChR2) and/or phase (e.g., pre- and post-stimulation) as fixed factors.

Results

Virus and fiber location

Viral expression was largely restricted to the medial and ventral OFC (Fig. 1D). Tracks of the optical fibers were identified in the VMS (Fig. 1F, G). ChR2-eYFP expression was confirmed in OFC (Fig. 1E) and in OFC projections in VMS (Fig. 1G), as targeted by Ahmari et al.20. Transfected fibers originating in the OFC and projecting to the VMS were also confirmed using the PEGASOS technique (Fig. 1H).

Self-grooming behavior

In contrast to the prior mouse results20, OFC–VMS repeated stimulation did not increase self-grooming in rats. There was no significant effect of active vs. eYFP-only virus on self-grooming, either before (p = 0.68), during (p = 0.80), or immediately after the stimulation (p = 0.85; Fig. 2A). Stimulation did not affect locomotor behavior (p = 1.00; Fig. 2B). There was no significant difference between ChR2 and control groups on self-grooming evaluated 1 h after stimulation (p = 0.80; Fig. 2C). We similarly did not replicate the finding that stimulation leads to long-term grooming changes; there were no differences in grooming expression 1 or 2 weeks after the end of the optogenetic stimulation protocol (p = 0.11; Fig. 2D).

Fig. 2: OFC–VMS repeated opto-stimulation did not increase self-grooming or locomotor behavior in rats.
figure 2

A Total duration of self-grooming behavior assessed for 5 min immediately before (pre), during (stim), and immediately after (post) 6 days of OFC–VMS opto-stimulation. B Total distance traveled during the 5-min opto-stimulation over six consecutive days. C Total duration of self-grooming behavior, assessed for 5 min at 1 h after OFC–VMS opto-stimulation, for six consecutive days. D Total duration of self-grooming behavior, each assessed for 5 min, 1 and 2 weeks after the end of the 6-day stimulation protocol. Markers in A–C represent mean, error bars show SEM. In D, markers represent individual animals, bars indicate mean and SEM. n = 9 for ChR2 and n = 8 for Control.

Full size image

Marble burying and nestlet shredding

OFC–VMS stimulation did not increase marble burying or nestlet shredding. There was no effect of active vs. eYFP-only virus on the marble burying test (p = 0.42; Fig. 3A). Marble burying, in fact, decreased over time in both the active stimulation and the control groups 1 h (exp(β) = 0.36, SE = 0.22, z = −4.60, p = 4.21e−6), 1 week (exp(β) = 0.48, SE = 0.20, z = −3.66, p = 2.54e−4, and 2 weeks (exp(β) = 0.07, SE = 0.46, z = −5.93, p = 3.11e−9) after stimulation in comparison to baseline. There was no effect of active vs. eYFP-only virus on the nestlet shredding test (p = 0.22; Fig. 3B).

Fig. 3: No effect of OFC–VMS opto-stimulation on non-grooming compulsive behaviors.
figure 3

A Number of marbles buried before (baseline) and after (post-stim, 1 and 2 weeks) the 6-day stimulation protocol. B Percentage of nestlet shredded before (baseline) and after (post-stim, 1 and 2 weeks) the stimulation protocol. C Change in reaction time (RT) during the set-shifting sessions, from before to after the 6-day stimulation protocol. D Change in the number of errors during the set-shifting sessions from before to after the stimulation protocol. Markers represent individual animals; bars indicate mean and SEM. n = 9 for ChR2 and n = 8 for control.

Full size image

Operant attentional set-shifting task

OFC–VMS stimulation did not affect set-shifting performance. There were no active vs. eYFP-only virus differences in reaction time (p = 0.08; Fig. 3C) or number of errors (p = 0.60; Fig. 3D) before and after the 6-day stimulation protocol.

Measures of circuit engagement

We verified that the OFC–VMS circuitry was activated, and that repeated stimulation did change their functional interaction in rats.

There was an increased c-Fos expression in the OFC after opto-stimulation of the OFC–VMS projections in ChR2 animals compared to eYFP-only controls (exp(β) = 1.80, SE = 0.20, z = 2.88, p = 0.004; Fig. 4A, B). There was no difference for c-Fos expression in the VMS (p = 0.56; Fig. 4C). No significant correlations were observed between c-Fos expression in the OFC (r = −0.05; p = 0.85) or VMS (r = −0.09; p = 0.72), and the grooming exhibited after the 6-day protocol of stimulation (Fig. 4D, E), further demonstrating a dissociation between induced plasticity and compulsive behavior.

Fig. 4: VMS opto-stimulation increased c-Fos expression in the OFC.
figure 4

A Representative images of a slice of the OFC demonstrating light-induced neural activation of OFC through optical stimulation of the VMS. DAPI labeling (blue), ChR2-eYFP viral expression (green), and c-Fos immunostaining (red). Scale bar 100 μm. Modified from Paxinos and Watson96. B Quantification of c-Fos-positive cells in the OFC of ChR2 animals compared to eYFP-only controls. *p < 0.05: different from the control group. C Quantification of c-Fos-positive cells in the VMS of ChR2 animals compared to eYFP-only controls. Markers represent individual animals; bars indicate mean and SEM. D Pearson’s correlation between c-Fos expression in the OFC and the duration of self-grooming. E Pearson’s correlation between c-Fos expression in the VMS and the duration of self-grooming. Both plots show grooming measured after the 6-day OFC–VMS opto-stimulation protocol. OFC: n = 9 for ChR2 and n = 5 for control. VMS: n = 10 for ChR2 and n = 6 for Ccntrol.

Full size image

We ran a second experiment in which we verified OFC–VMS plasticity electrophysiologically. In ChR2-transfected animals, there were clear electrical responses to light pulses (Fig. 5A, B). We were also able to measure ERPs at OFC and VMS in response to an electrical pulse delivered to the other structure, verifying in vivo connectivity that was primarily excitatory (principal peak consistent with depolarization, Fig. 5C, D). For the OFC electrical stimulation and VMS recording data, normalized to day 1 baseline (Fig. 5E), the responses increased significantly at day 2 (β = 0.34, SE = 0.08, t = 4.23, p = 1.05e−5), day 3 (β = 0.27, SE = 0.08, t = 3.32, p = 9.25e−4), day 4 (β = 0.63, SE = 0.10, t = 6.52, p = 1.06e−10), and day 5 (β = 0.60, SE = 0.08, t = 7.50, p = 1.26−e13) compared to day 1 of optical stimulation. This plasticity was most evident across days—there was only a significant intraday change from pre- to post-stimulation on testing day 4 (β = −0.57, SE = 0.19, t = −2.94, p = 3.33e−3). For the VMS electrical stimulation and OFC recording (Fig. 5F), there were changes only at day 2 (β = 0.22, SE = 0.07, t = 2.91, p = 3.62e−3) and day 3 (β = 0.21, SE = 0.07, t = 2.78, p = 5.50e−3) of optical stimulation in comparison to day 1. In contrast, for within-day changes in VMS response to OFC electrical stimulation (Fig. 5G), a significant difference between pre and post opto-stimulation appeared only on day 5 (β = −0.33, SE = 0.17, t = −2.00, p = 0.046). The within-day change decreased significantly at day 2 (β = −0.83, SE = 0.12, t = −7.01, p = 3.91e−12), day 3 (β = −0.52, SE = 0.12, t = −4.24, p = 2.37e−5), day 4 (β = −0.81, SE = 0.15, t = −5.28, p = 1.54e−7), and day 5 (β = −0.61, SE = 0.12, t = −4.90, p = 1.09e−6) compared to day 1 of optical stimulation. For the “bottom up” VMS electrical stimulation and OFC recording (Fig. 5H), no significant differences between pre and post opto-stimulation were observed. The response decreased significantly on day 4 (β = −0.77, SE = 0.12, t = −6.69, p = 3.43e−11) and day 5 (β = −0.90, SE = 0.10, t = −8.66, p < 0.01e−14) of optical stimulation in comparison to day 1. This second cohort again had no significant change in self-grooming behavior from OFC–VMS repeated stimulation (p = 0.76; Supplementary Fig. 1).

Fig. 5: OFC–VMS repeated opto-stimulation induced plasticity in rats.
figure 5

A In vivo recording in awake rats shows field responses in the OFC to optical stimulation of VMS projections. B Single-trial example of the population response shown in A. C Representative evoked response in VMS to OFC electrical pulses, pre- and post-repeated opto-stimulation. (Inset) In vivo recording in awake rats shows field responses in the VMS to electrical stimulation of OFC. D Representative evoked response in OFC to VMS electrical pulses, pre- and post-repeated opto-stimulation. (Inset) In vivo recording in awake rats shows field responses in the OFC to electrical stimulation of VMS. E VMS response to OFC stimulation, normalized to explore changes from day 1 baseline, showing increase in this connectivity metric between days. F OFC response to VMS stimulation, normalized to day 1 baseline, showing increase in connectivity only for days 2 and 3 of opto-stimulation. G VMS response to OFC stimulation, normalized by subtracting pre-stimulation voltage to explore within-day changes, showing significant difference between pre and post opto-stimulation only on day 5. H OFC response to VMS stimulation, also with pre-stimulation baseline subtracted, showing no significant difference between pre and post opto-stimulation. Note that E–H plot the area under the ERP curve calculated from individual stimulation trials across all rats, while C and D plot mean ERPs from single animals. As such, group-level differences are not readily visible in C and D. Markers represent mean, error bars show SEM. *p < 0.05: different from day 1; #p < 0.05: different from the measurement at the same day before opto-stimulation. n = 5.

Full size image

Discussion

Although we carefully mimicked the technical details of the original study20, and verified change in the targeted circuit, we could not reproduce in rats a model of compulsive behavior that has been previously reported in mice. This suggests that increased connectivity in the OFC–VMS pathway may not be sufficient for compulsivity. In a cohort that was explicitly and strongly powered to detect effect sizes below those of the original report, repeated OFC–VMS opto-stimulation did not increase the expression of self-grooming behavior in rats. This is inconsistent with a previous report for mice, in which the cortico-striatal stimulation led to a significant increase in self-grooming immediately before and 1 h after stimulation20. We also extended the evaluation of the potential effects of the OFC–VMS stimulation to other common assays of compulsivity. The repeated OFC–VMS stimulation protocol also did not affect marble burying, nestlet shredding, or operant set-shifting behaviors in rats. When combined with the high statistical power of this study, this argues in favor of a true negative result. From a more Bayesian perspective, our results suggest that any true behavioral effect of OFC–VMS hyperconnectivity in rats is sufficiently small to be of questionable use as a model of human disease.

In mice, acute OFC–VMS stimulation led to a large but transient increase in locomotion compared with controls, followed by a return to baseline levels after stimulation20. In rats, there was no significant difference for the animals’ locomotor behavior during the 5 min of stimulation over the 6 days of our protocol. It is possible that the opto-stimulation protocol caused less of a widespread striatum activation in rats than in mice. In fact, in the original study20, mice were tested at 1–5 mW to prevent pre-seizure/seizure activity that was seen to emerge at higher powers (S. Ahmari, personal communication). Cortico-striatal optogenetic stimulation seems to have a propensity to cause seizure activity in mice65. In rats, we rarely saw pre-seizure events with the 7 mW stimulation used. If the illumination field in mice was somewhat larger relative to the striatum, it could have caused greater cortical and striatal entrainment, including spillover from cognitive into motor circuitry. OFC–VMS stimulation might also trigger dopamine release and, consequently, increased locomotion—with a larger relative activation volume potentially resulting in increased dopamine release. However, it is hard to explain the divergence between the results solely by a volume of activation argument. We specifically changed the stimulation to compensate for the larger size of the rat’s brain (see below for more details).

Critically, we saw no behavior change even though the protocol induced expected changes in the OFC–VMS circuitry. Optogenetic stimulation of the OFC–VMS pathway caused acute neuronal activation, as measured by electrical responses to light pulses and by immediate early gene expression. Retrograde ChR2-induced action potentials may explain the increased c-Fos expression in the OFC, which was also observed in the Ahmari et al. study20. On the surface, the lack of increased c-Fos expression in the VMS could be interpreted to indicate that ChR2-generated depolarization of OFC–VMS terminals did not induce a sufficient glutamate release to modulate downstream striatal neuron activity. We do not believe this is a correct interpretation. In general, activation of PFC terminals in striatum does not induce robust immediate early gene activation, and we did not expect any in this study. For instance, another study using optogenetic stimulation of striatal terminals found no c-Fos change, but reported significant behavior change66. In fact, the original study did not report c-Fos increases in striatum20. c-Fos is expressed and inducible in the striatum in general—we were able to observe increased c-Fos-positive cells with electrical stimulation applied to the same site (Supplementary Fig. 2). Thus, our c-Fos data are consistent with successful target engagement and essentially replicates the original mouse study.

We provided further evidence of successful OFC–VMS engagement by going a step further than the original study, specifically showing that opto-stimulation induces directional synaptic plasticity across days. We demonstrated that opto-stimulation was sufficient to induce changes in synaptic weighting in the OFC–VMS direction, but caused minimal changes in the VMS-OFC direction. This implies that in rats, and presumably in higher species, compulsive behavior cannot be attributed to simple monosynaptic changes. Although not directly tested in this preparation, the same may be true for other monosynaptic findings reported in mice, e.g., the role of similar cortico-striatal circuitry in addiction67. The specific directionality of our findings, however, does argue that we are successfully measuring OFC–VMS circuit engagement.

This translational failure emphasizes the need to carefully consider species differences and suggests caution about overgeneralizing phenomena observed in a single species. Rodents are a vital model system for neurobiology due to the ability to use more invasive and comprehensive techniques68,69,70,71. Experiments that require altered protein expression in specific neurons most often use mouse models, due to the greater flexibility of genetic targeting. Physiological, anatomical, and psychopharmacological studies, however, have predominantly been performed in rats72,73,74. Although closely related, mice and rats present important genetic, anatomical, physiological, and behavioral differences that complicate direct comparisons between species71,75. The differences between our results and those of Ahmari et al.20 may in part be explained by environmental factors, such as housing enrichment, learning, and stress that can all affect behavior. It is also worth considering genetic background—rat studies are often conducted with outbred strains, whereas mouse studies often use inbred homozygotes. This likely reduces genetic diversity between experimental subjects, which may amplify effect sizes in mice. Given that we did see physiologic changes, however, we believe it is more valid to conclude that findings in the CSTC circuitry of mice may not necessarily be directly translatable in rats, and thus may also differ in other species. This is the flip side of the genetic argument—inbred genetics that cause a strong effect in one specific mouse strain may lead to detection of effects that do not generalize to other strains or rodent species.

These results do not appear to be attributable to minor variation in optogenetic technique. We used the same viral serotype as in the original study. Variations in transduction could then be linked primarily to the promoter or to potential differences in brain circuit complexity between mouse and rat. We used a different promoter (CaMKIIa, compared to the elongation factor-1 alpha, EF1α, used in Cre mouse lines in the original study), so the levels of expression could differ across the two animal models. However, given the less common use of Cre-driver lines in rats, several studies have shown that the CaMKIIa promoter can successfully engage excitatory neurons in cortical and other brain regions76,77,78,79,80,81. Our results cannot be explained by differences in viral delivery, since in our study a higher volume (1 µl) at a higher genomic titer (4.1 × 1012 vs. 1 × 1012 vg/ml) was administered to account for the larger brain of rats. Fluorescence imaging of ChR2/eYFP immunostaining at VMS shows that the region targeted by the optogenetic stimulation received transfected fibers originating at the OFC (as seen in Fig. 1). The OFC–VMS-transfected fibers were also clearly visualized using the PEGASOS technique. Our results similarly cannot be explained based on the light delivered to the VMS. Compared to the original study, although we also implanted 200 μm fibers, we used a larger numerical aperture (0.66) and higher light intensities that should result in the illumination of a wider area. Ahmari et al.20 used a solid-state 473 nm 100 mW laser, with light intensity set to 1–5 mW illumination at the fiber tip in the brain. In our approach, we used a 465 nm blue LED capable of generating 7 mW at the tip of the fiber, which corresponds to 55 mW/mm2. Light power densities of 2–5 mW/mm2 with a wavelength from 465–475 nm are sufficient to stimulate action potentials in neurons expressing ChR2 at typical experimental levels82. Finally, related to both these points, there was a clear difference in neural excitation between ChR2 and eYFP-only rats. In addition to the c-Fos results, our electrophysiological study confirmed that optogenetic stimulation of OFC-transfected projections in VMS acutely affected neuronal physiology (Fig. 5).

This failure of translation between rodent species suggests a need for caution when attempting to model compulsivity or other CSTC-linked human phenomena. There is functional homology across rodents and primates in at least three different regions of the striatum: the dorsal regions being specialized in motor aspects, such as planning and execution; the medial region involved in learning and attention; and the ventral region having a prominent role in reward and emotion processing12,83,84,85,86. Dopaminergic system organization in the striatum is also conserved across rats and mice87. However, the prolonged development and more complex and flexible behavioral repertoire of the rat compared with mice88 suggest that anatomic homology may not fully reflect circuit function. For example, rats use more complex strategies than mice do to locate the platform in the Morris water maze and, consequently, perform better than mice in learning spatial information88,89,90. Inhibition of adult neurogenesis in the hippocampus produces deficits in contextual fear conditioning behavior in rats but not mice91. Additional knowledge of functional homology, e.g., at the physiologic level, might be necessary for more accurate translational studies87.

Perhaps a stronger/prolonged protocol would be required to manipulate physiology to the degree of altering behavior in rats since, in humans, the establishment of compulsions can take years. We chose to stimulate the OFC–VMS projection for 6 days, unilaterally, as in the original study20, but it is conceivable that rats require longer stimulation windows. They may also require more intense, bilateral pathway manipulation to manifest a compulsive phenotype. Finally, it may also be necessary to consider the role of timing-dependent plasticity. Neither our work nor Ahmari et al.20 captured the timing of optogenetic stimulation relative to spontaneous grooming bouts. It may be that, by pure chance, the mice in the original study groomed frequently enough that optogenetic stimulation tended to coincide with their grooming. Mice in general show more stress responses than rats to being experimentally handled, disturbed, or moved between contexts, and grooming could be expressed as a distress behavior. If stimulation coincided with grooming and were in some way reinforcing, might have caused positive, self-reinforcing associations that led grooming to become habitized and overexpressed. With the advent of high-precision automatic behavior estimation tools92,93, this hypothesis could be tested in future work.

Taken at face value, our findings fit into an ongoing evolution in circuit theories of compulsivity. Initial CSTC hyperconnectivity theories emphasized OFC-originating circuits, including pathways targeting the ventral striatum19,60,94. There was a strong emphasis on anxiety and other negative-valence affective constructs. Those ideas are now evolving, with an increasing emphasis on more dorsal CSTC circuits involved in executive function and planning17 and on the role of other structures, such as amygdala94. Part of that evolution is a growing understanding that CSTC circuits are not segregated, but implement a spiral-like transfer of information between recurrent cortico-basal loops17,19,95. Although not yet proven, this interconnection between loops may increase with increasing brain size/complexity. As a result, grossly observable behaviors might be elicited in mice if the OFC–VMS circuit acts in relative isolation, but greater cross-loop integration in rats and primates might buffer or compensate for a circuit-limited perturbation, and an OFC–VMS circuit change might no longer produce behavior change. If true, this model suggests that for clinically severe compulsivity to emerge in a human brain, the CSTC circuitry might require multiple simultaneous “hits” that remove the capacity for compensation. Thus, a more translatable model might study the effect of multiple circuit disruptions in conjunction, or might combine this type of single-synapse manipulation on a background of either genetic loading or early life adversity. In that sense, our failure to translate this specific protocol highlights the need for rodent models that more fully capture the human pathway to illness. Developing those more robust models will be a significant challenge. For now, the use of different models that focus on specific aspects of compulsion-related disorders should be encouraged, as together they may better capture the multiple facets of the human disease.