Adaptability and reproducibility of a memory disruption rTMS protocol in the PharmaCog IMI European project

Abstract

Transcranial magnetic stimulation (TMS) can interfere with cognitive processes, such as transiently impairing memory. As part of a multi-center European project, we investigated the adaptability and reproducibility of a previously published TMS memory interfering protocol in two centers using EEG or fMRI scenarios. Participants were invited to attend three experimental sessions on different days, with sham repetitive TMS (rTMS) applied on day 1 and real rTMS on days 2 and 3. Sixty-eight healthy young men were included. On each experimental day, volunteers were instructed to remember visual pictures while receiving neuronavigated rTMS trains (20 Hz, 900 ms) during picture encoding at the left dorsolateral prefrontal cortex (L-DLPFC) and the vertex. Mixed ANOVA model analyses were performed. rTMS to the L-DLPFC significantly disrupted recognition memory on experimental day 2. No differences were found between centers or between fMRI and EEG recordings. Subjects with lower baseline memory performances were more susceptible to TMS disruption. No stability of TMS-induced memory interference could be demonstrated on day 3. Our data suggests that adapted cognitive rTMS protocols can be implemented in multi-center studies incorporating standardized experimental procedures. However, our center and modality effects analyses lacked sufficient statistical power, hence highlighting the need to conduct further studies with larger samples. In addition, inter and intra-subject variability in response to TMS might limit its application in crossover or longitudinal studies.

Introduction

Transcranial Magnetic Stimulation (TMS) is a non-invasive technique allowing painless stimulation of the brain, in which brief pulses of current flowing through a coil of wire generate a time-varying magnetic field pulse. The rate of change of the magnetic field determines the induction of a secondary current in a conducting living tissue such as the cortical surface, and this secondary current may lead to the depolarization of the underlying populations of neurons1. Although TMS is primarily used in the study of the corticospinal motor system in neurology and neurophysiology2, TMS and repetitive TMS (rTMS) have been widely used for many years in the fields of cognitive neuroscience and neuropsychology3. Depending on the experimental conditions, TMS can temporarily enhance cognitive functions4,5 or, conversely, transiently interfere with major cognitive domains, thereby helping to obtain causal inferences on the role of the stimulated region in behavior. Moreover, TMS can be coupled with information from functional neuroimaging techniques6, further enhancing its application in studies on cognitive neuroscience. Imaging information can be used to guide stimulation (increasing the spatial precision of the brain area to be stimulated) and to investigate the effects induced on cerebral networks in terms of their functional reorganization in response to the magnetic pulses and how this relates to a given behavioral outcome.

In this study, we incorporated the use of rTMS into one of the experimental arms within the European Commission Seventh Framework Programme (FP7/2007–2013, grant n° 115009), the Innovative Medicine Initiative’s (IMI) ‘PharmaCog’ project (http://www.imi.europa.eu/content/pharma-cog), which focuses on the early stages of drug development for Alzheimer’s disease (AD)7. A series of ‘cognitive challenge experiments’ (including TMS, but also sleep deprivation) were performed in healthy young human volunteers, which involved transiently disrupting cognitive domains relevant to AD. Once the efficacy of the challenge models was established, the reversibility of the induced dysfunction would be tested by employing distinct pharmacological products. Hence, the overarching idea was that the approach would produce experimental platforms that could be used to test for early indications of the efficacy of newly developed drugs.

The aim of the present study was to test the adaptability and reproducibility of a TMS protocol that has been previously reported2 to interfere with memory processes. We tested this protocol in two centers using electroencephalogram (EEG) or functional magnetic resonance imaging (fMRI) during the memory recognition phase (see below). To our knowledge, no studies have been published to date that have tested the replicability of the cognitive effects of a TMS protocol in separate experimental sites. The implementation of non-invasive brain stimulation protocols in large clinical trials requires the development of standardized protocols that can be used in multiple centers8. Another relevant, but yet untested, aspect for the potential incorporation of TMS protocols in clinical trials involving longitudinal or cross-over designs is to investigate the stability (i.e., test-retest reproducibility) of the observed findings, which was another aim of the present study.

In this study, we compared the effects of rTMS stimulation on recognition memory between the two centers at Marseille (MRS) and Barcelona (BCN) and the two modalities, EEG and fMRI. This manuscript focuses only on the behavioral findings (i.e., memory interference). Putative changes in brain activity/connectivity underlying the observed effects will be analyzed in separate publications.

Results

Effects of TMS on reaction times (RTs) during memory encoding and on visual analog scale (VAS) ratings

Although our main outcome variable was memory performance during the recognition phase of the memory task, we also analyzed the putative effects of rTMS on the accuracy of memory encoding, their respective reaction times (RTs) and subjective perception scales. There were no main effects on the accuracy of encoding. For the whole sample (n = 64), RTs were longer when the left dorsolateral prefrontal cortex (L-DLPFC) was under active stimulation only on experimental day 2. However, when focusing on the “sensitive” subsample that participated on both days 2 and 3, there were no differences in the RTs recorded on day 2 between L-DLPFC and vertex stimulation. Regarding the visual analog scale (VAS), volunteers showed less contentment and more annoyance after stimulation on day 2 (see Supplementary Material).

Main effects of TMS on memory performance (day 1 vs day 2)

Regarding the impact of TMS on recognition memory performance (n = 68), ANOVA showed a main effect for Condition (Hits %: F(1,64) = 11.95, p = 0.001, ηp2 = 0.157). A Condition x Time interaction was also observed for this variable (F(1,64) = 14.85, p < 0.0005, ηp2 = 0.188). No significant effects were observed for Time (Hits %: F(1,64) = 2.35, p = 0.131, ηp2 = 0.035) or for the factors Center (Hits %: F(1,64) = 0.004, p = 0.947, ηp2 < 0.0005) or Modality (Hits %: F(1,64) = 0.29, p = 0.589, ηp2 = 0.005). These results reveal that active rTMS to the L-DLPFC interfered with memory performance when compared to the stimulation of the vertex at both experimental sites, regardless of whether fMRI or EEG was used.

Post hoc analysis confirmed that memory performance was lower when the L-DLPFC was stimulated than when the vertex was stimulated only on day 2 (Hits %: t(67) = −5.09, p < 0.0005), when active stimulation was applied, but not on day 1 (Hits %: t(67) = 0.16, p < 0.872), when a sham coil was used as placebo (Fig. 1). These trends remained when sub-analyses considering each modality and center were undertaken (Fig. 2).

Figure 1
figure1

rTMS at the L-DLPFC induced memory interference. Behavioral results from the recognition memory tasks undertaken on experimental days 1 and 2 are shown as mean Hits %. Error bars correspond to the standard error of the mean (SEM).

Figure 2
figure2

Interference effects of TMS across centers and modalities. Hits % consistently showed a significant reduction in performance in response to rTMS at the L-DLPFC compared to the vertex on day 2 for the: (a) fMRI group (*t(55) = −4.16, p < 0.0005); (b) EEG group (*t(11) = −3.10, p = 0.01); (c) Barcelona group (*t(49) = −3.91, p < 0.0005) and (d) Marseille group (*t(17) = −3.38, p = 0.004). Error bars correspond to SEM.

Reproducibility of the effects of TMS (day 1 vs day 2 vs day 3)

As described in the Methods, subjects showing the largest response to rTMS interference on day 2 were invited to attend an equivalent session on day 3, which was conducted 15 days later on average. When comparing the data collected from these 21 individuals over the three days, a main effect for Condition was found (Hits %: F(1,17) = 13.09, p = 0.002, ηp2 = 0.435). A Time x Condition interaction was also significant (Hits %: F(1.403,34) = 17.05, p < 0.0005, ηp2 = 0.501), while there was no main effect for Time (Hits %: F(2,34) = 2.52, p = 0.095, ηp2 = 0.129). Post hoc analyses revealed significantly lower memory performance when the L-DLPFC was stimulated than when the vertex was stimulated only for day 2 (Hits %: t(20) = −10.05, p < 0.0005), but not for day 3 (Hits %: t(20) = −0.77, p = 0.448), when active rTMS was also applied, or day 1 (Hits %: t(20) = 0.48, p = 0.634) (Fig. 3). Similar to the main effects observed in the whole sample, we did not observe any significant effects of Center (Hits %: F(1,17) = 1.11, p = 0.306, ηp2 = 0.061) or Modality (Hits %: F(1,17) = 0.74, p = 0.402, ηp2 = 0.042).

Figure 3
figure3

Reproducibility of TMS-induced interference. Behavioral results from the recognition memory tasks undertaken on days 1, 2 and 3 are shown as mean Hits % in a subsample of the subjects (n = 21, p < 0.05). Error bars correspond to SEM.

Finally, to identify possible predictors of TMS interference, we compared baseline memory performance (i.e., on day 1 when sham rTMS was used) between the 26 individuals identified as sensitive to TMS (responders) and the 42 subjects who were not sensitive (non-responders) on day 2. We found that the responders exhibited lower memory performance at baseline for both sham conditions (L-DLPFC, Hits %: t(66) = 2.29, p = 0.025; vertex, Hits %: t(66) = 2.53, p = 0.014) than the non-responders (Fig. 4).

Figure 4
figure4

Baseline differences in memory performance (p < 0.05) between non-responders and responders to TMS for both L-DLPFC and vertex stimulation on experimental day 1. Error bars correspond to SEM.

Table 1 shows the mean performances and standard deviations (SD) for encoding and recognition for all the dependent variables from experimental days 1, 2 and 3.

Table 1 Summary data for all the performance dependent variables. Mean values ± SD for the whole sample included in the ANOVAs performed.

Discussion

In this study, we were able to independently replicate a previously published TMS protocol, although some modifications were made to the original protocol. Thus, overall, our findings suggest that TMS could be incorporated into standardized protocols for multi-center investigations aiming to transiently impair episodic memory in humans. However, we also observed inter-individual variability in response to TMS and failed to demonstrate reproducibility of interference in individuals who had initially responded to TMS.

Over the years, accumulating evidence has unequivocally demonstrated the capacity of non-invasive brain stimulation to modulate cognition in humans5,9,10 (for reviews). Despite this, there is a lack of standardized designs and procedures for modulating cognition, which is in sharp contrast to the widely established procedures used to investigate motor cortex functions with TMS11,12,13. Furthermore, in cognitive studies using non-invasive brain stimulation, attempts to replicate findings with published protocols are very scarce. For example, while meta-analytical evidence indicates that a single session of transcranial direct current stimulation (tDCS) modulates linguistic functions14, recent attempts to reproduce particular findings have failed to replicate such observations15.

To our knowledge, there are very few studies by independent groups that have been explicitly designed to replicate cognitive findings with TMS. To this end, we selected a memory interfering protocol whose effects have been reported in several studies, with overlapping samples used across studies in some cases16. Here, we demonstrate that an adaptation of a memory disrupting TMS protocol by two separate research groups was able to confirm the overall expected effects. However it should be noted that the results deriving from the comparison of both center and modality effects, were statistically underpowered in our study. Hence, further studies are needed including larger samples.

In our study, the average reduction in recognition memory induced by L-DLPFC stimulation compared to vertex stimulation was around 7%. This varies from previously reported reductions in performance, which have ranged from 20%17 to 24%18. The significant effect of TMS detected in this study could be due to the larger sample used here compared to previous studies (i.e., group sizes have typically ranged from 13 to 28 individuals). One of the studies by Rossi et al.16 used a larger sample (n = 66), but this was divided into two subgroups, old and young subjects. The different magnitudes of the reductions in performance that we observed compared to those of other studies might be also related to the fact that we made some adaptations to the TMS protocol to fulfill PharmaCog project requirements and standards7.

Adaptations of the published protocol included performing TMS using fMRI for guiding instead of anatomical landmark, as earlier studies have indicated that fMRI-guided TMS neuronavigation might produce the strongest behavioral effects (e.g.19). Using this approach, the stimulation target was placed posteriorly and laterally within the left middle frontal gyrus (L-MFG) compared to the average estimated F3 standard MNI space20 (Fig. 5). Previous studies on memory using TMS have found that the stimulation target might play an important role in the behavioral cognitive outcomes observed. For example, Blumenfeld et al.21 found that stimulating the left ventrolateral prefrontal cortex (L-VLPFC) before a verbal encoding task produced subsequent memory disruption, whereas TMS to the L-DLPFC facilitated recognition memory compared to vertex stimulation.

Figure 5
figure5

TMS at the L-DLPFC. Left hemisphere sagittal view. Green circle represents the stimulation point for the L-DLPFC, within MNI space coordinates (x, y, z) of −42, 10, 30. Blue circle represents the F3 location, within MNI space coordinates (x, y, z) of −34, 26, 44.

Our study sample was composed entirely of men, whereas previous investigations have included both genders. Data from studies using both tDCS and TMS protocols suggest that, overall, females may be more responsive to stimulation than men22 for motor23,24, visual25 and some cognitive domains26,27. Accordingly, an unpublished post hoc analysis from the original series in the literature using the adapted protocol found that most of the disruptive effect of TMS was observed in women (Rossi, S., personal communication). Nevertheless, no women were included in the study due to protocol restrictions.

Differences in task difficulty could have also contributed to the smaller reduction in performance that we observed compared to previous studies, as task difficulty is likely to interact with TMS28,29. However, overall performance in response to vertex (sham) stimulation was 79.04% (SD = 12.49) of hits in our study, which is comparable to those reported previously (i.e., hits of 74%17, 76.2%30, 79%31 and 72%18). Further, it should be noted that subjects were instructed with specific emphasis to intentionally remember the encoded stimulus in our protocol, whereas incidental memory encoding was undertaken in the other studies16,17,18. It has been reported that when considering a semantic level of processing at encoding (e.g., category classification equivalent to indoor/outdoor used in the present model), there are no significant differences in memory or brain activity patterns between intentional and incidental encoding29,32,33,34,35. However, differences in performance may arise when the level of processing (i.e., deep/semantic vs shallow encoding) is specifically manipulated29,33,36,37,38, which was not the case in this study or the previous ones. When comparing overall memory performances between our study and previous reports17,18, our results appear to provide further evidence of the importance of ‘level of processing’ over ‘incidental vs intentional encoding’ to explain different memory outcomes.

Finally, a direct comparison of the effects of sham and verum stimulation of the vertex revealed no differences and no disruption in memory. This was in accordance with previous findings that suggest that the vertex is a valid control for assessing memory function with TMS17,21,29,39.

Despite the overall effects observed, only 40% of the participants showed a significant response to TMS interference (i.e., the reduction in memory performance when comparing L-DLPFC to vertex stimulation was at least −1 SD from vertex mean performance). This is consistent with previous results, such as those of López-Alonso et al.40, who reported responsiveness values reaching 40% in the expected direction depending on the TMS and tDCS protocols used. However, other studies have reported higher rates of responders, including 60% amongst healthy young participants subjected to intermittent theta burst stimulation (iTBS) protocols41, 67%42, 76%43 and 78%44 among those subjected to paired-associative stimulation (PAS) protocols, and 75%45,46 for those subjected to TMS protocols.

Inter-individual variability is increasingly being recognized as an important factor explaining the findings and discordances in motor studies11,42. Inter-subject variability could be due to methodological issues, such as coil orientation47, subject characteristics, including age22,42 and gender23,25, the time of day48, genetics49, baseline level of excitability42,50,51 or short latency intracortical inhibition40 (SICI).

As described before, the only variable that predicted TMS response was baseline memory performance. Individuals with lower recognition memory performance at baseline were more susceptible to the disruptive effects of TMS. Indeed, it has been reported that participants with high memory performance at baseline may implement more efficient compensatory processes, making them more resistant to the effects of TMS than those with low memory performance52. This observation is consistent with earlier tDCS reports, in which baseline performance was linked to greater positive53,54 or negative55 cognitive effects of stimulation. It is also in line with a recent transcranial alternating current stimulation (tACS) investigation, in which tACS at the prefrontal cortex increased fluid intelligence capabilities in those with slow baseline performance, but not in those with fast baseline performance56. In our study, the larger response to TMS in those with lower baseline memory performance might indicate reduced resilience or less optimal engagement of brain plasticity mechanisms to counteract the effects of TMS. Differences in cognitive reserve, which has been proposed to reflect brain plasticity and is associated with greater efficiency of memory networks57, could in principle be associated with the observed differences between the subjects. However, the number of years of education, a common proxy for reserve, and a variable previously shown to interact with the effects of brain stimulation (i.e.58), were similar between the responders and non-responders. In any case, determining the mechanisms underlying resilience to the effects of TMS requires neurophysiological data, such as comparisons of brain activity/connectivity patterns during the memory encoding task that was used to guide TMS, which were not available in our study.

We were unable to replicate the initial effects of TMS on memory in the same individuals when they were subjected to TMS again 15 days later. Several studies attempting to reproduce the effects of TMS have reported negative findings44,45,46,59,60,61, while others have shown stable effects of TMS across different sessions41,62. Importantly, none of these reports have assessed reproducibility in the same individuals in distinct experimental sessions.

López-Alonso et al.40 used different brain stimulation protocols on the same subjects to evaluate intra-subject variability in their responses, reporting that 39%, 45% and 43% of the subjects responded as expected to PAS25, AtDCS and iTBS, respectively, but only 12% of the individuals responded to all the protocols in the expected direction. In our study, we selected 21 individuals who had presented a clear response to TMS for retesting 15 days later, but only 19% (n = 4) of these participants responded consistently to TMS during both experimental sessions. Hence, a particular response at one experimental time point may not be predictive of the same response at a later time point when considering particular individuals (see Fig. 6 for a clear depiction of the performance of the “sensitive” subjects across the experimental sessions). Although our findings indicate a lack of reproducibility of the adapted protocol used, it should be noted that these observations were based only on a smaller subgroup of individuals that attended sessions on both days 2 and 3. Furthermore, the selection of these individuals was biased on purpose as they were specifically selected for their greater response to TMS interference on day 2. This, therefore, does not exclude the possibility that statistical limitations, such as regression to the mean effects, could have been responsible for the lack of effect observed on day 363,64.

Figure 6
figure6

Scatter plot showing individual L-DLPFC-Vertex hits % subtraction (difference) for the subsample that attended all the sessions on days 1, 2 and 3 (n = 21). The thick black dashed line corresponds to the mean values for each day. Negative values correspond to the lower performance in response to L-DLPFC stimulation compared to vertex stimulation.

Intra-subject variability could have been due to incomplete overlapping of the stimulation site between the sessions65, fluctuations in the subjects’ attention within and between sessions66,67, the individual’s history of physical activity68 or variations in the levels of the stress hormone cortisol69,70. In our study, the first possibility seems unlikely to have contributed to our findings as TMS was applied using the same neuronavigated fMRI-based coordinates during the two experimental sessions. By contrast, some attentional biases during rTMS to the DLPFC could have affected our results, as subjective perception of annoyance was increased and contentment decreased after TMS on day 2. Moreover, RTs at encoding were higher when the L-DLPFC was stimulated than when the vertex was stimulated on day 2, which might reflect a distractibility effect on the RTs of responses, but not encoding accuracy. However, it should be noted that this latter effect was not observed when only considering the responders (n = 21). In addition, the blocks of vertex and L-DLPFC stimulation were administered in an interleaving and continuous manner throughout all the experimental sessions. Therefore, as stimulation was not stopped until the end of the encoding task, subjective ratings probably reflected the overall ratings, eclipsing the effects of L-DLPFC and vertex stimulation; thus, the observed changes cannot be specifically attributed to the effect of L-DLPFC stimulation on day 2.

In conclusion, our study replicated an existing cognitive TMS protocol, despite its adaptations for specific experimental purposes. We did not observe any differences in the cognitive effects of TMS between the research center or modality used, but these sub-analyses were underpowered (data not shown). Consequently, further studies with larger samples are needed in order to confirm a lack of center or modality effects. Our data suggest that recruiting individuals exhibiting low baseline memory performance may result in greater observable effects of TMS interference. Stimulating the vertex using either a real or sham TMS coil confirmed that the vertex is a good control for studying visual memory as its stimulation did not disrupt memory. Finally, the effects of TMS could not be reproduced across different time points in a subsample of previously responsive subjects, an issue that needs to be addressed in further TMS investigations.

Method

Memory interference task and experimental design

Our review of the literature9 identified a procedure used in several publications that involved the application of high frequency rTMS during visual memory encoding that disrupted memory performance during a later recognition phase17. After some adaptations (see Supplementary Material), we created three equivalent tasks to be undertaken in a counterbalanced order across three experimental days, as depicted in Fig. 7.

Figure 7
figure7

Study design. (a) General experimental design with four main time points and requirements that subjects had to meet to complete the whole study. SCR refers to screening. L-DLPFC vs VERTEX DROP ≥ 1 SD* refers to a reduction of at least 1 standard deviation (12.5%) in the Recognition performance during L-DLPFC stimulation condition compared to that during vertex stimulation condition. (b) Memory task performed on each experimental day (day 1, day 2 and day 3). MTH refers to motor threshold estimation. Drug test consisted of taking a urine sample and a breath test.

On screening day, subjects were familiarized with a short version of the memory task. Subjects had to meet the eligibility criteria based on the inclusion/exclusion criteria detailed in Supplementary Material. During the second visit (experimental day 1), volunteers performed a complete encoding-retrieval memory task while receiving sham TMS. Hence, results from day 1 were used to determine the baseline performance of each individual under sham brain stimulation. Individuals who could not correctly recognize at least 60% of the items (hits % vertex + hits % L-DLPFC/2) were excluded from the final sample. This was done to avoid including individuals who were performing to chance levels, which would correspond to 50% of the performance during recognition memory. On the third day (experimental day 2), selected subjects performed an equivalent version of the encoding-retrieval memory task while subjected to active TMS. Finally, 15 days later, a subsample of subjects exhibiting a decrease in memory performance of at least 1 SD in response to L-DLPFC stimulation when compared to vertex stimulation on day 2 were invited to undergo an identical session (experimental day 3) to test for reproducibility of the effects of TMS. A reduction of −1 SD was considered to reflect a ‘transient memory dysfunction’ induced by TMS. Although a drop of 1.5 SD is used to indicate cognitive impairment in a clinical context, we considered a decrease of 1 SD to be sufficient due to the experimental nature of our study and the fact that it was conducted on healthy young subjects. During both experimental days 2 and 3 (real/active TMS), individuals performed the recognition memory task either inside an MRI scanner or while wearing an EEG cap (see below for the distribution of groups). The study protocol was approved by the French ethics committee “SUD MÉDITERRANÉE I”, the French regulatory authority Agence Nationale de Sécurité du Médicament (ANSM) and the Spanish committee “Comité Ético de Investigación Clínica de l’Hospital Clínic” (CEIC) in Barcelona. The study was in accordance with the Declaration of Helsinki. All volunteers were properly informed and gave written consent. The study was registered in ClinicalTrials.gov for locations in Spain and France (number identifier: NCT01861639, registered on May 23rd 2013).

The memory encoding task consisted of 6 blocks containing 12 pictures each (50% indoor, 50% outdoor; see Fig. 8a). After a 30-minute break, subjects performed the recognition memory task, in which they were shown 48 new pictures and 24 old pictures during vertex and L-DLPFC stimulation (Fig. 8d). The recognition task was performed in the same experimental room as the encoding task on experimental day 1. On experimental days 2 and 3, subjects performed the recognition memory task in an MRI scanner or while wearing an EEG cap.

Figure 8
figure8

Memory task. In the encoding part, (a) each trial consisted of a fixation cross (variable timing), a red cross (warning 1 sec), a picture (1 sec) and a green cross (1 sec). Participants were asked to answer whether the picture was of an indoor or an outdoor scene by pressing the “z” or “m” key, respectively, on a standard computer keyboard after the appearance of the green cross. (b) Schematic depiction of the TMS protocol, which consisted of applying a 900-ms rTMS train 500 ms after the appearance of the picture. (c) Stimulation was administered in an alternating manner across the 6 blocks of pictures over two brain regions (vertex and L-DLPFC). The order of stimulation was randomly assigned to each subject that remained unchanged across the experimental days. In the recognition part of the task, (d) each trial included a fixation cross, a red cross (1 sec), a picture (2 sec) and a green cross (1 sec). Participants were asked to answer if they had seen or not each picture by pressing the “z” or “m” keys on a standard computer keyboard, respectively, or on an MRI-compatible keyboard where the left button corresponded to “saw pictures” and the right button to “did not see pictures”.

Sample

A total of 68 healthy young individuals (mean age: 24 years; SD: 4) participated in the study, and 21 completed all the three experimental sessions (fMRI group: 56, 44 from BCN; EEG group: 12, 6 from BCN). All the subjects were male due to protocol restrictions (see Supplementary Material).

MRI-guided TMS protocol

TMS was applied using a MagPro X100 magnetic stimulator (MagVenture A|S, Denmark) combined with an eXimia Navigated Brain Stimulation system (Nexstim, Finland) for the BCN subsample and a Magstim stimulator (Magstim Company Limited, USA, CE certification) combined with the neuronavigation system Brainsight 2.2 (Rogue Research Inc., Montreal, QC, Canada) for the MRS subsample. The resting motor threshold (rMTH) was determined at each experimental session as described in the International Standard Guidelines71. High-frequency (20 Hz) 900-ms TMS trains were applied 500 ms after the onset of the picture presentation (this timing of stimulation exerts the clearest effects on memory interference18) at a 90% intensity of the individual rMTH. Stimulation was administered in an alternating manner across the 6 blocks of pictures over two brain regions (vertex and L-DLPFC). The order of stimulation was randomly assigned to each subject that remained unchanged across the experimental sessions (see Fig. 8b,c). The vertex site (Cz location according to the 10–20 electrode placement72) was used as the control area, while the L-DLPFC site (determined from a previous fMRI memory study briefly described in Supplementary Material; L-DLPFC is widely associated with encoding processes73,74) was used as the experimental area. The region corresponds to the intersection between the Brodmann areas 9/46, the boundary between the L-MFG and the left inferior frontal gyrus (L-IFG) mean peak activation voxel according to the Montreal Neurological Institute (MNI) coordinates (x, y, z) of −42, 10, 30 (see Fig. 7). Neuronavigated stimulation with stereotactic registration was performed to ensure accuracy in the localization and position of the TMS coil. To obtain a subjective response to the rTMS administration, we collected responses to visual analog scales (VAS) before and immediately after rTMS administration on experimental days 1, 2 and 3. These VAS scores included ratings for nervousness, contentment, sadness, hope and annoyance. The subjects marked on a 100-mm horizontal line the point that they felt best represented their perception of their current state.

Data analyses

For encoding, accuracy (defined as the percentage of items correctly categorized as indoor or outdoor) and RTs (defined as the mean reaction times on accuracy responses) were analyzed. Mixed ANOVAs were performed. Different sample sizes were used because of corrupted data (see Table 1). Condition (Vertex vs L-DLPFC) and Time (day 1 vs day 2) were entered as within-subject factors, while Center (BCN vs MRS) was used as a between-subject factor. Modality (EEG vs fMRI) was not entered because there were no differences in the protocol for the encoding phase. The same analyses were performed for day 3, but with three Time levels (day 1, day 2 and day 3, n = 19; Bonferroni correction was used for multiple comparisons of the main effects).

For recognition, Hits % (correctly recognized pictures) and RTs (the time elapsed from the presentation of a picture to the subsequent recognition response for each modality subsample) were the main measures of memory performance. Mixed ANOVAs were performed to evaluate the effects of TMS on memory performance (Hits % and RT). Time (two levels: day 1 and day 2) and Condition (two levels: L-DLPFC and Vertex) were entered as within-subject factors, while Center (two levels: MRS and BCN) was considered a between-subject factor. Modality (two levels: EEG and fMRI) was entered as a between-subject factor only for the Hits % ANOVA. For RTs, mixed ANOVA was performed for EEG and fMRI subsamples separately. To test for stability of the effects of TMS, we applied the same statistical model, adding another level to Time (day 1, day 2 and day 3 (n = 21, Bonferroni correction was used for multiple comparisons of the main effects)). The Greenhouse-Geisser correction was used if necessary to correct for non-sphericity. False alarm (FA), which is a false recognized item, was included as a performance index in Table 1, regardless is not a discriminant variable for rTMS effect, but give us a comprehensive perspective of the initial performance level of the included subjects. All effects are reported as significant if p < 0.05. ANOVA’s main effects and interactions were further assessed using post hoc t tests. Data management and analysis were performed using the Statistical Package for the Social Sciences version 17.0 (SPSS Inc.).

To classify subjects as being sensitive to rTMS or not, mean performances during vertex stimulation were considered the benchmark and all performances during L-DLPFC stimulation condition that were below 12.5% of the corresponding performance during vertex stimulation (i.e., corresponding to −1 SD of the group distribution) were considered to be significantly disrupted. Therefore, subjects showing a reduction of at least 1 SD were considered responders and eligible to participate on day 3.

For VAS analysis, we calculated a “change” (after rTMS – before rTMS) for each day and scale, with positive values indicating higher ratings after rTMS. Comparisons for related samples were performed to analyze differences in the subjects’ perceptions between active and sham stimulation.

The datasets generated and/or analyzed in this study are available upon request.

References

  1. 1.

    Hallett, M. Transcranial magnetic stimulation and the human brain. Nature 406, 147–150 (2000).

    ADS  Article  PubMed  CAS  Google Scholar 

  2. 2.

    Rossini, P. M. & Rossi, S. Transcranial magnetic stimulation: Diagnostic, therapeutic, and research potential. Neurology 68, 484–488 (2007).

    Article  PubMed  Google Scholar 

  3. 3.

    Walsh, V. & Pascual-Leone, A. Transcranial magnetic stimulation: a neurochronometrics of mind. (MIT Press, 2003).

  4. 4.

    Guse, B., Falkai, P. & Wobrock, T. Cognitive effects of high-frequency repetitive transcranial magnetic stimulation: a systematic review. J. Neural Transm. 117, 105–122 (2010).

    Article  PubMed  Google Scholar 

  5. 5.

    Bartrés-Faz, D. & Vidal-Piñeiro, D. Noninvasive brain stimulation for the study of memory enhancement in aging. Eur. Psychol. 21, 41–54 (2016).

    Article  Google Scholar 

  6. 6.

    Paus, T. Imaging the brain before, during, and after transcranial magnetic stimulation. Neuropsychologia 37, 219–24 (1999).

    ADS  Article  PubMed  CAS  Google Scholar 

  7. 7.

    Galluzzi, S. et al. Clinical and biomarker profiling of prodromal Alzheimer’s disease in workpackage 5 of the Innovative Medicines Initiative PharmaCog project: a ‘European ADNI study’. J. Intern. Med. 279, 576–591 (2016).

    Article  PubMed  CAS  Google Scholar 

  8. 8.

    Floel, A. & Cohen, L. G. Recovery of function in humans: Cortical stimulation and pharmacological treatments after stroke. Neurobiol. Dis. 37, 243–251 (2010).

    Article  PubMed  CAS  Google Scholar 

  9. 9.

    Martin-Trias, P. et al. Translational Challenge Models in Support of Efficacy Studies: Neurobehavioral and Cognitive Changes Induced by Transcranial Magnetic Stimulation in Healthy Volunteers. CNS Neurol. Disord. Drug Targets 15, 802–815 (2016).

    Article  PubMed  CAS  Google Scholar 

  10. 10.

    Luber, B. & Lisanby, S. H. Enhancement of human cognitive performance using transcranial magnetic stimulation (TMS). Neuroimage 85, 961–970 (2014).

    Article  PubMed  Google Scholar 

  11. 11.

    Hamada, M., Murase, N., Hasan, A., Balaratnam, M. & Rothwell, J. C. The role of interneuron networks in driving human motor cortical plasticity. Cereb. Cortex 23, 1593–1605 (2013).

    Article  PubMed  Google Scholar 

  12. 12.

    Vallence, A. M. et al. Inter- and intra-subject variability of motor cortex plasticity following continuous theta-burst stimulation. Neuroscience 304, 266–278 (2015).

    Article  PubMed  CAS  Google Scholar 

  13. 13.

    Lin, T. et al. Effects of Theta Burst Stimulation on Suprahyoid Motor Cortex Excitability in Healthy Subjects. Brain Stimul. 10, 91–98 (2017).

    Article  PubMed  Google Scholar 

  14. 14.

    Price, A. R., McAdams, H., Grossman, M. & Hamilton, R. H. A meta-analysis of transcranial direct current stimulation studies examining the reliability of effects on language measures. Brain Stimul. 8, 1093–1100 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  15. 15.

    Vannorsdall, T. D. et al. Reproducibility of tDCS Results in a Randomized Trial. Cogn. Behav. Neurol. 29, 11–17 (2016).

    Article  PubMed  Google Scholar 

  16. 16.

    Rossi, S. et al. Age-Related Functional Changes of Prefrontal Cortex in Long-Term Memory: A Repetitive Transcranial Magnetic Stimulation Study. J. Neurosci. 24, 7939–7944 (2004).

    Article  PubMed  CAS  Google Scholar 

  17. 17.

    Rossi, S. et al. Prefontal cortex in long-term memory: an ‘ interference’ approach. 4 (2001).

  18. 18.

    Rossi, S. et al. Temporal dynamics of memory trace formation in the human prefrontal cortex. Cereb. Cortex 21, 368–373 (2011).

    Article  PubMed  Google Scholar 

  19. 19.

    Sack, A. T. et al. Optimizing Functional Accuracy of TMS in Cognitive Studies: A Comparison of Methods. J. Cogn. Neurosci. 21, 207–221 (2009).

    Article  PubMed  Google Scholar 

  20. 20.

    Keeser, D. et al. Prefrontal Transcranial Direct Current Stimulation Changes Connectivity of Resting-State Networks during fMRI. J. Neurosci. 31, 15284–15293 (2011).

    Article  PubMed  CAS  Google Scholar 

  21. 21.

    Blumenfeld, R. S., Lee, T. G. & D’Esposito, M. The effects of lateral prefrontal transcranial magnetic stimulation on item memory encoding. Neuropsychologia 53, 197–202 (2014).

    Article  PubMed  Google Scholar 

  22. 22.

    Todd, G., Kimber, T. E., Ridding, M. C. & Semmler, J. G. Reduced motor cortex plasticity following inhibitory rTMS in older adults. Clin. Neurophysiol. 121, 441–447 (2010).

    Article  PubMed  Google Scholar 

  23. 23.

    Chaieb, L., Antal, A. & Paulus, W. Gender-specific modulation of short-term neuroplasticity in the visual cortex induced by transcranial direct current stimulation. Vis. Neurosci. 25, 77–81 (2008).

    Article  PubMed  Google Scholar 

  24. 24.

    De Gennaro, L. et al. Callosal effects of transcranial magnetic stimulation (TMS): The influence of gender and stimulus parameters. Neurosci. Res. 48, 129–137 (2004).

    Article  PubMed  Google Scholar 

  25. 25.

    Kuo, M.-F., Paulus, W. & Nitsche, M. A. Sex differences in cortical neuroplasticity in humans. Neuroreport 17, 1703–1707 (2006).

    Article  PubMed  Google Scholar 

  26. 26.

    Fumagalli, M. et al. Brain switches utilitarian behavior: Does gender make the difference? PLoS One 5 (2010).

  27. 27.

    Knops, A., Nuerk, H. C., Sparing, R., Foltys, H. & Willmes, K. On the functional role of human parietal cortex in number processing: How gender mediates the impact of a ‘virtual lesion’ induced by rTMS. Neuropsychologia 44, 2270–2283 (2006).

    Article  PubMed  Google Scholar 

  28. 28.

    Beck, S. & Hallett, M. Surround inhibition is modulated by task difficulty. Clin. Neurophysiol. 121, 98–103 (2010).

    Article  PubMed  CAS  Google Scholar 

  29. 29.

    Innocenti, I. et al. Event-related rTMS at encoding affects differently deep and shallow memory traces. Neuroimage 53, 325–330 (2010).

    Article  PubMed  Google Scholar 

  30. 30.

    Floel, A. et al. Prefrontal Cortex Asymmetry for Memory Encoding of Words and Abstract Shapes. Cereb. Cortex 14, 404–409 (2004).

    Article  PubMed  Google Scholar 

  31. 31.

    Rossi, S. et al. Prefrontal and parietal cortex in human episodic memory: An interference study by repetitive transcranial magnetic stimulation. Eur. J. Neurosci. 23, 793–800 (2006).

    Article  PubMed  Google Scholar 

  32. 32.

    Becker, N., Kalpouzos, G., Persson, J., Laukka, E. J. & Brehmer, Y. Differential Effects of Encoding Instructions on Brain Activity Patterns of Item and Associative Memory. J. Cogn. Neurosci. 29, 545–559 (2017).

    Article  PubMed  Google Scholar 

  33. 33.

    Craik, F. I. M. & Tulving, E. Depth of processing and the retention of words in episodic memory. J. Exp. Psychol. Gen. 104, 268–294 (1975).

    Article  Google Scholar 

  34. 34.

    Stark, C. E. L. & Okado, Y. Making memories without trying: medial temporal lobe activity associated with incidental memory formation during recognition. J. Neurosci. 23, 6748–53 (2003).

    Article  PubMed  CAS  Google Scholar 

  35. 35.

    Buckner, R. L., Wheeler, M. E. & Sheridan, M. A. Encoding processes during retrieval tasks. J. Cogn. Neurosci. 13, 406–15 (2001).

    Article  PubMed  CAS  Google Scholar 

  36. 36.

    Vidal-Piñeiro, D. et al. Task-dependent Activity and Connectivity Predict Episodic Memory Network-based Responses to Brain Stimulation in Healthy Aging. Brain Stimul. 7, 287–296 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  37. 37.

    Otten, L. J., Henson, R. N. & Rugg, M. D. Depth of processing effects on neural correlates of memory encoding: relationship between findings from across- and within-task comparisons. Brain 124, 399–412 (2001).

    Article  PubMed  CAS  Google Scholar 

  38. 38.

    Otten, L. J. & Rugg, M. D. Task-dependency of the neural correlates of episodic encoding as measured by fMRI. Cereb. Cortex 11, 1150–60 (2001).

    Article  PubMed  CAS  Google Scholar 

  39. 39.

    Galli, G., Feurra, M., Pavone, E. F., Sirota, M. & Rossi, S. Dynamic changes in prefrontal cortex involvement during verbal episodic memory formation. Biol. Psychol. https://doi.org/10.1016/j.biopsycho.2017.02.008 (2017).

  40. 40.

    López-Alonso, V., Cheeran, B., Río-Rodríguez, D. & Fernández-Del-Olmo, M. Inter-individual variability in response to non-invasive brain stimulation paradigms. Brain Stimul. 7, 372–380 (2014).

    Article  PubMed  Google Scholar 

  41. 41.

    Hinder, M. R. et al. Inter- and intra-individual variability following intermittent theta burst stimulation: Implications for rehabilitation and recovery. Brain Stimul. 7, 365–371 (2014).

    Article  PubMed  Google Scholar 

  42. 42.

    Müller-Dahlhaus, J. F. M., Orekhov, Y., Liu, Y. & Ziemann, U. Interindividual variability and age-dependency of motor cortical plasticity induced by paired associative stimulation. Exp. Brain Res. 187, 467–475 (2008).

    Article  PubMed  Google Scholar 

  43. 43.

    Stefan, K., Wycislo, M. & Classen, J. Modulation of Associative Human Motor Cortical Plasticity by Attention. J. Neurophysiol. 92, 66–72 (2004).

    Article  PubMed  Google Scholar 

  44. 44.

    Fratello, F. et al. Modulation of corticospinal excitability by paired associative stimulation: Reproducibility of effects and intraindividual reliability. Clin. Neurophysiol. 117, 2667–2674 (2006).

    Article  PubMed  Google Scholar 

  45. 45.

    Maeda, F., Keenan, J. P., Tormos, J. M., Topka, H. & Pascual-Leone, A. Interindividual variability of the modulatory effects of repetitive transcranial magnetic stimulation on cortical excitability. Exp. brain Res. 133, 425–30 (2000).

    Article  PubMed  CAS  Google Scholar 

  46. 46.

    Maeda, F., Gangitano, M., Thall, M. & Pascual-Leone, A. Inter- and intra-individual variability of paired-pulse curves with transcranial magnetic stimulation (TMS). Clin. Neurophysiol. 113, 376–82 (2002).

    Article  PubMed  Google Scholar 

  47. 47.

    Talelli, P., Cheeran, B. J., Teo, J. T. H. & Rothwell, J. C. Pattern-specific role of the current orientation used to deliver Theta Burst Stimulation. Clin. Neurophysiol. 118, 1815–1823 (2007).

    Article  PubMed  CAS  Google Scholar 

  48. 48.

    Sale, M. V., Ridding, M. C. & Nordstrom, M. A. Factors influencing the magnitude and reproducibility of corticomotor excitability changes induced by paired associative stimulation. Exp. brain Res. 181, 615–26 (2007).

    Article  PubMed  Google Scholar 

  49. 49.

    Cheeran, B. et al. A common polymorphism in the brain-derived neurotrophic factor gene (BDNF) modulates human cortical plasticity and the response to rTMS. J. Physiol. 586, 5717–5725 (2008).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  50. 50.

    Hordacre, B. et al. Variability in neural excitability and plasticity induction in the human cortex: A brain stimulation study. Brain Stimul. 10, 588–595 (2017).

    Article  PubMed  Google Scholar 

  51. 51.

    Silvanto, J., Muggleton, N. & Walsh, V. State-dependency in brain stimulation studies of perception and cognition. Trends Cogn. Sci. 12, 447–454 (2008).

    Article  PubMed  Google Scholar 

  52. 52.

    Manenti, R., Cotelli, M., Robertson, I. H. & Miniussi, C. Transcranial brain stimulation studies of episodic memory in young adults, elderly adults and individuals with memory dysfunction: A review. Brain Stimul. 5, 103–109 (2012).

    Article  PubMed  Google Scholar 

  53. 53.

    Tseng, P. et al. Unleashing potential: transcranial direct current stimulation over the right posterior parietal cortex improves change detection in low-performing individuals. J Neurosci 32, 10554–10561 (2012).

    Article  PubMed  CAS  Google Scholar 

  54. 54.

    Hsu, T. Y., Tseng, P., Liang, W. K., Cheng, S. K. & Juan, C. H. Transcranial direct current stimulation over right posterior parietal cortex changes prestimulus alpha oscillation in visual short-term memory task. Neuroimage 98, 306–313 (2014).

    Article  PubMed  Google Scholar 

  55. 55.

    Learmonth, G., Thut, G., Benwell, C. S. Y. & Harvey, M. The implications of state-dependent tDCS effects in aging: Behavioural response is determined by baseline performance. Neuropsychologia 74, 108–119 (2015).

    Article  PubMed  Google Scholar 

  56. 56.

    Santarnecchi, E. et al. Individual differences and specificity of prefrontal gamma frequency-tACS on fluid intelligence capabilities. Cortex 75, 33–43 (2016).

    Article  PubMed  CAS  Google Scholar 

  57. 57.

    Bartrés-Faz, D. & Arenaza-Urquijo, E. M. Structural and functional imaging correlates of cognitive and brain reserve hypotheses in healthy and pathological aging. Brain Topogr. 24, 340–357 (2011).

    Article  PubMed  Google Scholar 

  58. 58.

    Berryhill, M. E. & Jones, K. T. tDCS selectively improves working memory in older adults with more education. Neurosci. Lett. 521, 148–151 (2012).

    Article  PubMed  CAS  Google Scholar 

  59. 59.

    Boroojerdi, B. et al. Reproducibility of intracortical inhibition and facilitation using the paired-pulse paradigm. Muscle Nerve 23, 1594–7 (2000).

    Article  PubMed  CAS  Google Scholar 

  60. 60.

    Humm, A. M., Z’Graggen, W. J., von Hornstein, N. E., Magistris, M. R. & Rösler, K. M. Assessment of central motor conduction to intrinsic hand muscles using the triple stimulation technique: normal values and repeatability. Clin. Neurophysiol. 115, 2558–66 (2004).

    Article  PubMed  CAS  Google Scholar 

  61. 61.

    Schilberg, L., Schuhmann, T. & Sack, A. T. Interindividual Variability and Intraindividual Reliability of Intermittent Theta Burst Stimulation-induced Neuroplasticity Mechanisms in the Healthy Brain. J. Cogn. Neurosci. 29, 1022–1032 (2017).

    Article  PubMed  Google Scholar 

  62. 62.

    Vernet, M. et al. Reproducibility of the effects of theta burst stimulation on motor cortical plasticity in healthy participants. Clin. Neurophysiol. 125, 320–326 (2014).

    Article  PubMed  Google Scholar 

  63. 63.

    Barnett, A. G., van der Pols, J. C. & Dobson, A. J. Regression to the mean: what it is and how to deal with it. Int. J. Epidemiol. 34, 215–220 (2004).

    Article  PubMed  Google Scholar 

  64. 64.

    Chiolero, A., Paradis, G., Rich, B. & Hanley, J. A. Assessing the Relationship between the Baseline Value of a Continuous Variable and Subsequent Change Over Time. Front. Public Heal. 1, 29 (2013).

    Google Scholar 

  65. 65.

    Carroll, T. J., Riek, S. & Carson, R. G. Reliability of the input-output properties of the cortico-spinal pathway obtained from transcranial magnetic and electrical stimulation. J. Neurosci. Methods 112, 193–202 (2001).

    Article  PubMed  CAS  Google Scholar 

  66. 66.

    Ellaway, P. H. et al. Variability in the amplitude of skeletal muscle responses to magnetic stimulation of the motor cortex in man. Electroencephalogr. Clin. Neurophysiol. 109, 104–13 (1998).

    Article  PubMed  CAS  Google Scholar 

  67. 67.

    Funase, K., Miles, T. S. & Gooden, B. R. Trial-to-trial fluctuations in H-reflexes and motor evoked potentials in human wrist flexor. Neurosci. Lett. 271, 25–8 (1999).

    Article  PubMed  CAS  Google Scholar 

  68. 68.

    Cirillo, J., Lavender, A. P., Ridding, M. C. & Semmler, J. G. Motor cortex plasticity induced by paired associative stimulation is enhanced in physically active individuals. J. Physiol. 587, 5831–5842 (2009).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  69. 69.

    Sale, M. V., Ridding, M. C. & Nordstrom, M. A. Cortisol Inhibits Neuroplasticity Induction in Human Motor Cortex. J. Neurosci. 28, 8285–8293 (2008).

    Article  PubMed  CAS  Google Scholar 

  70. 70.

    Clow, A. et al. Day differences in the cortisol awakening response predict day differences in synaptic plasticity in the brain. Stress 17, 219–223 (2014).

    Article  PubMed  Google Scholar 

  71. 71.

    Rossini, P. M. et al. Non-invasive electrical and magnetic stimulation of the brain, spinal cord, roots and peripheral nerves: Basic principles and procedures for routine clinical and research application: An updated report from an I.F.C.N. Committee. Clin. Neurophysiol. 126, 1071–1107 (2015).

    Article  PubMed  CAS  Google Scholar 

  72. 72.

    Jasper, H. H. Report of the committee on methods of clinical examination in electroencephalography. Electroencephalogr. Clin. Neurophysiol. Suppl. 10, 370–375 (1958).

    Article  Google Scholar 

  73. 73.

    Kapur, S. et al. Neuroanatomical correlates of encoding in episodic memory: levels of processing effect. Proc. Natl. Acad. Sci. USA 91, 2008–11 (1994).

    ADS  Article  PubMed  PubMed Central  CAS  Google Scholar 

  74. 74.

    Tulving, E., Markowitsch, H. J., Kapur, S., Habib, R. & Houle, S. Novelty encoding networks in the human brain: positron emission tomography data. Neuroreport 5, 2525–8 (1994).

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgements

This work was supported by the European Commission Seventh Framework Programme (FP7/2007–2013), the Innovative Medicine Initiative’s (IMI) ‘PharmaCog’ project. P. Martin-Trias holds a fellowship from the Ministerio de Educación y Ciencia, Spain (12135072325–79). We thank the volunteers who participated in this study.

Author information

Affiliations

Authors

Contributions

C.B., J.M., S.R., D.B.F., P.M.R., R.T. developed the research questions and designed the study. C.J., A.P.L., O.B., R.B. provided detailed comments and assistance on the initials versions. J.R., J.M., L.L., D.B.F. were in charge of the trial management. P.M.T., L.L., C.C.P., E.S. designed the experiments. C.B., N.M., J.S., S.R. adapted the protocol to E.E.G. conditions. P.M.T., E.S., S.F.C., J.J., D.B.F. adapted the protocol to M.R.I. conditions. P.M.T., L.L., E.S., C.C.P., S.F.C. assisted with participant recruitment and data acquisition. E.C., E.J. were in charge of data entry and verification. P.M.T., S.F.C., C.B., N.M., R.T., J.J. performed the data analyses. P.M.T., C.C.P., S.R., A.P.L., O.B., D.B.F. assisted in the writing of the manuscript. J.J., D.B.F., P.M.T., C.B., S.R., A.P.L. were involved in the interpretation of the results and discussion. All the authors have read and approved the final manuscript.

Corresponding author

Correspondence to David Bartrés-Faz.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Martin-Trias, P., Lanteaume, L., Solana, E. et al. Adaptability and reproducibility of a memory disruption rTMS protocol in the PharmaCog IMI European project. Sci Rep 8, 9371 (2018). https://doi.org/10.1038/s41598-018-27502-1

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing