Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

The response time paradox in functional magnetic resonance imaging analyses

Nature Human Behaviour volume 8, pages 349–360 (2024)Cite this article

Subjects

Abstract

Response times (RTs) are often the main signal of interest in cognitive psychology but are often ignored in functional MRI (fMRI) analyses. In fMRI analysis the intensity of the signal serves as a proxy for the intensity of local neuronal activity, but changes in either the intensity or the duration of neuronal activity can yield identical fMRI signals. Therefore, if RTs are ignored and pair with neuronal durations, fMRI results claiming intensity differences may be confounded by RTs. We show how ignoring RTs goes beyond this confound, where longer RTs are paired with larger activation estimates, to lesser-known issues where RTs become confounds in group-level analyses and, surprisingly, how the RT confound can induce other artificial group-level associations with variables that are not related to the condition contrast or RTs. We propose a new time-series model to address these issues and encourage increasing focus on what the widespread RT-based signal represents.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: How the amplitude and duration of neuronal signals interact to yield similar BOLD responses.
Fig. 2: Models assessed in the simulation study.
Fig. 3: Type I error as the RT difference between conditions increases.
Fig. 4: Power when the RT difference is 0.1 s as the condition difference increases.
Fig. 5: The relationship between signal duration and activation estimates and how it can cause RT effects to leak into group analyses when using ConsDurNoRT.
Fig. 6: Correlation between the contrast difference and the difference in average condition RT across participants as a function of the average difference in RT between conditions.
Fig. 7: Conjunction of the average, within-condition RT effect across the seven tasks.
Fig. 8: Examples of the underlying relationship between a behavioural construct and RTs.

Similar content being viewed by others

Data availability

The conjunction map (Fig. 7) and separate RT-based activation maps for each task used to generate the conjunction map are available in the form of 1 − P maps at https://neurovault.org/collections/13656/. The full, raw fMRI datasets are available on OpenNeuro at https://openneuro.org/datasets/ds004636/.

Code availability

The code for all analyses is shared at https://doi.org/10.5281/zenodo.8083510 (simulations) and https://doi.org/10.5281/zenodo.8083518 (real data analysis).

References

  1. Donders, F. C. Over de snelheid van psychische processen [on the speed of psychological processes]. Acta Psychol. 30, 412–431 (1969).

    Article  CAS  Google Scholar 

  2. Stroop, J. R. Studies of interference in serial verbal reactions. J. Exp. Psychol. 18, 643–662 (1935).

    Article  Google Scholar 

  3. Botvinick, M. M. et al. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001).

    Article  CAS  PubMed  Google Scholar 

  4. Botvinick, M. M. et al. Conflict monitoring versus selection-for-action in anterior cingulate cortex. Nature 402, 179–181 (1999).

    Article  CAS  PubMed  ADS  Google Scholar 

  5. MacDonald, A. W. et al. Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science 288, 1835–1838 (2000).

    Article  CAS  PubMed  ADS  Google Scholar 

  6. Kerns, J. G. et al. Anterior cingulate conflict monitoring and adjustments in control. Science 303, 1023–1026 (2004).

    Article  CAS  PubMed  ADS  Google Scholar 

  7. Logothetis, N. K. What we can do and what we cannot do with fMRI. Nature 453, 869–878 (2008).

    Article  CAS  PubMed  ADS  Google Scholar 

  8. Savoy, R. et al. Pushing the temporal resolution of fMRI: studies of very brief visual stimuli, onset variability and asynchrony, and stimulus-correlated changes in noise. In Proc. of the ISMRM 3rd annual meeting, Nice, France, p. 450 (1995).

  9. Jezzard, P., Matthews, P. & Smith, S. Functional MRI, an Introduction to Methods (Oxford Univ. Press, 2001).

  10. Grinband, J., Wager, T. D., Lindquist, M., Ferrera, V. P. & Hirsch, J. Detection of time-varying signals in event-related fMRI designs. NeuroImage 43, 509–520 (2008).

    Article  PubMed  Google Scholar 

  11. Ratcliff, R. A theory of memory retrieval. Psychol. Rev. 85, 59–108 (1978).

    Article  Google Scholar 

  12. Yarkoni, T. et al. BOLD correlates of trial-by-trial reaction time variability in gray and white matter: a multi-study fMRI analysis. PLoS ONE 4, e4257 (2009).

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  13. Grinband, J. et al. The dorsal medial frontal cortex is sensitive to time on task, not response conflict or error likelihood. NeuroImage 57, 303–311 (2011).

    Article  PubMed  Google Scholar 

  14. Carp, J. et al. Conditional differences in mean reaction time explain effects of response congruency, but not accuracy, on posterior medial frontal cortex activity. Front. Hum. Neurosci. https://doi.org/10.3389/fnhum.2010.00231 (2010).

  15. Yeung, N., Cohen, J. D. & Botvinick, M. M. Errors of interpretation and modeling: a reply to Grinband et al. NeuroImage 57, 316–319 (2011).

    Article  PubMed  Google Scholar 

  16. Brown, J. W. Medial prefrontal cortex activity correlates with time-on-task: what does this tell us about theories of cognitive control? NeuroImage 57, 314–315 (2011).

    Article  PubMed  Google Scholar 

  17. Grinband, J. et al. Conflict, error likelihood, and RT: response to Brown & Yeung et al. NeuroImage 57, 320–322 (2011).

    Article  PubMed  Google Scholar 

  18. Nachev, P. The blind executive. NeuroImage 57, 312–313 (2011).

    Article  PubMed  Google Scholar 

  19. Carp, J., Fitzgerald, K. D., Taylor, S. F. & Weissman, D. H. Removing the effect of response time on brain activity reveals developmental differences in conflict processing in the posterior medial prefrontal cortex. NeuroImage 59, 853–860 (2012).

    Article  PubMed  Google Scholar 

  20. Poldrack, R. A., Mumford, J. A. & Nichols, T. E. Handbook of Functional MRI Data Analysis (Cambridge Univ. Press, 2009).

  21. Ratcliff, R. & Murdock, B. B. Retrieval processes in recognition memory. Psychol. Rev. 83, 190–214 (1976).

    Article  Google Scholar 

  22. Weissman, D. H. & Carp, J. The congruency effect in the posterior medial frontal cortex is more consistent with time on task than with response conflict. PLoS ONE 8, 62405 (2013).

    Article  ADS  Google Scholar 

  23. Dubois, J. & Adolphs, R. Building a science of individual differences from fMRI. Trends Cogn. Sci. 20, 425–443 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  24. Marek, S. et al. Reproducible brain-wide association studies require thousands of individuals. Nature 603, 654–660 (2022).

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  25. Mumford, J. A., Poline, J.-B. & Poldrack, R. A. Orthogonalization of regressors in fMRI models. PLoS ONE 10, 0126255 (2015).

    Article  Google Scholar 

  26. Mumford, J. A. & Nichols, T. Modeling and inference of multisubject fMRI data. IEEE Eng. Med. Biol. Mag. 25, 42–51 (2006).

    Article  PubMed  Google Scholar 

  27. Fan, J. et al. Testing the efficiency and independence of attentional networks. J. Cogn. Neurosci. 14, 340–347 (2002).

    Article  PubMed  Google Scholar 

  28. MacDonald, A. et al. A convergent–divergent approach to context processing, general intellectual functioning, and the genetic liability to schizophrenia. Neuropsychology 19, 814–821 (2005).

    Article  PubMed  Google Scholar 

  29. Kirby, K. N. One-year temporal stability of delay-discount rates. Psychon. Bull. Rev. 16, 457–462 (2009).

    Article  PubMed  Google Scholar 

  30. Logan, G. & Bundesen, C. Clever homunculus: is there an endogenous act of control in the explicit task-cuing procedure? J. Exp. Psychol. Hum. Percept. Perform. 29, 575–599 (2003).

    Article  PubMed  Google Scholar 

  31. Logan, G. D. & Cowan, W. B. On the ability to inhibit simple and choice reaction time responses: a model and a method. J. Exp. Psychol. 10, 276–291 (1984).

    CAS  Google Scholar 

  32. DeJong, R., Coles, M. & Logan, G. Strategies and mechanisms in nonselective and selective inhibitory motor control. J. Exp. Psychol. Hum. Percept. Perform. 21, 498–511 (1995).

    Article  CAS  Google Scholar 

  33. Sochat, V. et al. The experiment factory: standardizing behavioral experiments. Front. Psychol. 7, 610 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  34. Bissett, P. G. et al. Cognitive tasks, anatomical MRI, and functional MRI data evaluating the construct of self regulation. Preprint at bioRxiv https://doi.org/10.1101/2023.09.27.559869 (2023).

  35. Esteban, O. et al. fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat. Methods 16, 111–116 (2019).

    Article  CAS  PubMed  Google Scholar 

  36. Greve, D. N. & Fischl, B. Accurate and robust brain image alignment using boundary-based registration. NeuroImage 48, 63–72 (2009).

    Article  PubMed  Google Scholar 

  37. Jenkinson, M., Bannister, P., Brady, M. & Smith, S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. NeuroImage 17, 825–841 (2002).

    Article  PubMed  Google Scholar 

  38. Cox, R. W. & S, H. J. Software tools for analysis and visualization of fMRI data. NMR Biomed. 10, 171–178 (1997).

    Article  CAS  PubMed  Google Scholar 

  39. Pruim, R. H. R. et al. ICA-AROMA: a robust ICA-based strategy for removing motion artifacts from fMRI data. NeuroImage 112, 267–277 (2015).

    Article  PubMed  Google Scholar 

  40. Abraham, A. et al. Machine learning for neuroimaging with scikit-learn. Front. Neuroinform. 8, 14 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  41. Smith, S. & Nichols, T. Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference. NeuroImage 44, 83–98 (2009).

    Article  PubMed  Google Scholar 

  42. Nichols, T. et al. Valid conjunction inference with the minimum statistic. NeuroImage 25, 653–660 (2005).

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Institutes of Health Science of Behavior Change Common Fund Program through an award administered by the National Institute for Drug Abuse (UH2DA041713; principal investigators: L. A. Marsch and R.A.P.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We also thank D. Weissman for discussions that helped in the writing of this paper.

Author information

Authors and Affiliations

  1. Department of Psychology, Stanford University, Stanford, CA, USA

    Jeanette A. Mumford, Patrick G. Bissett, Sunjae Shim, Jaime Ali H. Rios & Russell A. Poldrack

  2. Department of Psychology, University of Chicago, Chicago, IL, USA

    Henry M. Jones

Authors
  1. Jeanette A. Mumford
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrick G. Bissett
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Henry M. Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sunjae Shim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jaime Ali H. Rios
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Russell A. Poldrack
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.A.M. developed the project, wrote all the simulation code and much of the real data analysis code, helped with quality assessments of the real data and did most of the writing. P.G.B. helped with model construction, interpretation, data collection and writing of the paper. H.M.J., S.S. and J.A.H.R. helped in the collection, preprocessing, quality assessment and analysis of the real fMRI data. R.A.P. helped with model development, interpretation and writing of the paper.

Corresponding author

Correspondence to Jeanette A. Mumford.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks Markus Ullsperger and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Type I error as RT difference between conditions increases (ISI=3-6s).

This figure illustrates that results are similar to when the ISI ranged between 2-4s (result in main manuscript, Fig. 3). The two-alternative, forced-choice categorization task (Categorization) RT distribution was used in the top panels, while Stroop RT distribution was used in the bottom panels, both with an ISI between 3-6s and the inference of interest was for the 1-sample t-test of the condition difference effect with 100 subjects. 2500 simulations were used to calculate the error rate.

Extended Data Fig. 2 Power when the RT difference is 0.8s as the condition difference increases.

Only models that had controlled error rates in Fig. 3 (main text) are shown and since the RT range of this simulation is not within the range where ConsDurRTMod has controlled error rates, it is excluded. The two-alternative, forced-choice categorization task RT distribution was used in the top panels, while Stroop RT distribution was used in the bottom panels. An ISI between 2-4s was used and the inference of interest was for the 1-sample t-test of the condition difference effect with 100 subjects. Model 2, RTDur, is the true model for the left column (orange line) and model 1, ConsDurNoRT, is the true model for the right column (blue line), and the corresponding power curves indicate maximal power. When the signal duration scales with RT (left panel), ConsDurRTDur (red) has some power loss due to model misfit, while the 4 regressor interaction model (gray line) loses considerable power due to collinearity. When the duration does not scale with RT (right panels), the ConsDurRTDur model (red line) has similar power to the true model (blue), illustrating some power loss due to including an RT regressor in the time-series analysis. This power loss is not seen for shorter RT differences (.1s result in main paper Fig. 4).

Extended Data Fig. 3 Understanding ConsDurRTDur parameter estimates when RTDur model is true.

When the data follow the RTDur model, this implies an RT by condition interaction where the condition effects are 0 when the RT is 0. If using the ConsDurRTDur model to fit the data, the difference between the fitted constant duration regressors is approximately the RTDur difference, albeit with a loss in power, which is illustrated here. The figure illustrates the ConsDurRTDur model fit, β0 + β0ConsDur1 + β2ConsDur2 + β3 RTDur, to data based on RTDur model, BOLD = 1 + 1*RTDur1 + 4*RTDur2, where the RTs are evenly spaced between .5 and 1.2 for condition 1 and 2.5 and 3.5 for condition 2. The first 5 trials are condition 1 and second 5 trials are condition 2. The first two panels break down components of the ConsDurRTDur model fit to RTDur data, where the first panel is the intercept plus the RTDur effect (\({\hat{\beta }}_{0}+{\hat{\beta }}_{3}{{{\rm{RTDur}}}}\), green), which overestimates the BOLD for the first condition and underestimates it for the second condition. The middle panel shows how the condition 1 effect from ConsDurRTDur accounts for the overestimation in the first panel and the condition 2 effect accounts for the underestimation. Summing the green, orange and blue lines of the first two panels yields the black line in the third panel, showing similar model fits, which implies the condition difference effect from ConsDurRTDur (peak differences between yellow and blue lines of the middle panel) is a close approximation to the true effect from RTDur. This figure also illustrates that the condition versus baseline effects, alone, from ConsDurRTDur have limited interpretation as they reflect the projected effect when the RT is 0 and can be negative, so they have limited use.

Extended Data Fig. 4 RT-based activation, RT-adjusted condition comparisons and their overlap.

Comparison of RT-based activation network (yellow), specific condition comparison contrast activation (blue) and their overlap (green). The condition comparison contrast for a given task is described in the panel title. In all cases the ConsDurRTDur model was used and the networks are based off of a randomise TFCE family-wise error corrected p < 0.05 for a 2-sided one-sample t-test at the group level.

Extended Data Table 1 Contrast interpretation examples using the Stroop task
Full size table
Extended Data Table 2 Impact of orthogonalization on contrast interpretation
Full size table

Supplementary information

Supplementary Information

Supplementary Tables 1–10, task descriptions and data descriptions.

Reporting Summary

Peer Review File

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mumford, J.A., Bissett, P.G., Jones, H.M. et al. The response time paradox in functional magnetic resonance imaging analyses. Nat Hum Behav 8, 349–360 (2024). https://doi.org/10.1038/s41562-023-01760-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-023-01760-0

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing