Abstract
The flexibility to learn diverse tasks is a hallmark of human cognition. To improve our understanding of individual differences and dynamics of learning across tasks, we analyse the latent structure of learning trajectories from 36,297 individuals as they learned 51 different tasks on the Lumosity online cognitive training platform. Through a data-driven modelling approach using probabilistic dimensionality reduction, we investigate covariation across learning trajectories with few assumptions about learning curve form or relationships between tasks. Modelling results show substantial covariation across tasks, such that an entirely unobserved learning trajectory can be predicted by observing trajectories on other tasks. The latent learning factors from the model include a general ability factor that is expressed mostly at later stages of practice and additional task-specific factors that carry information capable of accounting for manually defined task features and task domains such as attention, spatial processing, language and math.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
The original and preprocessed versions of the data can be accessed at https://osf.io/g9zkf. Source data are provided with this paper.
Code availability
The code used to analyse the data, run the Bayesian PCA model and create figures and tables can be accessed at https://osf.io/g9zkf.
References
Spearman, C. ‘General intelligence’ objectively determined and measured. Am. J. Psychol. 15, 201–293 (1904).
Carroll, J. B. et al. Human Cognitive Abilities: A Survey of Factor-Analytic Studies (Cambridge Univ. Press, 1993).
Conway, A. R., Cowan, N., Bunting, M. F., Therriault, D. J. & Minkoff, S. R. A latent variable analysis of working memory capacity, short-term memory capacity, processing speed, and general fluid intelligence. Intelligence 30, 163–183 (2002).
Kovacs, K. & Conway, A. R. Process overlap theory: a unified account of the general factor of intelligence. Psychol. Inq. 27, 151–177 (2016).
Jensen, A. R. Clocking the Mind: Mental Chronometry and Individual Differences (Elsevier, 2006).
Schubert, A.-L., Hagemann, D. & Frischkorn, G. T. Is general intelligence little more than the speed of higher-order processing?. J. Exp. Psychol. 146, 1498–1512 (2017).
Kaufman, S. B., DeYoung, C. G., Gray, J. R., Brown, J. & Mackintosh, N. Associative learning predicts intelligence above and beyond working memory and processing speed. Intelligence 37, 374–382 (2009).
Van Der Maas, H. L. et al. A dynamical model of general intelligence: the positive manifold of intelligence by mutualism. Psychol. Rev. 113, 842–861 (2006).
Barbey, A. K. Network neuroscience theory of human intelligence. Trends Cogn. Sci. 22, 8–20 (2018).
Steyvers, M., Hawkins, G. E., Karayanidis, F. & Brown, S. D. A large-scale analysis of task switching practice effects across the lifespan. Proc. Natl Acad. Sci. USA 116, 17735–17740 (2019).
Steyvers, M. & Benjamin, A. S. The joint contribution of participation and performance to learning functions: exploring the effects of age in large-scale data sets. Behav. Res. Methods 51, 1531–1543 (2019).
Donner, Y. & Hardy, J. L. Piecewise power laws in individual learning curves. Psychon. Bull. Rev. 22, 1308–1319 (2015).
Ilin, A. & Raiko, T. Practical approaches to principal component analysis in the presence of missing values. J. Machine Learning Res. 11, 1957–2000 (2010).
Tipping, M. E. & Bishop, C. M. Probabilistic principal component analysis. J. R. Stat. Soc. B 61, 611–622 (1999).
Lim, Y.J. & Teh, Y.W. Variational Bayesian approach to movie rating prediction. In Proc. International Conference on Knowledge Discovery and Data Mining 15–21 (ACM, 2007).
Bell, R. M. & Koren, Y. Scalable collaborative filtering with jointly derived neighborhood interpolation weights. In Proc. Seventh IEEE International Conference on Data Mining 43–52 (IEEE Computer Society, 2007).
Driver, C. C. & Voelkle, M. C. Hierarchical Bayesian continuous time dynamic modeling. Psychol. Methods 23, 774–799 (2018).
Kievit, R. A. et al. Developmental cognitive neuroscience using latent change score models: a tutorial and applications. Dev. Cogn. Neurosci. 33, 99–117 (2018).
Isiordia, M. & Ferrer, E. Curve of factors model: a latent growth modeling approach for educational research. Educ. Psychol. Meas. 78, 203–231 (2018).
Ram, N. & Grimm, K. J. in Handbook of Child Psychology and Developmental Science (ed. Lerner, R. M.) 1–31 (Wiley, 2015).
McArdle, J. J., Ferrer-Caja, E., Hamagami, F. & Woodcock, R. W. Comparative longitudinal structural analyses of the growth and decline of multiple intellectual abilities over the life span. Dev. Psychol. 38, 115–142 (2002).
Preacher, K. J., Wichman, A. L., MacCallum, R. C. & Briggs, N. E. Latent Growth Curve Modeling (Sage, 2008).
McNeish, D., Dumas, D. G. & Grimm, K. J. Estimating new quantities from longitudinal test scores to improve forecasts of future performance. Multivariate Behav. Res. https://doi.org/10.1080/00273171.2019.1691484 (2019).
Rosenberg, M. D., Casey, B. & Holmes, A. J. Prediction complements explanation in understanding the developing brain. Nat. Commun. 9, 589 (2018).
Settles B., Brust, C., Gustafson, E., Hagiwara, M. & Madnani, N. Second language acquisition modeling. In Proc. Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications (eds Tetreault, J., Burstein, J., Kochmar, E., Leacock, C. & Yannakoudakis, H.) 56–65 (ACL, 2018).
Luttinen, J. & Ilin, A. Transformations in variational Bayesian factor analysis to speed up learning. Neurocomputing 73, 1093–1102 (2010).
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C. & Strahan, E. J. Evaluating the use of exploratory factor analysis in psychological research. Psychol. Methods 4, 272–299 (1999).
Abdi, H. in Encyclopedia for Research Methods for the Social Sciences (ed. Lewis-Beck, M. S. et al.) 792–795 (Sage, 2004).
Widaman, K. F., Ferrer, E. & Conger, R. D. Factorial invariance within longitudinal structural equation models: measuring the same construct across time. Child Dev. Perspect. 4, 10–18 (2010).
Salthouse, T. A. The processing-speed theory of adult age differences in cognition. Psychol. Rev. 103, 403–428 (1996).
Jensen, A. R. Regularities in Spearman’s law of diminishing returns. Intelligence 31, 95–105 (2003).
Griffiths, T. L. Manifesto for a new cognitive revolution. Cognition 135, 21–23 (2015).
Goldstone, R. L. & Lupyan, G. Discovering psychological principles by mining naturally occurring data sets. Topics Cogn. Sci. 8, 548–568 (2016).
Molenaar, D., Dolan, C. V., Wicherts, J. M. & van der Maas, H. L. Modeling differentiation of cognitive abilities within the higher-order factor model using moderated factor analysis. Intelligence 38, 611–624 (2010).
Tucker-Drob, E. M. Differentiation of cognitive abilities across the life span. Dev. Psychol. 45, 1097–1118 (2009).
Kievit, R. A. et al. Mutualistic coupling between vocabulary and reasoning supports cognitive development during late adolescence and early adulthood. Psychol. Sci. 28, 1419–1431 (2017).
Kievit, R. A., Hofman, A. D. & Nation, K. Mutualistic coupling between vocabulary and reasoning in young children: a replication and extension of the study by Kievit et al.(2017). Psychol. Sci. 30, 1245–1252 (2019).
Evans, N. J., Brown, S. D., Mewhort, D. J. & Heathcote, A. Refining the law of practice. Psychol. Rev. 125, 592–605 (2018).
Frischkorn, G. & Schubert, A.-L. Cognitive models in intelligence research: advantages and recommendations for their application. J. Intell. 6, 34 (2018).
Melby-LervÅg, M., Redick, T. S. & Hulme, C. Working memory training does not improve performance on measures of intelligence or other measures of ‘far transfer’ evidence from a meta-analytic review. Perspect. Psychol. Sci. 11, 512–534 (2016).
Simons, D. J. et al. Do ‘brain-training’ programs work? Psychol. Sci. Public Interest 17, 103–186 (2016).
Acknowledgements
Feedback on the task feature categorization framework was provided by A. Kaluszka, O. Claflin, A. Osman and N. Ng. The authors received no specific funding for this work.
Author information
Authors and Affiliations
Contributions
M.S. and R.J.S. planned the research. R.J.S. provided the data. M.S. analysed the data. M.S. and R.J.S. interpreted the results and wrote the paper.
Corresponding authors
Ethics declarations
Competing interests
R.J.S. is an employee of Lumos Labs and owns stock in the company. M.S. has no competing interests.
Additional information
Peer review information Primary Handling Editor: Marike Schiffer.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Task features for cognitive tasks grouped by process, stimulus, input method, and game design.
Solid circles indicate the presence of the task feature. The primary cognitive area associated with each task is given between parentheses.
Extended Data Fig. 2 Glossary of task features.
Definitions are provided for each process, stimulus, input method and game design feature used to describe the cognitive tasks.
Extended Data Fig. 3 Inferred latent learning factors with five factors.
Top panel: the heatmap visualizes the latent learning factors (columns) across games (rows). Positive (negative) values are visualized by brown (blue) colours. Each latent learning factor corresponds to a group of 8 columns, where the columns within a group correspond to different stages of practice (practice increases from left to right). Cognitive tasks are coloured according to primary task domain. Bottom panel: correlations between latent learning factors and manually derived task features. Positive (negative) correlations are illustrated by red (blue) colours. The hierarchical tree visualizes the similarity between the task features on the basis of the pattern of correlations between task feature and latent factors. Task features are grouped by process, stimulus, and game design features.
Extended Data Fig. 4 Inferred latent learning factors with seven factors.
Top panel: the heatmap visualizes the latent learning factors (columns) across games (rows). Positive (negative) values are visualized by brown (blue) colours. Each latent learning factor corresponds to a group of 8 columns, where the columns within a group correspond to different stages of practice (practice increases from left to right). Cognitive tasks are coloured according to primary task domain. Bottom panel: correlations between latent learning factors and manually derived task features. Positive (negative) correlations are illustrated by red (blue) colours. The hierarchical tree visualizes the similarity between the task features on the basis of the pattern of correlations between task feature and latent factors. Task features are grouped by process, stimulus, and game design features.
Extended Data Fig. 5 Inferred latent learning factors after oblique rotation (promax) that results in non-orthogonal factors.
Top panel: the heatmap visualizes the latent learning factors (columns) across games (rows). Positive (negative) values are visualized by brown (blue) colours. Each latent learning factor corresponds to a group of 8 columns, where the columns within a group correspond to different stages of practice (practice increases from left to right). Cognitive tasks are coloured according to primary task domain. The matrix shows the correlations between factors. Bottom panel: correlations between latent learning factors and manually derived task features. Positive (negative) correlations are illustrated by red (blue) colours. The hierarchical tree visualizes the similarity between the task features on the basis of the pattern of correlations between task feature and latent factors. Task features are grouped by process and stimulus features.
Extended Data Fig. 6 Changes in predicted learning curves based on changes in individual scores for individual factors.
For each factor (row), the blue, black, and red lines show the predicted learning curves for 10, 50, and 90 percentile scores on that particular factor and median scores on all other factors. Therefore, the red (blue) lines show the predicted learning curves for individuals who score high (low) on that factor while holding constant the contribution of other factors. Intercept differences are highlighted with circles at the start of the learning curve. To facilitate comparison, the vertical axis is the same for each particular task (column) but is different across tasks. Tasks are ordered by primary domain (colour).
Supplementary information
Supplementary Information
Supplementary Methods, Supplementary Results and Supplementary Figs. 1–4.
Source data
Source Data Fig. 2
Data means.
Source Data Fig. 3
Matrix values.
Source Data Fig. 4
Data means.
Source Data Fig. 5
Matrix values.
Source Data Extended Data Fig. 1
Matrix values.
Source Data Extended Data Fig. 3
Matrix values.
Source Data Extended Data Fig. 4
Matrix values.
Source Data Extended Data Fig. 5
Matrix values.
Rights and permissions
About this article
Cite this article
Steyvers, M., Schafer, R.J. Inferring latent learning factors in large-scale cognitive training data. Nat Hum Behav 4, 1145–1155 (2020). https://doi.org/10.1038/s41562-020-00935-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41562-020-00935-3