Psychol. Rev. http://doi.org/b8vr (2017)
Recent years have seen a lot of emphasis on improving the robustness and reproducibility of behavioural research. However, robust methods are clearly not enough for robust science: theoretical reasoning must be sound, too.
Regenwetter and Robinson identify a key problem in behavioural decision research: faulty heuristic reasoning from theoretical constructs (for example, preferences) to behaviour (for example, actual choices). Heterogeneity — both within and across individuals — is the norm. Yet researchers frequently ignore heterogeneity and, when linking hypothetical constructs and overt behaviour, use heuristic reasoning that exhibits one or both of the following fallacies: the fallacy of sweeping generalization, whereby average data across individuals are mistaken to say something about all, some, or even any individuals; or the fallacy of composition, whereby several distinct experiments carried out with different individuals are misinterpreted as converging evidence in support of a theory of individual behaviour. The remedy for logically inconsistent predictions and inferences, according to Regenwetter and Robinson, is to spell out the nature of heterogeneity within and across individuals and to formally model it when deriving theoretical predictions. The authors outline one potential approach to constructing theories that keep track of heterogeneity, and illustrate their approach by re-analysing ten existing datasets and discussing what inferences can and cannot be drawn from these data.
Regenwetter and Robinson provide a valuable guide to reasoning fallacies in behavioural decision research and how to avoid them. Ultimately, as the authors show, no amount of methodological rigour can compensate for poor reasoning.