Evolution of risk preference is determined by reproduction dynamics, life history, and population size

Alternative behavioral strategies typically differ in their associated risks, meaning that a different variance in fitness-related outcomes characterizes each behavior. Understanding how selection acts on risk preference is crucial to interpreting and predicting behavior. Despite much research, most theoretical frameworks have been laid out as optimization problems from the individual’s perspective, and the influence of population dynamics has been underappreciated. We use agent-based simulations that implement competition between two simple behavioral strategies to illuminate effects of population dynamics on risk-taking. We explore the effects of inter-generational reproduction dynamics, population size, the number of decisions throughout an individual’s life, and simple alternate distributions of risk. We find that these factors, very often ignored in empirical and theoretical studies of behavior, can have significant and non-intuitive impacts on the selection of alternative behavioral strategies. Our results demonstrate that simple rules regarding predicted risk preference do not hold across the complete range of each of the factors we studied; we propose intuitive interpretations for the dynamics within each regime. We suggest that studies of behavioral strategies should explicitly take into account the species’ life history and the ecological context in which selection acted on the risk-related behavior of the organism of interest.


Supplementary section 1: On the pervasiveness, time--scale, and risk--27 magnitude of risk--related behaviors 28 29
It is frequently underappreciated that an element of risk, a difference in the 30 expected variance in payoffs that is associated with alternative behavioral choices, is year--round water bodies tadpoles are exposed to high levels of predation 41 by fish, leading to low but predictable amphibian reproductive success. individual's lifetime. In all schemes, the mean risky payoff is equal to that of the safe 189 strategy, but the variance in these payoffs is different for each condition. The safe 190 strategy is strongly advantageous when the risky payoff has a high variance, and this 191 advantage decreases as the variance in risky payoffs decreases. 192 193 194

Supplementary section 3: A note on truncation selection 195
We do not study a simple but important case of truncation selection, because in 196 our framework it leads to a trivial result: the case of truncation at an absolute 197 payoff threshold. Under this scheme, only individuals that accumulated a payoff 198 greater than the threshold reproduce. As opposed to the scenario we studied, in 199 which the threshold value is determined by the distribution of accumulated payoffs 200 in the population, in this case the threshold is constant. This is likely to be a 201 reasonable approximation for the reproductive dynamics in many species, 202 particularly when intra--species competition is low compared to other limiting 203 factors such as inter--species competition, severe climate, or predation. One of the 204 most common thresholds of this sort is body size (e.g. 10-13 ). 205 In our framework the implementation of these reproductive dynamics leads to a 206 trivial result: if the payoff of the risk--averse players is above the set threshold, risk 207 aversion always spreads and fixes in the population, because all individuals that 208 utilize this strategy are selected for reproduction, whereas if the risk--averse players' 209 payoff is below the threshold, this strategy will be lost from the population within a 210 single simulation time step. In the case in which the risk averse payoff is equal to the 211 threshold, the outcome is dependent on a combination of the way the risk averse 212 players are accordingly dealt with (e.g., whether none, all, or half of them are 213 selected for reproduction) and the distribution of the payoffs of the risk--prone 214 strategy. These cases also yield trivial results, and have very limited generality. 215 These findings, although trivial, lead to a simple but powerful prediction: strong 216 risk preference is expected to evolve when the reproductive threshold is set to a 217 constant value; whether risk aversion or risk proneness would be preferred 218 depends on the value of the payoff distributions. A major caveat of this finding is 219 that this prediction is not readily translatable to predictions about the behavior of 220 most of the species in which a certain body--size is the truncation threshold: body 221 size is typically the product of a very large number of factors, particularly the 222 payoffs of repeated foraging bouts that are on the order of dozens, hundreds, or 223 thousands of foraging events. These conditions are expected to minimize the 224 difference between risk prone and risk averse foraging strategies, as discussed in the main text, perhaps to an extent that the difference between them is negligible 226 with respect to the reproductive threshold, suggesting that other factors that are not 227 considered in our framework would determine more prominently the behavioral 228 strategy that would be selected. 229 This scenario also highlights the potential importance of individual state--230 dependent risk preference, which is not considered in our framework. See

Supplementary section 5: the effect of population finiteness 286 287
In each of the scenarios that we studied, some of our findings diverge from the 288 prevalent portrayal of risk taking. Some of these are simply a result of our 289 exploration of reproductive dynamics that are different from those usually 290 considered. Beyond these, a primary factor that leads to divergence from the 291 common dogma is the explicit consideration of a population context. Here we briefly 292 highlight a few of these cases, and compare our results with the prediction of risk 293 preference in the absence of a population context. 294 1.
Proportional selection: we find that a risk averse strategy is preferred, but 295 even when the difference in variance between the two strategies is large, 296 the risk--seeking strategy fixes in a very large minority of cases. This 297 finding is in line with existing theory 8 , but is largely under--appreciated, as 298 the discussion frequently focuses on which strategy is preferred, and not 299 on its probability of fixation. In an infinite population a preference for one 300 strategy would not arise at all, as long as the variance is not correlated 301 among individuals 14 . 302

2.
Truncation selection: we find that strategy preference is sensitive both to 303 the truncation threshold and to the payoff distribution, including to 304 aspects of it beyond the variance per se. These findings are not 305 unexpected, but are largely under--appreciated, as is evidenced by the 306 common use of truncation selection in computational evolutionary 307 simulations without explicit consideration of the expected effects 308 highlighted by our findings. The truncation scheme that we apply is 309 meaningful only in a population context, as it is dependent on the rank 310 order of the individuals' payoffs. In the absence of a population context, a 311 truncation scheme with a constant threshold may be applied and leads to 312 a potentially different, trivial, outcome, as discussed in supplementary 3. 313

3.
Power--weighted selection (concave/convex fitness functions): we find 314 strong dependence of the outcome on population size (i.e. the 315 disadvantageous strategy may fix in many simulations, dependent on population size), and even a reversal of the expected preference between 317 the two strategies in scenarios in which the probability of receiving the 318 high or the lower payoff is very low and the population size is small. In the 319 absence of a population context, neither of these phenomena occur. 320

4.
Sigmoid--weighted selection: the shape of the sigmoid in our model 321 depends, as in classic evolutionary studies, on the fitness of the best--322 performing individual in the population (which is defined to have fitness 323 equal to 1); thus, it is inherently dependent on the population context. We 324 find a qualitative recapitulation of results found in the truncation selection 325 scenario, and most importantly we find that no full generalization can be 326 made regarding whether this scenario would favor risk aversion or risk 327 seeking: for any given sigmoid fitness function, payoff distributions can be 328 found that would favor risk--taking or risk aversion. The study of bet hedging deals with a similar question -how organisms cope 351 with variance in fitness--related outcomes -from a perspective that focuses on life 352 history and particularly on strategies of reproduction ( 14,16,28 ). It draws mostly on 353 theory in evolutionary biology, with less obvious links to other disciplines. 354 Accordingly, it is more common in bet hedging than it is in risk--sensitivity that 355 behavior is studied in a population context, with explicit consideration of the fitness 356 associated with alternative strategies and with consideration of evolutionary 357 timescales on the order of generations. 358 Perhaps the most common depiction of the question in risk--sensitivity 359 frameworks is that of an animal that forages on a limited energy budget and needs 360 to maximize its gains, typically towards a short--term goal such as crossing an 361 accumulated energy threshold in order to survive the night (but not always). Accordingly, the fitness function assumed is usually either a constant step function 363 (i.e., the threshold for survival is a pre--determined constant) or a concave function 364 with diminishing returns (see 15 , table 1, for a partial summary; some such examples  365 are in 17,29 ). Frequently, the key to the expected preference stems from Jensen's 366 inequality, supporting risk aversion, and risk preference depends on whether the 367 mean expected payoff of the risk--averse strategy is sufficient to cross the survival 368 threshold (and then it is preferred) or not ( 18 ). Models that relate risk--sensitivity in 369 the context of foraging to long--term fitness, in the context of reproduction, high 370 rates of background predation, or migration, for example, are few, but some such 371 attempts have been made, e.g. 15,30-32 . 372 Bet hedging can be explained by considering two possible states of the 373 environment and three phenotypes that fare in each of the environments differently. 374 For simplicity, we may refer to the two possible environmental states as dry and wet 375 (following Starrfelt and Kokko, S&K, 14 ), with inter--generational variability, and the 376 phenotypes as a wet--specialist, a dry--specialist, and a generalist (see also 33 ). An 377 evolutionary strategy that does not include bet--hedging would be, in these terms, 378 one in which a lineage specializes to one of the two environments, producing always 379 the dry--specialized or always the wet--specialized phenotype. Bet hedging can be of 380 one of two types: a conservative bet--hedging strategy, in which all individuals have 381 the generalist phenotype, and do reasonably well, but not very well, in both 382 environments, and a diversifying bet--hedging strategy, in which each individual 383 always has one of the specialized phenotypes, but whose genotype carries the 384 potential for both, and thus each individual's offspring are distributed in their 385 phenotype between the two specialized phenotypes, each with some pre--386 determined probability (that may be adapted to the probability of each 387 environmental condition's occurrence). This is the type of bet--hedging most 388 frequently studied (see, e.g., 34-36 ). 389 A fundamental principle that underlies many of the bet hedging results is that 390 when there is between--generational variation in the number of offspring, the 391 measure to be optimized is the geometric mean fitness, and not the arithmetic mean 392 minimization would be selected for, and so the arithmetic mean payoff is traded off