Macaque dorsal premotor cortex exhibits decision-related activity only when specific stimulus–response associations are known

How deliberation on sensory cues and action selection interact in decision-related brain areas is still not well understood. Here, monkeys reached to one of two targets, whose colors alternated randomly between trials, by discriminating the dominant color of a checkerboard cue composed of different numbers of squares of the two target colors in different trials. In a Targets First task the colored targets appeared first, followed by the checkerboard; in a Checkerboard First task, this order was reversed. After both cues appeared in both tasks, responses of dorsal premotor cortex (PMd) units covaried with action choices, strength of evidence for action choices, and RTs— hallmarks of decision-related activity. However, very few units were modulated by checkerboard color composition or the color of the chosen target, even during the checkerboard deliberation epoch of the Checkerboard First task. These findings implicate PMd in the action-selection but not the perceptual components of the decision-making process in these tasks.


29
TF and CF tasks: same data as in Figure 1B,C. A) Like monkey T, monkey Z showed a 30 decrease in success rates for the checkerboards with the highest color coherences in 31 the CFD task compared to the TF and CF tasks. B) Unlike monkey T, however, monkey 32 Z continued to show a substantial reduction in RTs for checkerboards with lower color 33 coherences in the CFD task (green line) compared to the TF task (blue line). onset (see Methods) at 220ms after the Targets cue appeared in the TF task and 40 180ms after the Checkerboard appeared in the CF task. Unit EPC032 (middle) did not 41 show a significant rapid response to the Targets in the TF task but showed a late rapid 42 decrease in activity (>600ms) to the Checkerboard in the CF task. Unit EPC132 43 (bottom) showed a significant rapid increase in response to the Targets at 280ms after 44 they appeared in the TF task, and a significant rapid response increase 220ms after the 45 Checkerboard appeared in the CF task. The response to the Checkerboard was 46 markedly stronger and more rapid for the 100% coherence than for the 20% and 4% 47 coherences. EPC132 showed a significant main effect of evidence Strength in the 48 ANOVA, but did not show a significant linear regression to either the signed color 49 coherence or the signed evidence for reach direction during the Checkerboard-50 observation period of the CF task.   PMd units in monkey Z but not monkey T exhibited responses to the first sensory 123 information provided in each task 124 125 The response of many PMd units in monkey Z to the appearance of the Target cues in 126 the TF task is consistent with a similar activation of PMd units in a 2-Target memorized-127 delay task. That activity after the presentation of the two colored target cues has been 128 interpreted as a simultaneous co-activation of two PMd populations that prefer reaches 129 to the two potential targets before a second monochromatic instructional cue identifies 130 the correct target 1,2 . This activity is sustained during the post-Targets memory-delay 131 period of the 2-Target task, and is a putative neural correlate of a memorized 132 representation of the Target cue information in each trial, expressed in the space of 133 potential actions in PMd (c.f., Horwitz et al 3 ).

135
The responses of the units to the appearance of the Checkerboard cues in the CF task 136 likely do not implicate PMd in the perceptual interpretation of the color evidence since 137 the responses were largely insensitive to the salient dimensions (dominant color, signed 138 color coherence) of the stimuli ( Figure 5, 6). Alternatively, the activation may reflect a 139 similar process of representation of potential actions as in the TF task. Once a unit in 140 monkey Z was chosen for recording in a given recording session, it was tested with the 141 same two opposite target locations in the task-defined preferred reach direction of the 142 unit and the opposite direction for hundreds of trials in the two tasks. Therefore, it is 143 reasonable to expect that monkey Z had some level of implicit prior knowledge of where 144 the two targets would appear at the start of each trial in both tasks and all that it lacked 145 was the specific color-location conjunction, which it obtained when the targets 146 appeared. The activity during the Checkerboard observation period in the CF task may 147 have reflected the accumulating knowledge of the dominant color of the checkerboard in 148 neural circuits outside of PMd. This may have enabled a covert activation of the 149 simultaneous representations of the two anticipated potential actions even before the 150 targets actually appeared in the CF task since that accumulating sensory evidence will 151 eventually support one or the other of the two colored targets once they appeared. This 152 coactivation of the two PMd populations preferring the two targets may have in turn 153 contributed to the shorter RTs in the CF task than the TF task for even the 100% 154 checkerboards. Critically, however, the linear regression ( Figure 5), ROC ( Figure 6) and 155 ANOVA (Supplemental Figure 3) showed that these activations during the first-cue 156 observation periods in PMd of monkey Z did not reflect the final decision-related 157 processes as defined here because it did not predict any aspect of monkey Z's 158 differential choice behavior after the second visual cue appeared in each task. histological localization of penetration sites has not yet been done. However, similar 166 stereotactic coordinates for chamber implants and extensive relative overlap of 167 recording penetrations within the chambers in the two monkeys suggest that is not the 168 main cause ( Figure 1E).

170
Another possible explanation is the different training history of the two monkeys.

171
Monkey T only ever experienced L/R targets and was initially trained in the TF task in 172 which the targets remained visible for the duration of each trial. Thus, its original training 173 experience did not require the establishment of a memorized trace of the target 174 information. In contrast, monkey Z was trained for many months in the 1T and 2T tasks 175 with targets in 8 different directions that varied from trial to trial and with two long 176 sequential memory-delay periods during which the monkey had to remember the spatial 177 location (1T task) and color-location conjunctions of the targets in each trial before 178 selecting a target (2T task), and initiating a reach (1T and 2T tasks). The 1T and 2T 179 tasks were also used in all neural recording sessions to search for task-related units. 180 Engagement of PMd during the memory-delay periods of the 1T and 2T tasks may have 181 facilitated task performance for monkey Z, which was carried over to the TF and CF 182 tasks.

184
Another possible contributing factor is the target location placement in the tasks. For 185 monkey Z, targets were placed in spatial locations along the preferred-opposite 186 movement direction axis of each unit to maximize the difference in their directional 187 activity in the TF and CF tasks, and would change from unit to unit. For monkey T, in 188 contrast, target locations were fixed to the left and right of center, and units were 189 recorded for that single movement axis, regardless of what their preferred reach 190 directions might have been.

192
Finally, the lack versus presence of activity changes after the checkerboards appeared 193 in the CF/CFD tasks may have reflected a difference in the strategy that the two 194 monkeys adopted to perform the tasks. Monkey T may have adopted a strategy 195 whereby it attempted to store a purely "sensory" mnemonic representation of the 196 checkerboard stimuli but largely deferred the interpretation of the checkerboard 197 evidence until the appearance of the targets. This resulted in no PMd responses and 198 longer RTs in the CFD task compared to the TF task. In contrast, monkey Z appeared to 199 largely commit to a categorical decision about the dominant color of the checkerboard 200 while observing it in the CF task, resulting in a substantial reduction in RTs after the 201 targets appeared. This may have been accompanied by a covert activation of the two 202 competing action-related PMd populations, like in the TF task, while monkey Z 203 deliberated on the color evidence in the checkerboard, even though the targets had not 204 yet appeared. This is further reinforced by the finding that monkey Z's RTs remained 205 drastically shorter in a modified version of the CF task that had the same temporal 206 structure as the CFD task. This showed that monkey T's prolonged RTs in the CFD task 207 compared to the TF task were not due solely to the imposed memory-delay period but 208 rather to how and when it interpreted that task-relevant sensory evidence provided by 209 the checkerboards.

211
How did the monkeys convert checkerboard color coherence into reach actions? 212 213 The In contrast, the checkerboards used in these tasks comprised sets of small squares 227 whose colors are easily and rapidly discriminable, and contained no input signal "noise" 228 comparable to the variable numbers of dots moving in the coherent and random 229 directions from frame to frame in the RDK stimuli. The checkerboard stimuli presented 230 in half of the trials to monkey Z were dynamic and changed every 50ms, but the 231 numbers of blue and yellow squares in each stimulus stream remained fixed in a given 232 trial and only their positions within the checkerboard changed from image frame to 233 image frame. In contrast, in the other half of the trials for monkey Z and all of the trials 234 for monkey T, a single static checkerboard of R/G or B/Y squares appeared for the 235 duration of the observation period in each trial, so that the physical properties of the 236 sensory stimulus that the monkeys experienced did not change across time. Despite 237 these differences in the visual stimuli, the two monkeys showed remarkably similar 238 chronometric and psychophysical trends in the TF task (present study; 20 ). Furthermore, 239 human and non-human subjects also showed very similar performance when viewing 240 either dynamic or static checkerboard stimuli ( 2,21 ; present study). Thus, whereas the 241 motion sensations evoked by RDK stimuli require dynamically changing stimuli across 242 time, the assessment of the color evidence in the checkerboards was relatively 243 insensitive to the presence or absence of continually updated sensory inputs. This does 244 not, however, preclude momentary stochastic noise generated within the central neural 245 circuits that process even the static checkerboard visual input (both monkeys) and store 246 it in short-term working memory (monkey T). 247 248 The similarity of task performance in the TF task is also striking given another difference 249 in the checkerboard stimuli experienced by the two monkeys. The checkerboards used 250 in monkey T's experiments contained only task-salient R and G squares. In contrast, the 251 checkerboards used in monkey Z's experiments contained 100 task-salient B and Y 252 squares against a background of 125 task-irrelevant R squares. This reduced the 253 overall density of task-relevant color information in the checkerboards for monkey Z and 254 required it to identify the task-relevant information from among the "distractor" red 255 squares. Despite this difference, the psychophysical curves and psychophysical 256 thresholds of the two monkeys were very similar in the TF task, and the RTs for the 257 high-coherence checkerboards were actually shorter in monkey Z than monkey T 258 (Figure 1).

260
Despite the absence of color evidence "noise" in the checkerboards and the insensitivity 261 of task performance to static versus dynamic stimuli, the monkeys took longer to choose 262 a colored target when the checkerboard coherence decreased 2,20,21 . In RDK stimuli, this 263 effect has been explained by a sequential-sampling process that takes longer to identify 264 the direction of the weak coherent-motion signal generated by MT neurons against a 265 high level of motion direction noise. For the checkerboard stimuli, in contrast, it 266 presumably reflects a longer period of time required to determine whether the 267 checkerboard was predominantly one or the other of the two task-salient colors as the 268 numbers of squares of the two easily-discriminable colors became more similar. This 269 may require longer re-sampling of the sensory input while observing the checkerboard 270 (monkey Z) or from a noisy working-memory trace of the checkerboard (monkey T; 22,23 , 271 Shushruth and Shadlen, CoSyNe abstract). We can assume that the color evidence is 272 initially processed by neurons in the parvocellular "color-opponent" pathway 24-27 . 273 However, to our knowledge, there have been no studies of neural responses in that 274 pathway to multi-colored checkerboard stimuli like ours in color discrimination tasks 275 analogous to the many studies of visual motion processing in MT.

277
The perceptual decision could be considered as a pure color discrimination problem 278 since the subjects had to estimate the dominant color of the checkerboards in order to 279 identify the reach target whose color matched that of the checkerboard. However, 280 similar dichromatic dot arrays have been used in studies of numerosity, the ability of 281 subjects to estimate relative numbers of visual objects 28-30 . Subjects likewise showed 282 longer RTs when the relative numbers of objects in the stimuli are similar 30,31 . These 283 results have been interpreted as consistent with a process of sequential sampling and 284 accumulation of evidence across time and across space within the stimuli 30-33 , but did 285 not speculate on the nature of the sensory evidence that was being sampled, unlike 286 RDK stimuli. Furthermore, the checkerboards that we used have inherent in them 287 several potential confounding "low level" physical properties identified in numerosity 288 studies that are independent of the presumably "higher level" sense of relative numbers 289 per se, including the relative area of the checkerboard occupied by squares of each 290 color, their total circumference, and the relative degree of spatial contiguity of squares 291 of the same color (i.e., how often they cluster to share a common border) 30,34-37 . Indeed, 292 the colored squares did not have a neutral-colored border and so would form larger 293 monochromatic "clumps" when contiguous, rather than remaining visible as discrete 294 squares ( Figure 1D). All of these factors could have contributed to the monkeys' 295 estimation of the dominant color of the checkerboards, independent of any estimate of 296 relative numbers of squares. Furthermore, the number and density of squares in our 297 checkerboards were usually higher than normally used in numerosity studies and more 298 closely resemble what are called "textures", which follow different psychophysical laws 299 than dot arrays with smaller numbers of elements 28 . 300 301 However, this study was not designed to study numerosity or to examine what specific 302 properties of the checkerboards the subjects used to make the relative color estimates. 303 Instead, the checkerboards were chosen as a means to present stimuli with different 304 levels of competing evidence for two alternative reach choices, using a stimulus 305 dimension (color) that has no inherent natural association with the directionality of motor 306 output. Our findings indicate that PMd units express activity pertaining to the likelihood 307 of different action choices provided by the checkerboard stimuli, independent of the 308 critical decision-relevant physical property of the sensory input on which those action 309 likelihoods are based, in this case its dominant color. 310 311 Important questions not directly addressed by this study are where are the neural 312 correlates of the critical color-related information on which the action decisions were 313 based and how are they transformed into color-independent evidence supporting the 314 action choices? A strong candidate is the dorsolateral prefrontal cortex 38,39 . We have 315 preliminary evidence that the specific color/location conjunctions of the spatial target 316 cues and color/location matching rules after the checkerboard appeared in each trial are 317 expressed in lateral prefrontal cortex around the principal sulcus while a monkey 318 performed a TF task (Coallier et al., 2008, SfN abstract). 319 320 The effect of checkerboard color coherence on task performance and PMd neural 321 activity is consistent with a number of different computational decision-making models, 322 including drift-diffusion 17,18,30,40,41 ; gated stochastic accumulation 42,43 , urgency 323 gating 44,45 , and independent-race 46-48 . Nevertheless, we acknowledge that our neural 324 data are correlational and are not proof of a causal relationship between PMd activity 325 and either perceptual or motor decisions. Furthermore, until more neurophysiological 326 findings are available about the sources and nature of sensory signals that are being 327 processed while subjects estimate the relative amounts of colored squares in the 328 dichromatic checkerboard stimuli, and how those sensory signals are transformed into 329 action-related information, we prefer to remain agnostic as to the computational 330 mechanisms that underlie the task performance of the subjects. 331