Cortical activity patterns predict speech discrimination ability

Article metrics

Abstract

Neural activity in the cerebral cortex can explain many aspects of sensory perception. Extensive psychophysical and neurophysiological studies of visual motion and vibrotactile processing show that the firing rate of cortical neurons averaged across 50–500 ms is well correlated with discrimination ability. In this study, we tested the hypothesis that primary auditory cortex (A1) neurons use temporal precision on the order of 1–10 ms to represent speech sounds shifted into the rat hearing range. Neural discrimination was highly correlated with behavioral performance on 11 consonant-discrimination tasks when spike timing was preserved and was not correlated when spike timing was eliminated. This result suggests that spike timing contributes to the auditory cortex representation of consonant sounds.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Spectrograms of each speech sound grouped by manner and place of articulation.
Figure 2: Neurograms depicting the onset response of rat A1 neurons to 20 English consonants.
Figure 3: Predictions of consonant discrimination ability based on onset response similarity.
Figure 4: Behavioral discrimination of consonant sounds.
Figure 5: Both average A1 responses and trial-by-trial neural discrimination predicted consonant discrimination ability when temporal information was maintained.
Figure 6: Predictions of consonant discrimination ability based on nearest-neighbor classifier.
Figure 7: Neural discrimination using the onset activity pattern from individual multiunit sites was best correlated with behavior.

References

  1. 1

    DeWeese, M.R., Hromadka, T. & Zador, A.M. Reliability and representational bandwidth in the auditory cortex. Neuron 48, 479–488 (2005).

  2. 2

    Parker, A.J. & Newsome, W.T. Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, 227–277 (1998).

  3. 3

    Liu, J. & Newsome, W.T. Correlation between speed perception and neural activity in the middle temporal visual area. J. Neurosci. 25, 711–722 (2005).

  4. 4

    Pruett, J.R. Jr., Sinclair, R.J. & Burton, H. Neural correlates for roughness choice in monkey second somatosensory cortex (SII). J. Neurophysiol. 86, 2069–2080 (2001).

  5. 5

    Romo, R. & Salinas, E. Flutter discrimination: neural codes, perception, memory and decision making. Nat. Rev. Neurosci. 4, 203–218 (2003).

  6. 6

    Britten, K.H., Shadlen, M.N., Newsome, W.T. & Movshon, J.A. The analysis of visual motion: a comparison of neuronal and psychophysical performance. J. Neurosci. 12, 4745–4765 (1992).

  7. 7

    Narayan, R., Grana, G. & Sen, K. Distinct time scales in cortical discrimination of natural sounds in songbirds. J. Neurophysiol. 96, 252–258 (2006).

  8. 8

    Schnupp, J.W., Hall, T.M., Kokelaar, R.F. & Ahmed, B. Plasticity of temporal pattern codes for vocalization stimuli in primary auditory cortex. J. Neurosci. 26, 4785–4795 (2006).

  9. 9

    Walker, K.M., Ahmed, B. & Schnupp, J.W. Linking cortical spike pattern codes to auditory perception. J. Cogn. Neurosci. 20, 135–152 (2008).

  10. 10

    Ahissar, E. et al. Speech comprehension is correlated with temporal response patterns recorded from auditory cortex. Proc. Natl. Acad. Sci. USA 98, 13367–13372 (2001).

  11. 11

    Orduna, I., Mercado, E. III, Gluck, M.A. & Merzenich, M.M. Cortical responses in rats predict perceptual sensitivities to complex sounds. Behav. Neurosci. 119, 256–264 (2005).

  12. 12

    Wang, L., Narayan, R., Grana, G., Shamir, M. & Sen, K. Cortical discrimination of complex natural stimuli: can single neurons match behavior? J. Neurosci. 27, 582–589 (2007).

  13. 13

    Bertoncini, J., Bijeljac-Babic, R., Blumstein, S.E. & Mehler, J. Discrimination in neonates of very short CVs. J. Acoust. Soc. Am. 82, 31–37 (1987).

  14. 14

    Blumstein, S.E. & Stevens, K.N. Acoustic invariance in speech production: evidence from measurements of the spectral characteristics of stop consonants. J. Acoust. Soc. Am. 66, 1001–1017 (1979).

  15. 15

    Fowler, C.A., Brown, J.M., Sabadini, L. & Weihing, J. Rapid access to speech gestures in perception: evidence from choice and simple response time tasks. J. Mem. Lang. 49, 396–413 (2003).

  16. 16

    Jongman, A. Duration of frication noise required for identification of English fricatives. J. Acoust. Soc. Am. 85, 1718–1725 (1989).

  17. 17

    Kuhl, P.K. & Miller, J.D. Speech perception by the chinchilla: voiced-voiceless distinction in alveolar plosive consonants. Science 190, 69–72 (1975).

  18. 18

    Reed, P., Howell, P., Sackin, S., Pizzimenti, L. & Rosen, S. Speech perception in rats: use of duration and rise time cues in labeling of affricate/fricative sounds. J. Exp. Anal. Behav. 80, 205–215 (2003).

  19. 19

    Miller, G.A. & Nicely, P.E. An analysis of perceptual confusions among some English consonants. J. Acoust. Soc. Am. 27, 338–352 (1955).

  20. 20

    Steinschneider, M., Fishman, Y.I. & Arezzo, J.C. Representation of the voice onset time (VOT) speech parameter in population responses within primary auditory cortex of the awake monkey. J. Acoust. Soc. Am. 114, 307–321 (2003).

  21. 21

    Steinschneider, M., Reser, D., Schroeder, C.E. & Arezzo, J.C. Tonotopic organization of responses reflecting stop consonant place of articulation in primary auditory cortex (A1) of the monkey. Brain Res. 674, 147–152 (1995).

  22. 22

    Tavabi, K., Obleser, J., Dobel, C. & Pantev, C. Auditory evoked fields differentially encode speech features: an MEG investigation of the P50m and N100m time courses during syllable processing. Eur. J. Neurosci. 25, 3155–3162 (2007).

  23. 23

    Young, E.D. Neural representation of spectral and temporal information in speech. Phil. Trans. R. Soc. Lond. B Biol. Sci. 363, 923–945 (2008).

  24. 24

    Steinschneider, M. et al. Intracortical responses in human and monkey primary auditory cortex support a temporal processing mechanism for encoding of the voice onset time phonetic parameter. Cereb. Cortex 15, 170–186 (2005).

  25. 25

    Cooke, J.E., Zhang, H. & Kelly, J.B. Detection of sinusoidal amplitude modulated sounds: deficits after bilateral lesions of auditory cortex in the rat. Hear. Res. 231, 90–99 (2007).

  26. 26

    Dewson, J.H. III, Pribram, K.H. & Lynch, J.C. Effects of ablations of temporal cortex upon speech sound discrimination in the monkey. Exp. Neurol. 24, 579–591 (1969).

  27. 27

    Heffner, H.E. & Heffner, R.S. Effect of restricted cortical lesions on absolute thresholds and aphasia-like deficits in Japanese macaques. Behav. Neurosci. 103, 158–169 (1989).

  28. 28

    Rybalko, N., Suta, D., Nwabueze-Ogbo, F. & Syka, J. Effect of auditory cortex lesions on the discrimination of frequency-modulated tones in rats. Eur. J. Neurosci. 23, 1614–1622 (2006).

  29. 29

    Wetzel, W., Ohl, F.W., Wagner, T. & Scheich, H. Right auditory cortex lesion in Mongolian gerbils impairs discrimination of rising and falling frequency-modulated tones. Neurosci. Lett. 252, 115–118 (1998).

  30. 30

    Steinschneider, M., Volkov, I.O., Noh, M.D., Garell, P.C. & Howard, M.A. III. Temporal encoding of the voice onset time phonetic parameter by field potentials recorded directly from human auditory cortex. J. Neurophysiol. 82, 2346–2357 (1999).

  31. 31

    Wong, S.W. & Schreiner, C.E. Representation of CV-sounds in cat primary auditory cortex: intensity dependence. Speech Commun. 41, 93–106 (2003).

  32. 32

    Kluender, K.R., Diehl, R.L. & Killeen, P.R. Japanese quail can learn phonetic categories. Science 237, 1195–1197 (1987).

  33. 33

    Ramus, F., Hauser, M.D., Miller, C., Morris, D. & Mehler, J. Language discrimination by human newborns and by cotton-top tamarin monkeys. Science 288, 349–351 (2000).

  34. 34

    Sinnott, J.M. & Brown, C.H. Perception of the American English liquid /ra-la/ contrast by humans and monkeys. J. Acoust. Soc. Am. 102, 588–602 (1997).

  35. 35

    Toro, J.M., Trobalon, J.B. & Sebastian-Galles, N. Effects of backward speech and speaker variability in language discrimination by rats. J. Exp. Psychol. Anim. Behav. Process. 31, 95–100 (2005).

  36. 36

    Dooling, R.J., Okanoya, K. & Brown, S.D. Speech perception by budgerigars (Melopsittacus undulatus): the voiced-voiceless distinction. Percept. Psychophys. 46, 65–71 (1989).

  37. 37

    Foffani, G. & Moxon, K.A. PSTH-based classification of sensory stimuli using ensembles of single neurons. J. Neurosci. Methods 135, 107–120 (2004).

  38. 38

    Weliky, M., Fiser, J., Hunt, R.H. & Wagner, D.N. Coding of natural scenes in primary visual cortex. Neuron 37, 703–718 (2003).

  39. 39

    Grace, J.A., Amin, N., Singh, N.C. & Theunissen, F.E. Selectivity for conspecific song in the zebra finch auditory forebrain. J. Neurophysiol. 89, 472–487 (2003).

  40. 40

    Buonomano, D.V. & Merzenich, M. A neural network model of temporal code generation and position-invariant pattern recognition. Neural Comput. 11, 103–116 (1999).

  41. 41

    VanRullen, R., Guyonneau, R. & Thorpe, S.J. Spike times make sense. Trends Neurosci. 28, 1–4 (2005).

  42. 42

    Richmond, B.J., Optican, L.M. & Spitzer, H. Temporal encoding of two-dimensional patterns by single units in primate primary visual cortex. I. Stimulus-response relations. J. Neurophysiol. 64, 351–369 (1990).

  43. 43

    Ohl, F.W. & Scheich, H. Orderly cortical representation of vowels based on formant interaction. Proc. Natl. Acad. Sci. USA 94, 9440–9444 (1997).

  44. 44

    Qin, L., Chimoto, S., Sakai, M. & Sato, Y. Spectral-shape preference of primary auditory cortex neurons in awake cats. Brain Res. 1024, 167–175 (2004).

  45. 45

    Versnel, H. & Shamma, S.A. Spectral-ripple representation of steady-state vowels in primary auditory cortex. J. Acoust. Soc. Am. 103, 2502–2514 (1998).

  46. 46

    Buonomano, D.V. & Merzenich, M.M. Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267, 1028–1030 (1995).

  47. 47

    Kawahara, H. Speech representation and transformation using adaptive interpolation of weighted spectrum: vocoder revisited. Proc. 1997 IEEE Int. Conf. Acoust., Speech, Signal Process 2, 1303–1306 (1997).

  48. 48

    Engineer, N.D. et al. Environmental enrichment improves response strength, threshold, selectivity, and latency of auditory cortex neurons. J. Neurophysiol. 92, 73–82 (2004).

  49. 49

    Rennaker, R.L., Ruyle, A.M., Street, S.E. & Sloan, A.M. An economical multi-channel cortical electrode array for extended periods of recording during behavior. J. Neurosci. Methods 142, 97–105 (2005).

  50. 50

    Rennaker, R.L., Street, S., Ruyle, A.M. & Sloan, A.M. A comparison of chronic multi-channel cortical implantation techniques: manual versus mechanical insertion. J. Neurosci. Methods 142, 169–176 (2005).

Download references

Acknowledgements

The authors would like to thank J. Roland, R. Jain and D. Listhrop for assistance with microelectrode mappings. We would like to thank R. Rennaker for technical assistance and training and for providing microelectrode arrays and inserter. We would also like to thank M. Perry, C. Heydrick, A. McMenamy, A. Meepe, C. Dablain, J. Choi, V. Badhiwala, J. Riley, N. Hatate, P. Kan, M. Lazo de la Vega and A. Hudson for help with behavioral training. We would also like to thank S. Blumstein, Y. Cohen, H. Read, S. Denham, L. Miller, S. Edelman, V. Dragoi, H. Abdi, P. Assmann, X. Wang and R. Romo for their suggestions about earlier versions of the manuscript. This work was supported by grants from the US National Institute for Deafness and Other Communicative Disorders and the James S. McDonnell Foundation.

Author information

C.T.E., C.A.P., R.S.C. and A.C.R. collected behavioral training data. C.T.E., C.A.P., Y.H.C., R.S.C., V.J. and K.Q.C. recorded anesthetized cortical responses. J.A.S. recorded awake cortical responses. M.P.K. and C.T.E. wrote the manuscript and performed data analysis. All authors discussed the paper and commented on the manuscript.

Correspondence to Crystal T Engineer or Michael P Kilgard.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–8, Supplementary Table 1, Supplementary Data (PDF 1110 kb)

Supplementary Video 1

A1 spatiotemporal activity evoked by consonant onset. The video illustrates the spatiotemporal activity patterns evoked in primary auditory cortex by the onset of twenty different consonant sounds. The color of each polygon indicates the multi-unit activity at each of 63 recording sites in a single rat. The map of characteristic frequency (CF) illustrates the topographic organization of tone frequency tuning. The blue lines under each activity map indicate the average firing rate for all 63 A1 sites. The red lines mark the time at which each spatial activity pattern occurs relative to response onset. (MPG 2459 kb)

Supplementary Video 2

A1 spatiotemporal activity evoked by consonant-vowel syllables. The video illustrates the spatiotemporal activity patterns evoked in primary auditory cortex by words beginning with twenty different consonant sounds followed by /a/ as in 'sad'. The color of each polygon indicates the multi-unit activity at each of 63 recording sites in a single rat. The map of characteristic frequency (CF) illustrates the topographic organization of tone frequency tuning. The blue lines under each activity map indicate the average firing rate for all 63 A1 sites. The red lines mark the time at which each spatial activity pattern occurs relative to response onset. (MPG 4245 kb)

Rights and permissions

Reprints and Permissions

About this article

Further reading