Mapping the neuronal building blocks of human language with language models

Main

Language is a fundamental cognitive process unique to humans that allows us to communicate complex and highly diverse meanings. It also enables us to construct new and potentially inexhaustible expressions through grammatical rules that govern how we arrange and combine words and that can generalize across sentences beyond simple repetition. Linguistic studies based on behavioural observations have accurately described how we use the properties of words, such as their grammatical function and syntactic relationships, to construct unique sentences 1 , 2 , 3 . They have also described how we use the hierarchical structure of phrases and sentences to communicate specific ideas and thoughts 6 . Recent neuroimaging 7 , 8 , 9 , 10 , 11 and electrocorticographic recordings 5 , 12 , 13 , 14 , 15 , in turn, have identified broadly distributed regions across the frontotemporal cortex that reliably engage in speech production and sentence construction 16 , 17 , 18 , 19 and that differentiate grammatically well-formed sentences from unstructured word lists or sounds, suggesting their sensitivity to syntactic structure 20 , 21 . They have also found neural responses that reflect the merger of words into phrases 6 and their semantic properties 22 , 23 , revealing a broad macroscopic network of cortical areas that could support human language.

However, understanding the microscopic organization and cortical landscape by which linguistic information is encoded by neurons in humans or the cellular processes that underlie natural speech has remained a longstanding challenge. While recent investigations have revealed how the phonetic components of words are encoded by neurons 24 , 25 , they do not reveal the cellular processes by which we produce meaningful speech or through which we arrange and combine words into phrases and sentences. To convey meaning through language, for example, humans use abstract grammatical categories (such as adjectives and nouns) and dependencies (such as the relationship between nominal subjects and objects) that can generalize across sentences and that can describe complex relationships such as actions and outcomes. Yet, how such grammatical features are encoded by neurons or whether they generalize across sentences largely remains undefined. Little is also known about whether neurons can represent the higher-order syntactic structure of sentences or how they encode their constituent phrases.

Another prominent question in neurolinguistics is whether syntactic information is dissociable to some degree from that of semantic information, or how these core aspects of language are represented at a cellular scale 26 , 27 . Whereas previous imaging studies have suggested that linguistic information is probably represented broadly across the brain 28 , 29 , little is also known about how such information is distributed across cortical regions or whether speech processes are lateralized at a basic cellular level. Finally, little is known about how the local activities of individual neurons and their tuning properties relate to those of the populations’ broader field potential patterns, or what their local organization within the cortex may be.

Here we performed single-neuronal recordings across the human frontotemporal cortex and tracked their action potential (AP) and local field potential (LFP) activities over long-term durations as participants produced natural speech. Using speech tracking, parsing, modelling and decoding techniques, we describe the detailed organization and encoding properties of the neurons. We also describe their distribution, regionality and lateralization, together providing a detailed examination of linguistic information encoding during language production at a combined micro (cellular), meso (local population) and macro (regional) scale.

Recording neurons during natural speech

Recordings were obtained from neurons across the human frontotemporal cortex using semichronically implanted microelectrode arrays (96 channel configuration) 30 , 31 , 32 . These arrays were implanted as part of planned neurosurgical care for epilepsy monitoring 30 , 33 , 34 and were located in the frontal 35 , 36 , anterior temporal 20 , 37 and posterior temporal 38 , 39 , 40 regions that have been shown to reliably engage in speech production and sentence construction 41 , 42 , 43 and to display robust language-selective responses 41 , 42 , 44 by validated language localizers 28 , 41 , 45 (Fig. 1a ). They were also placed in participants who were awake, after implantation, and were therefore capable of producing natural speech, together providing a rare opportunity to study the activities of individual neurons during natural language production (see the ‘Microelectrode recordings’ section of the Methods ). In total, we recorded from 579 putative neurons in 8 participants (3 female, 5 male, aged 27 to 52 years) and 14 sessions and only well-isolated single units with stable waveform morphologies c…