CHDH Seminar: Brain activation elicited in reading sentences reflects distributional and experiential knowledge

Presenter: Andrew J. Anderson

Date: Thursday 12 December

Time: 4 - 5 PM

Location: Latham Theatre (Room 102 Redmond Barry Building, University of Melbourne)

Title: Brain activation elicited in reading sentences, and when recollecting personal experiences reflects distributional and experiential knowledge

Abstract: Language gives humans the unique ability to communicate about historical events, theoretical concepts and fiction. Although words are learnt through language and defined by their relations to other words in dictionaries, our understanding of word meaning presumably draws heavily on our non-linguistic sensory, motor, interoceptive and emotional experiences with words and their referents. Behavioral experiments lend support to the intuition that word meaning integrates aspects of linguistic and non-linguistic “experiential” knowledge. Direct neural evidence has been lacking. Indeed, functional neuroimaging studies aiming at decoding language meaning from neural activity have mostly relied on distributional models of word semantics, which are based on patterns of word co-occurrence in text corpora. This talk firstly presents initial evidence that both experiential and linguistically acquired knowledge can be detected in brain activity elicited in sentence reading. Attributes of peoples’ sensory, motor, social, emotional and cognitive experiences with words were modeled using crowd-sourced behavioral ratings. We demonstrate that functional Magnetic Resonance Imaging (fMRI) activation elicited in sentence reading is more accurately decoded when this experiential attribute model is integrated with a distributional model than when either model is applied in isolation. In a second experiment, we examine how distributional and experiential models can be applied to explain autobiographical memories. Participants underwent fMRI as they recollected common personal experiences (e.g. weddings, dancing). We construct personalised experiential and distributional models, and apply them to discriminate individual identify from associated fMRI activation. This showcases that personal memories can be quantitatively measured using (re)imagination, neuroimaging, and distributional/experiential modeling.

Bio: Dr. Andrew Anderson is a Research Assistant Professor in the department of Neuroscience, University of Rochester (USA). He researches how the meaning of language is represented in the human brain and how this can break down in clinical conditions. He typically uses functional Magnetic Resonance Imaging (fMRI) and Electroencephalography (EEG) to measure brain activity patterns elicited in language comprehension. Then, he applies computational modelling and machine learning to decode semantic information from brain activity. An emphasis of his work has been on natural language comprehension. Recent contributions include introducing the first computational model to predict fMRI activity associated with sentence meaning, and revealing an EEG-based index associated with narrative speech comprehension. In moving forward, he seeks to expose how meaning is exchanged between brains in conversation, and how this process is constrained by personalities, social cues and context. Advances may help in characterising neurodegenerative and developmental disorders associated with language and memory, and provide guidance for artificial intelligence research.