Compositionality of letter shape in word recognition

Abstract

As you read this sentence, your brain just performed a miraculous task of converting collections of letter shapes into meaning. Reading is a cultural invention that is thought to exploit the intrinsic recognition abilities of our visual system, but it also leads to widespread changes in the brain. How do visual representations change to enable efficient reading i.e. our ability to read words at a glance? It is widely believed that learning to read should lead to the formation of novel detectors for letter combinations, thereby creating word responses that are not predictable from single letters. Alternatively, reading could lead to separable or compositional word responses that are predictable from familiar letters or scripts. There is insufficient evidence to resolve this fundamental question in the literature. In my thesis, I have performed 3 main studies to address this fundamental question. In the first study, I explored the changes in representation associated with reading expertise. To address this, I compared the visual representations of readers and non-readers of two Indian languages, Telugu and Malayalam. I found a subtle change in visual representation with reading expertise, but surprisingly it decreased the interaction between letters of a word, thereby, making the letters of a word more compositional. Using fMRI, I found the locus of this effect in higher visual areas. In the second study, I explored the nature of visual representation that enable us to read words with spelling mistakes (jumbled words). To address this, I built computational models to predict the visual similarity between any two strings. This model is compositional in nature i.e. response of a word can be predicted using its individual letters. Interestingly, the time taken to identify a jumbled word or to classify it as a nonword is dependent solely on the visual similarity. This result extends the intrinsic capabilities of our visual system in word recognition. In the third study, I investigated the underlying neural correlates..