Back to search

FRIHUMSAM-Fri prosj.st. hum og sam

Vision shapes language: Referential communication in blind, sighted and newly-sighted individuals

Alternative title: Syn former språk: Referensiell kommunikasjon blant blinde, seende og personer som får synet tilbake

Awarded: NOK 11.4 mill.

This research project aims to investigate the contribution of visual processes to verbal communication; in particular, the act of referring to the physical world around us. In order to investigate this question, the communicative abilities of sighted and blind children will be studied independently and comparatively. The project will also investigate the referential abilities of a third and unique population, the Prakash children of Delhi: newly-sighted children who gained vision after they acquired language. Since language acquisition takes place either with or without vision, studying the communicative development of newly-sighted children will give me an extraordinary opportunity: to isolate the contribution of visual processes to the development of communication. My first research question is therefore unprecedented: how do the communicative abilities of blind children change once they gain vision? The project will combine three research strands: blind, sighted and newly-sighted communication, and compare the communicative abilities of school children with those of adults. The main focus of the three research strands will be the process of 'audience design', or how speakers tailor their referential expressions to the needs of their interlocutors. To investigate this question, I will use referential communication tasks in which pairs of participants interact with one another to arrange various objects in a display. My main interest in this type of task is to see how speakers refer to the different objects in the display, and whether their choice of referential expression reveals that children are taking their interlocutor's perspective (e.g., by avoiding ambiguity). The speakers' eye-movements will be recorded in order to investigate how visual processes affect the generation of referential expressions. I will also adopt a longitudinal perspective, following the communicative development Prakash children over a period of 4 years. Seven months into the project, we have piloted an eye-tracking task on two groups of Prakash children and a group of sighted controls and are getting ready to run the final version of the task in the new year. This study will look at how children follow perceptual cues in referential communication (e.g., gaze following) and how these abilities develop in the Prakash children. We have also piloted a social cognition task on blind children and are also planning to run the actual experiment in the new year. The aim of that study is to evaluate blind children's understanding of effort and reward from other people's behavior. In 2018-2019, we completed 3 experiments investigating the communicative abilities of the Prakash kids with a focus on their attention to faces. While their communication was comparable to sighted controls, we noticed that the Prakash children did not make eye contact or follow their interlocutor's head movement to orient towards new referents. We are now in the process of designing one more task, where we hope that the Prakash children will start showing an interest in faces if they get adequate feedback. In 2019-2020, we piloted the last task of our study investigating children's attention to faces in referential communication. Unfortunately, because of the COVID-19 pandemic, we have been unable to complete that study. However, since the lockdown started in March 2020, we have written up 6 papers, which are currently under review in top journals (i.e. Cognition; Journal of Memory and Language; Cognitive Psychology; Nature Communications; Journal of Child Language; Linguistics and Philosophy). This is in addition to the 5 papers we already published in 2020. Since it is unclear when the ongoing pandemic will be resolved, we have also been working on adapting our interactive tasks to be delivered via Skype/Zoom, both to the Prakash Girls schooled at Silver Linings and to the control children we are recruiting online at a local school in Delhi. Data collection has started this week and we hope to have completed two studies by January 2021. In 2020-2021, we adapted our experimental tasks to be conducted on-line because of the restrictions imposed by the COVID pandemic. We were able to complete two tasks with Prakash children, neurotypical controls and adults in the Delhi area. These two papers have already been written up and submitted, and are currently under review with Developmental Psychology and Neuropsychologia. In addition, we have finished coding and analyzing a third study (including two separate experiments) and are currently writing it up. We also have an additional seven papers currently under review. Between 2021-2023, we published all submitted papers and completed four more studies: (1) a storytelling task with Hindi-speaking children and adults, (2) a spatial-cognition task with blind children, (3) an interactive task using eye-tracking with Turkish-speaking adults, and (4) an online demonstrative-choice task in 10 languages.

FROM THE GRANT PROPOSAL: "This project has groundbreaking potential in several research areas, providing cutting edge evidence that will bear on a number of theoretical and empirical debates in the cognitive sciences (e.g. theoretical pragmatics, developmental psychology and computational linguistics) and innovative applications in the design of intervention programs for blind and newly-sighted individuals. I expect that the 10 studies outlined in this proposal should provide results for at least 10 papers in top journals in Psychology (e.g. Science, Psychological Science, Nature Human Behavior, and Proceedings of the National Academy of Sciences), Linguistics (e.g. Language, Journal of Memory & Language, and Language, Cognition & Neuroscience) and Philosophy (e.g. Mind & Language). The results of our studies will also be presented at international conferences by the members of the research team (e.g. CogSci, CUNY, XPrag or Neuroscience)." ACTUAL OUTCOMES AND IMPACTS BASED ON THE PROJECT RESULTS: As promised in the grant proposal, the studies conducted in this research project addressed important debates in Experimental Pragmatics, Social Cognition, Computational Science and Neuroscience. Since the start of the project in 2018, we published a total of 26 papers, which is 16 more than I estimated in the grant proposal. These papers appeared in top journals such as PNAS (x2), Science Advances, Psychological Review (x2), Scientific Reports, Cognition (x2), Journal of Experimental Psychology: General, Journal of Experimental Psychology: Learning, Memory & Cognition (x2), Cognitive Development (x2), Infant Behavior & Development, Cognitive Science, Child Development, Neuropsychologia (x2), Journal of Memory & Language (x2), Glossa Psycholinguistics and Cognitive Psychology -- among others. The high productivity of this research project is all the more remarkable given that we had to conduct and publish many of our studies during the COVID pandemic. Despite these difficulties, we continued to present our work at international conferences such as XPrag, CUNY (now called HSP), CogSci and ESPP. Our collaboration with Silver Linings (the school for the blind where a number of Prakash girls are receiving tailored education through Project Prakash) was also a great success that improved the school experience of the Prakash girls in our studies.

This research project aims to investigate the contribution of visual processes to verbal communication; in particular, the act of referring to the physical world around us. In order to investigate this question, the referential abilities of sighted and blind individuals will be studied independently and comparatively. The project will also investigate the referential abilities of a third and unique population, the Prakash children of Delhi: newly-sighted children who gained vision after they acquired language. Since language acquisition takes place either with or without vision, studying the communicative development of newly-sighted children will give me an extraordinary opportunity: to isolate the contribution of visual processes to the development of referential communication. My first research question is therefore unprecedented: how do the communicative abilities of blind children change once they gain vision? The project will combine three research strands: blind, sighted and newly-sighted communication, and compare the referential abilities of school children with those of adults. The main focus of the three research strands will be the pragmatic process of 'audience design' in referential communication, or how speakers tailor their referential expressions to the needs of their interlocutors. To investigate this question, I will use referential communication tasks in which pairs of participants interact with one another to arrange various objects in a display. My main interest in this type of task is to see how speakers refer to the different objects in the display, and whether their choice of referential expression reveals perspective taking (e.g., by avoiding ambiguity). The speakers' eye-movements will be recorded in order to investigate how visual processes affect the generation of referential expressions. I will also adopt a longitudinal perspective, following the pragmatic development of Prakash children over a period of 4 years.

Publications from Cristin

No publications found

No publications found

No publications found

No publications found

Funding scheme:

FRIHUMSAM-Fri prosj.st. hum og sam