I investigate human multimodal communication. In particular, I am interested in how communicative intent and the pragmatics of conversation influence the interplay between bodily signals and speech, as well as how social cues and context may modulate the perception and communicativeness of co-speech gestures and other bodily signals and their integration with speech.
My recent projects use paradigms that embed multimodal language production and comprehension in more situated contexts with a special focus on the interplay between different communicative modalities, including speech, gesture, facial signals and eye gaze. In addition to investigating how these modalities interact in the process of semantic comprehension, I explore the role of the visual modalities in pragmatic communication, including turn-taking and social action formation and recognition.
I have recently been awarded an ERC Consolidator grant. This new project entitled “Communication in Action (CoAct): Towards a model of contextualised action and language processing”. The project investigates the role of visual communicative signals in situated, face-to-face language production and comprehension using a variety of novel techniques (including Virtual Reality and motion capture).
I am also interested in the cognitive representations underlying the production of gestures, in language evolution, the development of gestural communication (in childhood and ageing), and the use of gesture in the context of impaired (e.g. Parkinson’s) and verbally challenging (e.g. pain sensation) communication.
In addition to the European Research Council (Consolidator grant), my research has been supported with grants from the European Union Horizon Actions (Marie Curie Fellowship), Economic & Social Research Council (UK), Parkinson UK, The Leverhulme Trust (UK), the British Academy (UK), Volkswagen Stiftung (Germany) and the German Science Foundation (DFG, Mercator Fellowships).
I co-direct the Nijmegen Gesture Centre, together with Asli Özyürek.
Please also see my my research gate profile or my google scholar profile.
Affiliations
Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen(NL)
Max Planck Institute for Psycholinguistics, Nijmegen (NL)
Contact details Donders Centre for Cognition / Donders Institute
Email j.holler@donders.ru.nl | Visit personal page
Contact details MPI
Email judith.holler@mpi.nl | Visit personal page
CoSI lab 