Last update: May 12, 2022


Grenoble, France

Organisers: GIPSA-lab (CNRS), LPNC lab (CNRS), IVèS, ULB.

Location: Room “Mont Blanc”, Building ENSE3, Domaine universitaire de Saint Martin d’Hères, Grenoble, France. Click here to see the access map.

Tuesday May 17 – Welcome drink

19:30 Welcome drink at Caffe Forte in the city center

4 Place de Lavalette, 38000 Grenoble, France

Wednesday May 18 – Tutorials for ESRs

Chair: Anne Vilain

8:30 Welcome and coffee

09h00 Audiovisual speech and sensory-motor interactions

Jean-Luc Schwartz

Introduction to principles and models of multimodal speech perceptual processing, focus on speech-reading – Introduction to auditory vs. motor vs. perceptuo-motor theories of speech communication

09h40 Introduction to speech production and phonetics

Anne Vilain

Articulatory and acoustic characterisation of speech sounds of the world’s languages – Guidelines for speech sounds analysis – Some principles of speech production development 

10h20 Coffee break

10h40 Somatosensory system

Takayuki Ito

The talk will introduce the orofacial somatosensory function in speech production and perception in studies using the method of dynamical mechanical perturbation, integrating behavioral and neuro-physiological data together with theoretical principles about the somatosensory function in speech

11h20 Speech technologies 

Denis Beautemps and Jérémy Huart

The different relay services of the IVèS Company will be presented. They allow the tele-communication between deaf and hearing people with the help of interpreters and cuers. Projects of automatization of some aspects of the service to allow extension will be also presented.

12h00 Lunch

13h30 Cued Speech

Cécile Colin

Why and when was CS implemented (a bit of history) – The principles underlying CS (very rapidly as probably already known) – Some words about CS production, cf anticipation of the hand with respect to the lips and then how perceivers take advantage of this anticipation – The impact of CS on speech perception (and speech in noise) and production and on speech-related abilities (reading, verbal short term memory, …) = the main part – Factors modulating hand-lips (and sound) integration – The cerebral bases of CS perception

14h10 Introduction to machine learning

Thomas Hueber

Basics of (deep) machine learning (and its application to automatic speech processing) – I will review the basics of machine learning with a special focus on deep learning. The goal is to present the main architectures of deep neural networks with a minimum of math involved in order to demystify some aspects of this very active research field. I will also briefly present some software tools powered by deep learning which can be taken (almost) « off the shelf » and used for different speech processing tasks (for analyzing, segmenting, decoding, or generating a speech signal).

14h50 Coffee break

15h15 – 17h15: Parallel hands on:

* Speech analysis, in relation with Anne Vilain’s tutorial; required equipment for the hands on: personal computer with Praat and Phon installed (open access).

* Deep learning, in relation with Thomas Hueber’s tutorial; required equipment for the hands on: personal computer + gmail address + basics on programming (i.e. Python, Matlab, R); the programs used in the session are written in Python.

Thursday May 19 – ESRs’ scientific achievements


8:30 Welcome and coffee

Work Package 1: Biological diversity in plasticity and adaptation

Chair: Andrej Kral

9h00 Automatic landmark localization in CT images using deep learning

Yifan Wang (ESR 2), Samuel John, Björn Lyxell, Holger Blume, Thomas Lenarz and Andrej Kral

9h25 Neural mechanisms of cross-modal plasticity in hearing impairment 

Niloofar Tavakoli (ESR 3), Rüdiger Land and Andrej Kral

9h50 Can vibrotactile speech information enhance intelligibility?

Alina Schulte (ESR 4), Jeremy Marozeau, Andrej Kral, Søren Riis and Hamish Innes-Brown

Work Package 2: Multimodality and optimisation of cognitive resources

Chair: Jean-Luc Schwartz

10h15 The effect Cued Speech (CS) perception on audiovisual speech processing with typically hearing adults who are naïve or experts towards CS

Cora Caron (ESR 5), Jean-Luc Schwartz, Coriandre Vilain, Clémence Bayard, Jacqueline Leybaert, Axelle Calcus and Cécile Colin

10h40 Coffee break

11h10 Auditory-somatosensory integration in speech perception in individuals with hearing-impairment

Monica Ashokumar (ESR 6), Jean-Luc Schwartz and Takayuki Ito

11h35 Speech-in-noise understanding and cognitive factors in children with normal hearing and hearing impairment

Lyan Porto (ESR 7), Jan Wouters and Astrid van Wieringen

12h00 Novel word detection and dynamic learning in cochlear implant simulated speech: a pilot study in Danish

Julia Chiossi (ESR 8), François Patou, Elaine Ng, Lone Percy-Smith, Kathleen Faulkner and Björn Lyxell

12h25 Lunch

14h00 Theory of mind, language and cognition in deaf and deafblind children (Usher syndrome) with cochlear implants 

Kristina Burum (ESR 9), Björn Lyxell and Simon Sundström

14h25 Multilingual data collection and cued speech generation

Sanjana Sankar (ESR 10), Denis Beautemps and Thomas Hueber

Work Package 3: Environment and enhancement of language skills

Chair: Cécile Colin

14h50 The acquisition of multiple spoken languages in deaf children with Cochlear implants who grow up in plurilingual contexts

Elettra Casellato (ESR 11), Ruth Swanwick and Edward Killan

15h15 Multimodal communication in the presence of sensory and communication asymmetries

Nathalie Czeke (ESR 12), Ruth Swanwick and Edward Killan

15h40 Coffee break

16h10 Picture written naming task to assess lexical orthography in French in children who are deaf and hard of hearing

Elodie Sabatier (ESR 13), Fabienne Chetail and Jacqueline Leybaert

16h35 Speech perception in children with cochlear implants: A study of Auditory Verbal Therapy and French Cued Speech

Lucie Van Bogaert (ESR 14), Laura Machart, Anne Vilain, Hélène Loevenbruck and Consortium EULALIES

17h00 The phonological body: Body movements to accompany speech perception and production of foreign speech sounds in a noisy environment. 

Marie Joe Kfoury (ESR 15), Axelle Calcus and Brigitte Charlier

17h25 – 18h00 General discussion of the ESR scientific achievements

20h00 Dinner at the restaurant “Les Jardins de Sainte Cécile” in the city center.

18 Rue de l’Alma, 38000 Grenoble, France

Friday May 20 – Joint discussions

Chair: Jean-Luc Schwartz and Sophie Antoine

8h30 Welcome and coffee

9h00-11h00 Round table on the potential use of Comm4CHILD developments

starting with:

Aïda Regel Poulsen (PAG): Making Ends Meet

In this presentation I shall focus to implementation of UN CRPD in teaching and choice of pedagogical methods to include HoH pupils 
  – human rights and responsibilities.

Leo De Raeve (FEAPDA): (missing) incidental learning by people with a severe to profound hearing loss

Although people with a severe hearing loss can come to good speech perception in good listening situations, they often miss a lot of information in more difficult listening environments, which also results in weak incidental learning. To come to incidental learning, you not only need good speech perception skills in difficult listening situations, but also good higher cognitive skills. For those with severe to profound hearing loss, it is hard to develop these skills, which impacts not only their speech perception and language development, but also their social-emotional functioning and behaviour.  The question for rehabilitation is: how can we increase their auditory skills, their higher cognitive skills or how can we compensate these skills (visual, tactile,…) ?

Pre-workshop materials:

* United Nations’ Convention on the Rights of People with Disabilities (click here to download)

* Documentary short film ‘Hear There & Everywhere’ on living with hearing loss (click here to watch)

* Short film about a 10 years old HoH girl in Denmark (link upon request to

11h00 Coffee break

11h30-12h30 Standard Comm4CHILD meetings 

12h30-13h00 Closing remarks

15h Climbing up the Bastille