Neural, muscular and articulatory encoding of speech

Faculty : Prasanta Kumar Ghosh, EE dept and Aditya Murthy, CNS dept.

Understanding how thoughts get converted to speech is a fundamental question in cognitive science.

Human speech is produced as a result of movement of several speech articulators including lips, tongue, jaw, velum, glottis and epiglottis creating time varying vocal tract shapes. It requires precise temporal coordination of these articulators. A tightly orchestrated sequence of muscle commands is, in turn, required in order to guide the articulators along elaborate paths within the oral cavity. The degree and nature of these muscle activations are determined by the planning and execution in the brain. While speech acoustic signal encodes both linguistic and paralinguistic information through spectro-temporal characteristics of different speech sounds, called phonemes, deriving invariant representations from

redundant speech signal remains a challenge. Unlike wideband speech signal, signals recorded from either brain or muscle activation or articulatory motion are believed to directly provide a parsimonious representation of the speech production process as they encode target sound specific information mimicking the speech planning more closely than complex acoustic signal. The research problem in this proposal is to quantify the information encoding process from simultaneously recorded neural signal, muscle activation signal and articulatory motion signal in conjunction with speech. This will require recording of EEG signal, EMG signal capturing facial muscle activations and EMA recording of articulatory motion while a subject speaks. Discovering the latent temporal patterns underlying the recorded modalities both separately & jointly and the transformations that occur between neural, muscle and articulatory patterns will be the objectives of this research. We anticipate that this knowledge will help us to better understand and treat speech disorders.

Description of background needed from the student for taking up this research:

In order to take up this research problem, a student is expected to be trained in both Engineering and Neuroscience courses including probability & random process, linear algebra, digital signal processing, system neuroscience, cognitive neuroscience, theoretical & computational neuroscience as well as advanced topics including speech signal processing, neural signal processing, machine learning for signal processing. As the research topic is primarily experimental, it is expected that a student taking this problem up will have the enthusiasm to plan, set up and carry out the required experiments with human subjects.