Description of our facility for simultaneous recordings of acoustic, optical (3-D motion and video), and physiological speech data. This facility was developed to collect data for the project titled "KDI - Segmental and Prosodic Optical Phonetics for Human and Machine Speech Processing (NSF)."

Using our recorded speech data as a drive signal, we have developed tools to synthesize visual speech tokens with a variety of representations. 

We have also recorded and synthesized point-light non-speech face movements for study in our experiments.

 
We have conducted a McGurk experiment in which electrophysiological measurements were obtained.

 
Viewing the speaker's face enhances speech detection in noisy environments. We are investigating whether the effects of the face can be replicated with a non-speech visual stimulus.