House Ear Institute logo
sitemap
about newsroom research education children services support
departments scientists conferences administration Technology Transfer
...research
Communication Neuroscience
KDI DEMOS

 
Data Acquisition
A facility was developed to record synchronized streams of audio, video, 3D motion (Qualisys(tm)), and electromagnetic midsaggital articulography (EMA) data for the purposes of studying speech. The exact combination and configuration of data streams collected can vary as a function of the experimental question. This work was part of a collaborative project by researchers at the House Ear Institute (L.E. Bernstein, E.T. Auer) and UCLA (A. Alwan, P. Keating, J. Jiang). 

Motion Capture Data Acquisition(Work supported by National Science Foundation KDI Grant 9996088.)

In the video clip (kdidemo1.mpg 168k mpeg), 3D motion and EMA are shown being recorded. The talker is saying /ba/.
Pattern Playback
(points)
The 3D time series data collected with the Qualisys(tm) motion capture system can be used to synthesize point-light displays.  In this example, the points correspond to markers placed around a talker's lips.

Motion CaptureIn these video clips (kdidemo2.mpg 128k mpeg and kdidemo3.mpg 452k mpeg), the motion of the eight points is being controlled by the 3D time series data from eight reflectors loctated around a talker's lips. The talker is saying /ba/. The second clip (kdidemo3.mpg 452k mpeg) is the full motion capture data set (recorded at 120Hz). Video playback is 30Hz, resulting in a presentation rate of one quarter real-time.
(Note: The motion tracks used in this video clip were not from the previous video clip demonstrating data acquisition.)
Pattern Playback
(lip model)
The same 3D time series data can also be used to drive the motion of a 3D model (D. Tee,1999).

Model Control PointsIn these video clips (kdidemo4.mpg 146k mpeg and kdidemo5.mpg 519k mpeg), the motion of our 3D lip model is being controlled by motion data from a talker saying /ba/. The second clip (kdidemo5.mpg 519k mpeg) is the full motion-capture data set (recorded at 120Hz). Video playback is 30Hz, resulting in a presentation rate of one quarter real-time.