A facility was developed to record synchronized streams of audio, video, , and data for the purposes of studying speech. The exact combination and configuration of data streams collected can vary as a function of the experimental question. This work was part of a collaborative project by researchers at the (L.E. Bernstein, ) and (, , J. Jiang).
(Work supported by KDI Grant 9996088.)
In the video clip (kdidemo1.mpg 168k mpeg), 3D motion and EMA are shown being recorded. The talker is saying /ba/.
Pattern Playback (points)
The 3D time series data collected with the system can be used to synthesize point-light displays. In this example, the points correspond to markers placed around a talker's lips.
In these video clips (kdidemo2.mpg 128k mpeg and kdidemo3.mpg 452k mpeg), the motion of the eight points is being controlled by the 3D time series data from eight reflectors loctated around a talker's lips. The talker is saying /ba/. The second clip (kdidemo3.mpg 452k mpeg) is the full motion capture data set (recorded at 120Hz). Video playback is 30Hz, resulting in a presentation rate of one quarter real-time. (Note: The motion tracks used in this video clip were not from the previous video clip demonstrating data acquisition.)
Pattern Playback (lip model)
The same 3D time series data can also be used to drive the motion of a 3D model (D. Tee,1999).
In these video clips (kdidemo4.mpg 146k mpeg and kdidemo5.mpg 519k mpeg), the motion of our 3D lip model is being controlled by motion data from a talker saying /ba/. The second clip (kdidemo5.mpg 519k mpeg) is the full motion-capture data set (recorded at 120Hz). Video playback is 30Hz, resulting in a presentation rate of one quarter real-time.
(Work supported by KDI Grant 9996088.)
In these video clips (
In these video clips (