High-presence, low-bandwidth, apparent 3D video-conferencing with a single camera

Small digital video cameras have become increasingly common, appearing on portable consumer devices such as cellular phones. The widespread use of video-conferencing, however, is limited in part by the lack of bandwidth available on such devices. Also, videoconferencing can produce feelings of discomfort in conversants due to a lack of co-presence. Current techniques to increase co-presence are not practical in the consumer market due to the costly and elaborate equipment required (such as stereoscopic displays and multicamera arrays). To address these issues, this paper describes a real-time, full frame-rate video-conferencing system that provides simulated threedimensionality via motion parallax in order to achieve a higher level of co-presence. The system uses a deformable 3D face model to track and re-synthesize each user's face using only a single monocular camera, so that only the (few tens of) parameters of the model need be transferred for each frame. This both provides motiontracking for the simulated 3D experience and reduces the bandwidth requirements of the video-conference to the order of a few hundred bytes per frame. Bandwidth and processor usage for the algorithms are reported. Possible implications and applications of this technology are alsodiscussed.