Rudolf Maarten Bolle

Absolute head pose estimation from overhead wide-angle cameras

Most surveillance cameras have a wide-angle field of view and are situated unobtrusively at overhead positions. For this type of application, head pose estimation is very challenging because of the limitations of the quality and resolution of the incoming data. In addition, even though the absolute head pose is constant, the head pose in camera view changes depending upon the location of head with respect the camera. We present a solution to estimate absolute coarse head pose for wide-angle overhead cameras by integrating 3D head position and pose information. The work involves image-based learning, pose correction based on 3D position, and real-time multicamera integration of low-resolution imagery. The system can be applied to an active face catalogger to obtain the best view of the face for surveillance, to customer relationship management to record behavior in retail stores or to virtual reality as an input device.

Security, privacy, and health

Much of the current pervasive computing research concentrates on devices and the communication between them. However, an important aspect of pervasive devices is their interface with the physical world—particularly, how they acquire information about their users. One rich medium (and the dominant one through which people receive information) is vision. As such, we expect future pervasive computing environments to depend on vision for passive perception of people. The PeopleVision project at the IBM T.J.Watson Research Center is tackling this problem, focusing on the privacy issues involved in such visual information gathering, both in pervasive computing environments and in video surveillance systems. The PeopleVision system takes an object-oriented approach to video. It understands the video stream, decomposing it into people, objects, and areas of interest. It then abstracts or selectively re-renders this information based on the intended user. Client processes receive abstract information about people in the environment according to issued requests. Access control lists that can grant privileged processes access to richer (and more intrusive) information verify these requests. For example, the list might grant a face recognition security system access to facial images but tell the air conditioning process only how many people are in each room. Access control lists also govern the information delivered to security guards, supervisors, and ordinary users. Re-rendering delivers reconstructed video, which preserves some objects unchanged and blanks out other areas of the image or replaces the area with a computergraphics rendering that preserves relevant information. However, it does not convey more privacy-sensitive details. During ordinary use, a guard may only view silhouettes of people in the surveillance area, hiding irrelevant but privacy-sensitive information such as race, gender, and appearance. We have developed a privacy camera—a single device combining a camera and processor that implements video-understanding algorithms. With this device, we can ensure that the privacyintruding video is never available or only leaves the device in an encrypted form. All of the processed data leaving the device can also be encrypted, ensuring maximum privacy protection for the people in the pervasive computing environment. For more information, contact Andrew Senior at IBM T.J. Watson Research Center, PO Box 704 Yorktown Heights, NY 10598-0218; aws@watson.ibm.com.