Body Part Recognition: Making Kinect Robust

Add to your list(s) Download to your calendar using vCal

• Jamie Shotton, MSR Cambridge
• Wednesday 01 June 2011, 14:15-15:15
• Lecture Theatre 1, Computer Laboratory.

If you have a question about this talk, please contact Stephen Clark.

Last November, Microsoft launched Xbox Kinect (http://www.xbox.com/kinect), a revolution in gaming where your whole body becomes the controller—you need not hold any device or wear anything special. Human pose estimation has long been a “grand challenge” of computer vision, and Kinect is the first product that meets the speed, cost, accuracy, and robustness requirements to take pose estimation out of the lab and into millions of living rooms.

In this talk we will discuss some of the challenges of pose estimation and the technology behind Kinect, detailing our new approach which forms one of the core algorithms inside Kinect: body part recognition. Deriving from our earlier work that uses machine learning to recognize categories of objects in photographs, body part recognition uses a classifier to produce an interpretation of pixels coming from the Kinect depth-sensing camera into different parts of the body: head, left hand, right knee, etc. Estimating this pixel-wise classification is extremely efficient, as each pixel can be processed independently on the GPU . The classifications can then be pooled across pixels to produce hypotheses of 3D body joint positions for use by any suitable skeletal tracking algorithm. Our approach has been designed to be robust, in two ways in particular. Firstly, we train the system with a vast and highly varied training set of synthetic images to ensure the system works for all ages, body shapes & sizes, clothing and hair styles. Secondly, the recognition does not rely on any temporal information, and this ensures that the system can initialize from arbitrary poses and prevents catastrophic loss of track, enabling extended gameplay for the first time. We further discuss the huge promise this technology holds for many other applications.

This talk is part of the Wednesday Seminars - Department of Computer Science and Technology series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• Graduate-Seminars
• Guy Emerson's list
• Interested Talks
• Lecture Theatre 1, Computer Laboratory
• School of Technology
• Trust & Technology Initiative - interesting events
• Wednesday Seminars - Department of Computer Science and Technology
• bld31
• computer science
• se393's list

Note that ex-directory lists are not shown.