Model-Based Hand Tracking with Texture, Shading and Self-occlusions

Add to your list(s) Download to your calendar using vCal

• Martin De La Gorce, Ecole des Ponts Paris Tech
• Thursday 10 December 2015, 10:00-11:00
• Auditorium, Microsoft Research Ltd, 21 Station Road, Cambridge, CB1 2FB.

If you have a question about this talk, please contact lecturescam.

Please be aware that this event may be recorded. Microsoft will own the copyright of any recording and reserves the right to distribute it as required.

I will first present a model-based approach to 3D hand tracking from monocular video. The 3D hand pose, the hand texture and the illuminant are dynamically estimated through minimisation of an objective function. Derived from an inverse problem formulation, the objective function enables explicit use of texture temporal continuity and shading information, while handling important self-occlusions and time-varying illumination. The minimisation is done efficiently using a quasi-Newton method, for which we propose a rigorous derivation of the objective function gradient. Particular attention is given to terms related to the change of visibility near self-occlusion boundaries that are neglected in existing formulations. In doing so we introduce new occlusion forces and show that using all gradient terms greatly improves the performance of the method. Experimental results demonstrate the potential of the formulation.

I will then briefly present a more recent work for the 3D reconstruction of a piecewise-planar watertight surface from range images, typically indoor laser scans with millions of points. We formulate the surface reconstruction problem as a integer programming problem where we assign a binary label to elements of a partition of the scene volume that is based on detected and hypothesised planes. Our major contributions is a regularisation of the reconstructed surface w.r.t. the length of edges and the number of corners. Compared to classical area-based regularisation, it better captures surface complexity and is therefore better suited for man-made environments, such as buildings. We approximately solve the integer program using a simple continuous relaxation. Experiments show that it is fast and reaches near-optimal solution.

References: M. de la Gorce, N. Paragios and David Fleet. Model-Based Hand Tracking with Texture, Shading and Self-occlusions. IEEE Conference in Computer Vision and Pattern Recognition (CVPR), Anchorage 2008 http://www.gigovery.com/research_page/publications/CVPR_2008_de_La_Gorce.pdf

Alexandre Boulch, Martin de La Gorce, Renaud Marlet Piecewise-Planar 3D Reconstruction with Edge and Corner Regularization. Comput. Graph. Forum http://onlinelibrary.wiley.com/doi/10.1111/cgf.12431/pdf

This talk is part of the Microsoft Research Cambridge, public talks series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Auditorium, Microsoft Research Ltd, 21 Station Road, Cambridge, CB1 2FB
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge talks
• Chris Davis' list
• Guy Emerson's list
• Interested Talks
• Microsoft Research Cambridge, public talks
• Optics for the Cloud
• PMRFPS's
• School of Technology
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• personal list
• rp587

Note that ex-directory lists are not shown.