Being able to recover the shape of 3D deformable surfaces from a single video stream would make it possible to field reconstruction systems that run on widely available hardware without requiring specialized devices. However, because many different 3D shapes can have virtually the same projection, such monocular shape recovery is inherently ambiguous. In this survey, we will review the two main classes of techniques that have proved most effective so far: The template-based methods that rely on establishing correspondences with a reference image in which the shape is already known, and non-rigid structure-from-motion techniques that exploit points tracked across the sequences to reconstruct a completely unknown shape. In both cases, we will formalize the approach, discuss its inherent ambiguities, and present the practical solutions that have been proposed to resolve them. To conclude, we will suggest directions for future research.
Table of Contents
Early Approaches to Non-Rigid Reconstruction
Formalizing Template-Based Reconstruction
Performing Template-Based Reconstruction
Formalizing Non-Rigid Structure from Motion
Performing Non-Rigid Structure from Motion
About the Author(s)Matthieu Salzmann
, Toyota Technological Institute at Chicago
Mathieu Salzmann received his B.Sc and M.Sc degrees in computer science in 2004 from EPFL (Swiss Federal Institute of Technology). He obtained his PhD degree in computer vision in 2009 from EPFL. He then joined the International Computer Science Institute and the EECS departement at UC Berkeley as a postdoctoral fellow. Recently, he joined TTI Chicago as a Research Assistant Professor. His research interests include non-rigid shape recovery, human pose estimation, and optimization techniques for computer vision.Pascal Fua
, Swiss Federal Institute of Technology
Pascal Fua received an engineering degree from Ecole Polytechnique, Paris, in 1984 and the Ph.D. degree in Computer Science from the University of Orsay in 1989. He joined EPFL (Swiss Federal Institute of Technology) in 1996 where he is now a Professor in the School of Computer and Communication Science. Before that, he worked at SRI International and at INRIA Sophia-Antipolis as a Computer Scientist. His research interests include shape modeling and motion recovery from images, analysis of microscopy images, and Augmented Reality. He has (co)authored over 150 publications in refereed journals and conferences. He has been an associate editor of IEEE journal transactions for Pattern Analysis and Machine Intelligence and has often been a program committee member, area chair, and program chair of major vision conferences.