In this talk, we will introduce a fundamental computational tool, namely TILT, for extracting rich low-rank structures in images and videos, respectively. TILT utilizes the same transformed Robust PCA model for the visual data: Dt = A + E, and exploit modern high-dimensional convex optimization to extract the low-rank structures A from the visual data D, despite image domain transformation t and sparse corruptions E. We will show how this seemingly simple tool can help unleash tremendous information in images and videos that we used to struggle to get. We believe these new tools will bring disruptive changes to many challenging tasks in computer vision and image processing, including feature extraction, image correspondence or alignment, camera calibration 3D reconstruction, and object recognition, etc.

This is joint work with John Wright of Columbia, Emmanuel Candes of Stanford, Zhouchen Lin of MSRA, and my students Zhengdong Zhang, Xiao Liang of Tsinghua University, and Arvind Ganesh of UIUC.