Multi-View Geometry and Computational Photography using Non-Classical Cameras

Abstract

This tutorial is meant as an introduction to the design, modeling and implementation of non-classical (multi-perspective) cameras for several computer vision and computational photography applications. The tutorial will provide an overall view of developing a complete system (capture, modeling, and synthesis/reconstruction) as well as provide sufficient details for calibration and modeling such non-central cameras. We hope to provide enough fundamentals to satisfy the technical specialist as well as tools/software to aid graphics and vision researchers, including graduate students.

Course description

A perspective camera captures the visual information in the scene from a single viewpoint. In contrast, a multi-perspective camera combines visual information from several viewpoints into a single image. Such cameras offer new forms of sampling the visual world. Multi-perspective cameras have been used for several applications such as wide-angle imaging and 3D reconstruction, capturing light fields, rendering novel/un-conventional views and digital refocusing.

In this tutorial, we provide a practical guide on designing, modeling and implementation of multi-perspective cameras for several computer vision and computational photography applications. We will discuss several such cameras including classical omnidirectional cameras, non-central catadioptric cameras using mirrors, wide-angle cameras using refractive spheres and light field cameras using masks and lenslets.

Accurate physics based modeling of such cameras is necessary for them to be useful. We will discuss analytical forward projection models as well as calibration algorithms that allow precise 3D reconstruction and will discuss artifacts when using a central approximation. We will also give useful pointers to the use of algebraic geometry tools like Groebner Basis functions for designing and solving equations arising from non-classical geometry problems.

Course Slides in pdf

Section 1. Introduction

Section 2. Modeling of Non-Central Cameras

Section 3. Calibration

Section 4. Synthesis and Rendering

Section 5. Future Directions (Unsolved problems)

Zip file containing all Videos used in Tutorial

Back to my homepage