Home

ACCV 2007 · Oral presentation Computer vision

Detecting and Segmenting Un-Occluded Items by Actively Casting Shadows

Tze Koh, Amit Agrawal, Ramesh Raskar, Steve Morgan, Nicholas Miles, and Barrie Hayes-Gill

Scene of overlapping items and a binary segmentation mask identifying the un-occluded items
From an overlapping arrangement of objects to a mask of the un-occluded items, using shadow and silhouette information.

The idea

Use shadows to reveal
what lies in front.

With illumination from different directions near the camera, un-occluded items cast shadows on other items or the background while remaining unshadowed themselves. These shadow relationships let us identify and segment items without first estimating a depth map.

Active illumination
Multiple images with changing light direction
Occlusion cues
Use cast shadows and object silhouettes
Texture-independent cues
Ignore texture edges when determining occlusions
No depth map required
Segment items directly from illumination changes
01

Segmentation
examples

Original scenes appear on the left, with the extracted un-occluded items on the right. Examples include overlapping objects, colorful cutouts, and crayons.

Three scene-and-segmentation comparisons showing overlapping objects, colorful animal cutouts, and crayons, with the un-occluded items extracted
Real-scene examples of shadow-based segmentation. Items are objects, or parts of objects, enclosed by depth edges.
02

Abstract

We present a simple and practical approach for segmenting un-occluded items in a scene using cast shadows. By ‘items’, we refer to objects (or part of objects) enclosed by depth edges. Our approach utilizes the fact that under varying illumination, un-occluded items will cast shadows on occluded items or background, but will not be shadowed themselves.

We employ an active illumination approach by taking multiple images under different illumination directions, with illumination source close to the camera. Our approach ignores the texture edges in the scene and uses only the shadow and silhouette information to determine the occlusions. We show that such a segmentation does not require the estimation of a depth map, which can be cumbersome, expensive and often fails due to the lack of texture and presence of specular objects in the scene.

Our approach can handle complex scenes with self-shadows and specularities. Results on several real scenes along with the analysis of failure cases are presented.

03

Ramesh Raskar, Kar-han Tan, Rogerio Feris, Jingyi Yu, and Matthew Turk, Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering using Multi-Flash Imaging.