A long-standing research problem in computer graphics is to reproduce the
visual experience of walking through a large photorealistic environment
interactively. On one hand, traditional geometry-based rendering systems fall
short of simulating the visual realism of a complex environment. On the other
hand, image-based rendering systems have to date been unable to capture and
store a sampled representation of a large environment with complex lighting and
visibility effects.
We present a "Sea of Images", a practical approach to dense sampling,
storage, and reconstruction of the plenoptic function in large, complex indoor
environments. We use a motorized cart to capture omnidirectional images every
few inches on a eye-height plane throughout an environment. The captured images
are compressed and stored in a multi-resolution hierarchy suitable for real-time
pre-fetching during an interactive walkthrough. Later, novel images are
reconstructed for a simulated observer by re-sampling nearby captured images.
Our system acquires 15,254 images over 1,050 square feet at an average image
spacing of 1.5 inches. The average capture and processing time is 7 hours. We
demonstrate realistic walkthroughs of real-world environments reproducing
specular reflections and occlusion effects while rendering 15-25 frames per
second.
Sea of Images (186MB .avi or 24MB .avi)
Feature Globalization (166MB .avi or 15MB .avi)
Plenoptic Stitching (158MB .avi or 13MB .avi)
Princeton Graphics Group Projects Page
