PrincetonComputer SciencePIXL GroupPublications → [Shi et al. 2018] Local Access
PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction

European Conference on Computer Vision (ECCV) oral presentation, September 2018

Yifei Shi, Kai Xu, Matthias Nießner,
Szymon Rusinkiewicz, Thomas Funkhouser
Abstract

We introduce a novel RGB-D patch descriptor designed for detecting coplanar surfaces in SLAM reconstruction. The core of our method is a deep convolutional neural network that takes in RGB, depth, and normal information of a planar patch in an image and outputs a descriptor that can be used to find coplanar patches from other images. We train the network on 10 million triplets of coplanar and non-coplanar patches, and evaluate on a new coplanarity benchmark created from commodity RGB-D scans. Experiments show that our learned descriptor outperforms alternatives extended for this new task by a significant margin. In addition, we demonstrate the benefits of coplanarity matching in a robust RGBD reconstruction formulation. We find that coplanarity constraints detected with our method are sufficient to get reconstruction results comparable to state-of-the-art frameworks on most scenes, but outperform other methods on established benchmarks when combined with traditional keypoint matching.
Link
Citation

Yifei Shi, Kai Xu, Matthias Nießner, Szymon Rusinkiewicz, and Thomas Funkhouser.
"PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction."
European Conference on Computer Vision (ECCV) oral presentation, September 2018.

BibTeX

@inproceedings{Shi:2018:PPC,
   author = "Yifei Shi and Kai Xu and Matthias Nie{\ss}ner and Szymon Rusinkiewicz
      and Thomas Funkhouser",
   title = "{PlaneMatch}: Patch Coplanarity Prediction for Robust {RGB-D}
      Reconstruction",
   booktitle = "European Conference on Computer Vision (ECCV) oral presentation",
   year = "2018",
   month = sep
}