We approach the problem of online appearance reconstruction from RGB-D images with our novel DeepSurfel representation that leverages the implicit grid representation to support flexible topologies and 2D surface-aligned feature maps to encode high-frequency appearance information and improve scalability to large scenes.
We further present an end-to-end trainable online appearance fusion pipeline that fuses information provided by RGB images into the proposed scene representation and is trained using self-supervision imposed by the reprojection error with respect to the input images.
DeepSurfel representation is well-suited for online updates of appearance information and is compatible with classical texture mapping as well as learning-based approaches.
