Occupancy Networks: Learning 3D Reconstruction in Function Space
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, Andreas Geiger | CVPR 2019
In a Nutshell 🥜
Existing representation (i.e., voxel volumes, point clouds, meshes) all have the problem of discretising the 3D space. Mescheder et al.1 suggests a novel method of encoding a 3D representation as a continuous function to circumvent the problem.
Formally, the function is as the following:
where f is a function of a random point in space and a feature. The output is a probability between 0 to 1 that suggest the confidence that such point in space is occupied by an object or not. Essentially, the function becomes a binary classifier of any points in 3D space that can be learnt via a neural network. The feature can be anything from an image to a pre-existing 3D representation such as a point cloud.
During inference, we perform marching cube algorithm and continuously find the occupancy confidence via the learnt function until we achieve the resolution we want.
The results show that the new type of 3D representation can be highly helpful in tasks such as 3D reconstruction from single images.
Some Thoughts ðŸ’
Implicit 3D representations have become a recent trend in computer vision, as they have a fixed space for unlimited resolution.
The trade-off of such approach will be inference time, which will rise gradually when we want to achieve higher and higher resolutions.
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019). Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4460-4470).