3D reconstruction is the process of capturing the 3D shape of a real world object. The algorithm may take as input one or more depth maps of the object, possibly from different angles. Alternatively, the algorithm may be provided only with 2D images or 2D video and asked to infer the depth coordinate. The algorithm must then generate a smooth and accurate representation of the 3D object. 3D facial reconstruction is especially difficult because humans have evolved to detect even very slight differences in facial structure.
3D reconstruction is an active field of research with many applications in computer graphics and animation, medical imaging, robotics, landscape and vegetation scanning (including LiDAR), facial recognition and cybersecurity.
Many difficult and interesting mathematical problems arise when trying to capture digital representions of 3D objects. Firstly, the process of 3D scanning itself gives rise to a number of issues including dealing with sensor accuracy limitations and noise, including reflective or transparent surfaces. Some existing solutions require very expensive scanners, and thus there is scope for researching solutions that use only inexpensive and widely available equipment, such as a smartphone.
Once a depth map has been captured there is the problem of deciding how to combine multiple depth maps or images into a single coherent representation, and how to interpolate between data points in the point cloud to yield a smooth and accurate representation of the original object. Furthermore, algorithms are often computationally intensive which renders them useless for real time applications. Designing efficient yet effective algorithms for real time 3D reconstruction is therefore an even more difficult problem.
Computer vision researchers at the University of Nottingham have developed an algorithm which uses machine learning / AI (in this case, a neural network) to reconstruct a 3D face model from a single 2D photograph.
Another research team have created accurate facial reconstruction technology using video captured from a smart phone.
Interested in developing 3D reconstruction technology for your industry? Our research consultants are excited to collaborate with you to develop 3D vision algorithms for all your applications.