Building site inspection supported by AR

In the business world, approved and certified products are central for success. One of DNV GL's businesses is product certification, including the certification of the development of ships ranging from design to operation. BUISAR supports a particular stage of this. While the ship is under construction, DNV-GL's experts need to visit the shipyard to investigate whether the ship is built according to the certified design specification. So far, this does typically require manual measurements using laser meters or classical meter, and compare measurements with design papers. Naturally, this process requires a lot of walking and climbing by the expert, and patience to perform accurate measurements. BUISAR's goal is to reduce effort that goes into accurate measuring. A combination of reconstruction methods from the research fields Computer Vision and Photogrammetry should allow the automated creation of a 3D model of the ship that is under construction. By placing a known object into this scene, all distances in this model of the real ship can be expressed in centimeters. The expert can compare this model of the real ship with the original design model to find out whether the size and placement of all ship components are in accordance with the certified design. However, before deviations can be detected easily, the model and design must be perfectly aligned with each other, and BUISAR explores how to do this. After this, sizes, distances and deviations from the approved design can be shown in AR glasses worn by inspectors. To make sense of such numbers, the information in the inspector's view must be placed accurately and correctly according to the location in the real ship. For that, BUISAR must know exactly where in space the AR glasses are at all times. Achieving this is the third research challenge of the BUISAR project.

One of the main businesses of project partner DNV-GL is the certification of ships, and it is this use case from which the ideas of the BUISAR project originate, although the expected results will be more widely applicable. In the future, AR technology will support DNV-GL's assessors in their task of evaluating whether ships (and their components and sub-components) are built according to rules and approved specifications. The current state-of-the-art in visual reconstruction promises that the alignment of point clouds (3D-3D registration), which is an essential requirement of such AR inspection systems, is a theoretically solved problem that requires only performance improvements. However, experiments that we conducted both in the BUISAR pre-project and in the EU project LADIO show that this is not the case. Registration of isolated objects is very successful, registration of large environments at unknown scales or objects in an environment, does usually fail without intense user intervention. The project will therefore create a ground truth dataset that will allow researchers to validate their 3D-3D registration techniques. The dataset will be used to assess and report the registration quality of the algorithms that we discovered in the pre-project.