Simulation and validation of underwater scenes for two-media optical 3D reconstruction

Abstract

Optical 3D reconstruction in environments with complex light paths such as water-air surface interactions continues to be challenging, particularly due to the inherent refraction effects. These effects compromise assumptions taken in standard photogrammetric methods like the traditional Multi-View Stereo and newer approaches like Neural Radiance Fields (NeRFs). Addressing these limitations is critical for monitoring coastal and riparian ecosystems, for flood-risk modeling, as climate change intensifies river flooding, and in general to satisfy increasing demands for 3D topo-bathymetric data. To evaluate models explicitly built to consider a change in refractivity along the image ray, simulation can be employed. In this study, we present a simulation and validation framework designed to investigate these challenges by synthesizing controlled water scenes with artificial camera trajectories and evaluating them with 2D and 3D (Cloud-to-Mesh, completeness) metrics. For that, a total of 130 images with a resolution of 1024 × 768 pixels were simulated to model both a water-free scene and a submerged scene. The results indicate that refractive effects must be explicitly accounted for, as a water depth of 3.5 m led to errors on the order of 1 m, when refraction was not taken into account. Furthermore, NeRFs proved to be particularly well suited for 3D analysis of photo-bathymetric surveys, achieving a completeness that was 21 % higher than conventional MVS methods. The simulation workflow is particularly beneficial for the development and testing of specialized NeRF-variants designed to better account for the complexities introduced by refraction at air-water interface.