Underwater structured-light 3D imaging method based on FP-DiffNet

The problem

Fringe-projection profilometry measures shape by projecting sinusoidal fringes onto an object and reading the phase of the fringes the camera sees. Its methods and accuracy were established with air as the medium. Under water, several effects act at once. Absorption and scattering attenuate the signal with distance. Small-angle forward scattering from suspended particles blurs the fringes, large-angle backscattering adds background light and noise, and refraction at the water–air interface shifts the light paths. The fringes lose contrast and blur, so phase retrieval accumulates errors and the point cloud can lose regions or be distorted.

Restoration methods such as polarization or stereo imaging rely on prior modelling of the water’s optical parameters, while enhancement methods such as histogram equalization or colour correction can introduce colour bias and extra noise. Deep networks trained end-to-end avoid the priors but act as black boxes without explicit physical constraints, which can distort the restored fringes and bias high-precision phase retrieval.

How it works

FP-DiffNet is a fringe-restoration front end placed between acquisition and phase retrieval. It treats restoration as a conditional denoising diffusion process (DDPM): a U-Net learns to turn noise into a clean fringe image, guided by the degraded underwater image, and a simplified physical model of the degradation keeps the result consistent with underwater light transport. The restored fringes then go through the usual phase-shifting pipeline.

1. Model the degradation. Following the Jaffe–McGlamery model, the light reaching the camera splits into direct transmission (attenuated with distance), forward scattering (a Gaussian blur of the image) and backscattering (background light and noise). Refraction at the water–air interface adds a depth-dependent shift of the rays.
2. Train a conditional diffusion model. High-SNR fringe images captured in air are the targets, and fringe images of the same scenes captured in turbid water are the conditions. A clean image is noised towards Gaussian noise over T = 1000 diffusion steps, and a U-Net with self-attention learns to predict the injected noise from the noisy image, the step index and the underwater image. The loss combines an image-similarity term (SSIM), the noise-prediction error and a physics-consistency term, plus an L1 sparsity penalty on the weights.
3. Restore by reverse diffusion. Sampling starts from pure Gaussian noise and removes noise one step at a time, with the underwater observation fed to the network at every step, until a clean fringe image remains.
4. Correct with physics. A simplified degradation operator, a Gaussian blur for forward scattering plus a noise term for backscattering, links the estimate to the observation. Every k = 50 reverse steps, a gradient step with strength α = 0.01 reduces the residual between the observation and the simulated degradation of the estimate. Refraction is not in this operator; the U-Net learns it from the data.
5. Retrieve phase and shape. The restored images have the form of standard N-step phase-shifting fringes, so phase retrieval runs unchanged: the arctangent formula gives the wrapped phase, unwrapping gives the absolute phase, and its difference from a reference plane is converted into 3D coordinates with the calibrated projector–camera geometry.

Results

The system pairs an industrial camera (1800 × 1600 pixels) with a digital projector (1280 × 1024 pixels) mounted about 0.6 m above an acrylic tank, using a three-frequency four-step phase-shifting scheme. Turbidity was raised by adding 4, 5, 6 and 7 mL of milk to 13 L of clear water, giving four levels T1–T4. The dataset covers five objects in random poses, split 4:1 into 7200 training and 1800 test images, and all reported results are on the test set. Errors are measured against ground-truth fringe images, phases and reconstructions. From T1 to T4, PSNR falls only from 43.02 to 42.42 dB, and the cosine similarity of the wrapped phase to the reference stays between 0.9951 and 0.9962. The remaining phase errors concentrate at object boundaries, shadows and phase discontinuities. Restoring one 512 × 512 fringe image takes about 430 ms on an NVIDIA GeForce RTX 4090.

The authors state the limits. T4 is the most turbid level tested; beyond it they expect restoration to fail once direct transmission falls below the noise floor and the fringe information is lost. The framework applies to sinusoidal phase-shifting patterns, and other structured-light configurations or underwater conditions may need fine-tuning with configuration-specific data.