Deep diffusion model-based dual structured light for large-format complex surface imaging

The problem

Phase-shifting structured light measures 3D shape with high accuracy by projecting sinusoidal fringes and reading their phase in a camera image. A single projector–camera unit, however, sees an object from only one angle. On large or complex surfaces, protrusions block the projected light or the camera’s view, leaving shadowed and blind regions without phase information, and the measurable area is limited. Measuring from several viewpoints, or turning the object on a turntable, fills the gaps but is slow and impossible for objects that cannot be moved.

Adding a second projector–camera head covers what the first cannot see, but only if the two heads can measure at the same time with overlapping views. When both projectors are on, each camera records the two fringe patterns superimposed in the overlap, an aliased pattern that leads to incorrect reconstruction. Earlier ways of separating such patterns need extra hardware (colour filters, special optical paths), colour or orientation multiplexing, or a strict projection order, and some give up simultaneous measurement altogether.

How it works

The system pairs two synchronised projector–camera heads with locally overlapping fields of view and runs them simultaneously. Instead of preventing the aliasing with hardware, it removes it in software: a conditional diffusion model learns to recover, from each aliased image, the clean fringe pattern of one projector. After that, each head works as a conventional structured-light unit, and the overlap is used to fuse the two results.

1. Two synchronised heads. Two projectors and two cameras are connected by synchronous control lines, so both heads project and capture at the same time. Each head measures part of the surface, and their fields of view overlap locally.
2. Aliasing as a sum. In the overlap a camera pixel integrates light from both projectors, so its intensity is the sum of the two projected patterns’ contributions. Writing u for the pixel and l_n for the pattern of projector n, I(u) = I(u; l₁) + I(u; l₂) (background light is assumed not to affect the phase). One head’s pattern is therefore the measured intensity minus the other head’s pattern, I(u; l_n) = I(u) − I(u; l_3−n), which the authors read as a denoising problem, with the other head’s pattern as the noise.
3. Conditional diffusion recovers the clean fringe. A U-Net with self-attention is trained in a conditional denoising diffusion framework. Training pairs come from the system itself: one head on alone gives the clean pattern x₀, both heads on give the aliased pattern y, which serves as the condition. At inference the model starts from Gaussian noise and denoises it step by step, guided by y, until it reaches the dealiased fringe.
4. Phase and 3D per head. With clean fringes, the four-step phase-shifting algorithm gives the wrapped phase and multi-frequency heterodyne unwrapping the absolute phase. Triangulation with each head’s calibrated projector and camera parameters then turns it into a partial 3D point cloud.
5. Fusion with ICP. The overlapping region provides corresponding points, and the iterative closest point (ICP) algorithm estimates the rotation and translation that merge the two partial point clouds into the complete surface.

Results

The prototype uses two TJ20U2 projectors and two CMOS cameras with a resolution of 3072 × 2048. Fringe patterns with periods of 100, 110, 120 and 130, each in four phase steps of 0.5π and with varied initial phase, produced 4043 training and 1583 test samples of 512 × 512 pixels; the model was trained with T = 4000 diffusion steps for 800 epochs. Without dealiasing, fringe aliasing introduces significant errors in both heads’ point clouds and in the fused result; with the diffusion model, the object is reconstructed successfully. An earlier derivative-based recovery method, applied to the same aliased fringes, distorts the fringes (noticeably denser, with local intensity amplification) and gives a noisy point cloud with pronounced layering. On a prism chosen to accentuate occlusion, fusion widened the reconstruction range and filled sparse areas that a single head leaves.

The authors define the scope of the evaluation as truly simultaneous dual-head measurement with a local overlap. Other aliasing-removal methods depend on specific hardware or projection schemes, such as colour or orientation multiplexing, optical-path separation or controlled projection sequences, and could not be run on this configuration without changing it. Results are therefore compared with non-aliased single-projector measurements as the reference and with conventional phase shifting applied directly to the aliased fringes as the baseline.