Deep transformer-enhanced dynamic structured light for three-dimensional moving object imaging

This paper addresses a core limitation of phase-shifting structured light: when the object moves while multiple fringe patterns are being projected and captured, the phase shifts are no longer the intended fixed values. The resulting motion-induced errors can produce ripples, phase-unwrapping mistakes, and inaccurate 3D shape recovery.

Problem setting

Conventional phase-shifting profilometry (PSP) assumes a static scene during the projection sequence. In dynamic scenes, each surface point can move both in the image plane and along the depth direction, causing different additional phase shifts at different pixels. Earlier compensation methods often assume simplified motion, cancel only part of the phase error, or require extra projected patterns.

The key idea of this work is to track pixel correspondences directly in fringe images and use those correspondences inside the phase-error model. This allows local, pixel-wise compensation instead of a single global motion correction.

Motion and phase model

The paper decomposes 3D motion into in-plane displacement in the image and out-of-plane displacement along depth. Both components change the observed fringe phase. By modeling these changes over a short sliding window of fringe images, the method estimates the unknown phase error and recovers corrected phase values for each pixel.

Motion model. Object motion changes the pixel correspondence between projected fringe patterns and introduces local phase-shift errors; compensation recovers a more faithful 3D surface.

Transformer-based correspondence

The correspondence module is based on COTR, a transformer-style matching network. It is fine-tuned on fringe-image pairs, using supervised point correspondences obtained from blank-image optical-flow estimates. Given pre-motion and post-motion fringe images, the model predicts where each target pixel moves, enabling sub-pixel matching under structured-light illumination.

COTR-based matching. The network tracks corresponding pixels between fringe frames, supplying the motion information needed for phase-error estimation.

Validation

The validation experiments first use a ceramic plate and standard objects to compare against PSP and previous compensation methods. The proposed method reduces ripple artifacts and better matches ground-truth profiles, especially under larger motion amplitudes where uncompensated PSP visibly fails.

Validation. Extracted figures show motion maps, reconstructed surfaces, error maps, and profile comparisons against PSP and previous compensation methods.

Dynamic scenes

The method is further evaluated on complex motion: multi-object scenes, rotating statues, rotating fans with weak texture, and continuously changing rotating targets. These cases test local motion differences, smooth low-texture surfaces, and motion trajectories that are not well described by a single global translation.

Complex scenes. The method estimates motion-induced phase errors and improves reconstruction quality for rotating, multi-object, and weak-texture targets.

Robustness and limitation

The experiments show that transformer correspondence plus phase compensation can substantially reduce dynamic artifacts compared with direct PSP. The paper also notes that very large out-of-plane motion can still create difficult phase-unwrapping cases, suggesting future combinations with geometric phase unwrapping or Gray-code-style constraints.

Continuous motion. Extracted result figures show how the compensated pipeline behaves across rotating scenes and complex motion cases.

Takeaway

The practical value is a dynamic structured-light pipeline that does not require the target to remain stationary during multi-shot fringe acquisition. By bringing transformer correspondence into the physics of PSP phase compensation, the method improves 3D measurement for moving industrial parts, rotating objects, and other scenes where stopping the object is impractical.