3D reconstruction of the moving object with double-shooting based on phase shifting profilometry

The problem

Phase-shifting profilometry (PSP) is one of the most commonly used 3D measurement techniques, valued for its high accuracy, robustness and full-field measurement. A projector casts at least three phase-shifted sinusoidal fringe patterns onto the object, and a camera views them from another angle. The object’s height distorts the fringes, the phase of the fringes describes that distortion, and the phase plus the system parameters give the 3D shape. Because several patterns are used, traditional PSP requires the object to stay still: if it moves while the patterns are projected and captured, measurement errors appear.

Frame rate is a second limit. A projector typically shows 8-bit fringe patterns at up to 120 Hz, while a camera easily exceeds 1000 Hz, so the frame rate of the reconstructed point clouds is limited by the projector. Defocusing a projector that shows binary patterns can exceed 1000 Hz, but the amount of defocus is hard to control and quantify, and high-speed cameras and projectors raise the cost considerably. The paper asks how to reconstruct moving objects at a high frame rate with low-cost equipment.

How it works

A trigger controller is added between the camera and the projector, so the camera captures each projected fringe pattern twice and collects more fringe information about the moving object. The paper then analyses how the object’s motion changes the phase across three consecutive images, builds a fringe model that includes the motion, extracts the phase under this mixed phase shift, and reuses the captured images to obtain more reconstructed frames.

1. Double shooting. The projector shows its N phase-shifted patterns at its normal rate, and the camera is triggered twice within each projection period. Every projected pattern therefore yields two camera images of the moving object.
2. Mixed phase shift. Two images taken under the same pattern differ only by the phase shift caused by the object’s motion. Images taken under different patterns contain this motion-induced shift as well as the ordinary PSP phase shift. For three images I₁, I₂, I₃, the shifts are δ₁ = δ₂ = 0 and δ₃ = 2π/N.
3. Fringe model with motion. An object motion tracking algorithm computes the rotation–translation matrix that describes the object’s motion. It gives the coordinate transform (f_n, g_n) between images, and each image is written as I_n = a + B cos h_n + C sin h_n, with h_n = ϕ(f_n, g_n) + δ_n, where ϕ is the phase of the reference plane and a is the ambient light intensity.
4. Least-squares phase. For each pixel, a, B and C are found by minimising the squared difference between the model and the captured images. B and C carry the phase change φ caused by the object’s shape (B = cos φ and C = −sin φ), so it is read as φ = arctan(−C / B).
5. Image reuse. The captured images are reused: every three adjacent images give one frame of object phase, whereas traditional PSP builds each frame from its own group of images. A new phase frame therefore appears with each new image, rather than after a full set of patterns. This raises the reconstruction frame rate of the moving object while the projector speed stays limited.

Results

The experimental system consists of an industrial camera (OPT-CC1-048A-RM-G, 1800 × 1600 pixels), a projector (TJ-S50, 1280 × 720 pixels) and a motorised translation stage. The measured object is a mask. The camera runs at 20 frames/s, the stage moves at 20 mm/s, and the field of view is 70 mm × 80 mm. The mask is first reconstructed at rest with traditional PSP, and this result serves as ground truth for the error evaluation. On the moving stage, the two images taken under one projected pattern contain only the object’s motion, while images under different patterns contain both motion and fringe phase shift. The rotation–translation matrix comes from an object motion tracking algorithm, and every three adjacent images give one phase map. The proposed method reconstructs the moving mask, whereas traditional PSP applied to the moving object shows large errors.

The authors name the accuracy of the object tracking algorithm as the most important limitation: it directly sets the reconstruction accuracy, and existing tracking algorithms have difficulty describing large motions. The method can only reconstruct objects undergoing 2D translation; because of the limits of motion tracking, it cannot handle 2D rotation or 3D motion. It does not require the object to move at constant speed, but the camera must capture clear fringe images of the moving object.