3D reconstruction of moving object by double sampling based on phase shifting profilometry

The problem

Phase-shifting profilometry (PSP) is one of the most commonly used 3D measurement techniques, valued for its high precision, robustness and full-field measurement. A projector casts at least three phase-shifted sinusoidal fringe patterns onto the object, and a camera views them from another angle. The object’s height distorts the fringes, and the 3D shape is computed from the phase of the deformed fringes and the system parameters. Because several images are needed, traditional PSP requires the object to stay static during reconstruction. Once it moves during fringe projection and capture, errors are introduced.

Earlier work tracks the object’s motion and compensates the resulting error, estimates the motion between the 3D data of adjacent frames, or uses a high-speed imaging system with shifted Gray code. One approach that estimates motion from the object’s start and end states is limited to uniform motion, and deep-learning methods that recover phase from a single fringe pattern need time-consuming network training and are hard to run in real time. The authors therefore look for a motion-error compensation method suited to real-time phase-shift measurement.

How it works

The method changes how the images are captured and how the phase is computed. The camera is triggered twice during every projected fringe pattern, so two images taken under the same fringe differ only by the object’s motion. With the motion known from tracking, a least-squares model recovers the object’s phase from three consecutive images, and overlapping groups of three images give one phase map each.

1. Two shots per fringe. A trigger control device is added to the camera and projector. The projector projects N fringe images at a set rate and triggers the camera twice in each projection cycle: I₁₁ and I₁₂ are captured while fringe I₁ is projected, I₂₁ and I₂₂ while I₂ is projected, and so on.
2. Separate motion from phase shift. Two images taken under the same projected fringe, such as I₁₁ and I₁₂, have no PSP phase shift between them (δ₁ = 0), so they differ only by the object’s motion. Images taken under different fringes also carry the PSP phase shift δ₂ = 2π/N. The three images I₁₁, I₁₂ and I₂₁ are therefore analysed together.
3. Track the motion. The rotation matrix and translation vector describing the object’s motion are obtained by tracking the object. From them follows the coordinate transformation (f_n, g_n) that relates each point’s position before and after the motion.
4. Solve for the phase by least squares. Each of the three images is written as A + B cos(ϕ + δ_k) + C sin(ϕ + δ_k), where A, B and C are unknowns and ϕ is the reference-plane phase at the transformed coordinates. Minimising the squared error over the three images (N = 3) leads to a linear system D X = E for X = [A B C]^T, and the phase change caused by the object is φ = tan⁻¹(−C/B). The object is then reconstructed from the phase and the system parameters.
5. Multiplex the images. Every three consecutive images give one object phase map, so the groups overlap and a new frame starts with each new image. Traditional PSP needs three projected patterns for each frame.

Results

The method is verified by simulation. Fringe patterns are simulated for a static object and for an object that moves from left to right by 10 pixels between each two fringe patterns. For comparison, the static patterns are reconstructed with traditional PSP. In the moving case, images taken under the same projected fringe contain only the object’s motion, while images under different fringes contain both the fringe phase shift and the motion. The rotation matrix and translation vector of the motion are computed by tracking the object, the phase is computed with the proposed model, and with multiplexing every three images give an object phase map. These phase maps at different frames give a high-frame-rate 3D reconstruction of the moving block (Fig. 5). Traditional PSP applied to the same images shows large errors (Fig. 6).

In their conclusion, the authors note that the method only needs a newly derived formula and the projection of the first two fringe patterns of a three-step phase shift to reconstruct the moving object, and that it doubles the shooting frame rate at low cost without losing the advantages of PSP. The comparison with traditional PSP is shown visually in Figs. 5 and 6.