FPD: Fringe Photometric Deflectometry when Fringe Meets Photometric Stereo

The problem

Many industrial parts are neither purely matte nor purely mirror-like. The back of a phone, for example, has a glossy logo on a surface whose specular reflection is weaker, and defects such as scratches and fingerprint smudges have to be found on both. Two established optical methods each cover only half of this. Photometric stereo (PS) recovers surface normals from images taken under several illumination directions; it works well on diffuse (Lambertian) regions but degrades where specular highlights appear. Phase Measuring Deflectometry (PMD) measures surface gradients from the reflection of sinusoidal fringes; it is accurate on specular surfaces but unreliable on matte areas, where fringe contrast is low.

As a result, existing inspection methods often fail to detect subtle defects when specular highlights or diffuse textures dominate. Few earlier works merge the two methods: one combined photogrammetry with PS, another fused line-structured light with PS but had difficulties in highly reflective areas.

How it works

FPD captures the data for both methods in one cycle, runs a PMD branch and a PS branch, and fuses their outputs pixel by pixel. The PMD branch also measures how specular each pixel is, and that measurement sets the fusion weights.

1. One capture cycle. A projector displays three kinds of patterns: binary patterns that simulate four-quadrant illumination for PS, and horizontal (fringe-x) and vertical (fringe-y) sinusoidal fringes for PMD. The camera records 12 images per cycle.
2. PMD branch. With four phase-shifted fringe images per pixel, the mean intensity is A, the specular component S is computed from the fringe modulation, and the diffuse component is D = A − S; glossiness is S / A. Multi-frequency unwrapping gives the absolute phase, whose x and y derivatives form the phase gradient magnitude P.
3. PS branch. The four binary-pattern images, using the PMD specular and diffuse reflection maps as prior information, form an intensity matrix of images × pixels. Its singular value decomposition gives the surface normals, from which the surface gradients and the mean curvature H are computed, with CUDA acceleration.
4. Weights from reflectance. Each pixel gets w_PMD = S / (S + D) and w_PS = D / (S + D). Glossy pixels lean on PMD, more diffuse pixels on PS, and the transition between them is smooth.
5. Fusion. The result is R = w_PMD·P + w_PS·H. PMD dominates where fringes are reliable, PS where fine diffuse detail needs to be kept.

Results

The test object is the back of an iPhone, which shows both diffuse and specular reflection. It was imaged on a platform with a projector, a camera and mounting structures, with a 12-image cycle of binary, fringe-x and fringe-y patterns, and processed on a PC with an Intel Core i5-13500 CPU and an NVIDIA GeForce RTX 3060 GPU. The comparison is between the three reconstructed maps of the same capture: the PS mean curvature map, the PMD phase gradient map and the FPD fusion result, which the authors find more complete and consistent across both types of feature. Defect detection was then run on each of the three maps with Smart3, a commercial inspection software from OPT Machine Vision: scratch detection, and smudge detection by line extraction. FPD has also been integrated into commercial defect inspection software.

The authors name calibration of the integrated system and height measurement for complex surfaces as future work.