SemReg: Semantics-Constrained Point Cloud Registration

Point cloud registration is the task of finding the rigid transformation that best aligns two 3D scans of the same scene captured from different viewpoints. Geometric matching methods based on local shape descriptors are powerful but brittle: textureless walls, repetitive furniture, and symmetric objects all create correspondence ambiguities that cause alignment failures. SemReg resolves this fundamental limitation by treating semantic labels as first-class geometric constraints. The key insight is that semantic category membership is invariant to viewpoint and illumination, providing a complementary signal that can eliminate geometrically degenerate correspondences. Technically, SemReg integrates semantic feature embeddings into a transformer-based correspondence network, jointly optimising geometric and semantic consistency in the matching layer. The result is a registration system that remains reliable precisely in the cases where purely geometric methods fail—low-overlap captures and symmetry-rich scenes. Accepted at ECCV 2024, a top-tier computer vision venue, SemReg achieves state-of-the-art performance on the standard 3DMatch, 3DLoMatch, and KITTI benchmarks, with the largest improvements concentrated in the hardest, low-overlap evaluation splits.

Algorithm principle

The algorithmic problem is to estimate a rigid transformation between two partially overlapping point clouds. The method builds discriminative local or sub-cloud features, searches for reliable correspondences, and then uses those correspondences to solve for the final alignment while suppressing ambiguous matches from repeated geometry, low overlap, or noisy regions.

Visual material

The local archive does not currently include a matched PDF for this entry, so additional method and result figures are pending. Once the PDF is added, the page can be regenerated with representative visuals.

Results and impact

The reported evaluation in European Conference on Computer Vision, 293-310 positions the method as a practical contribution to the surrounding 3D vision and industrial inspection workflow. The experiments are used to show whether the proposed representation or pipeline improves robustness, accuracy, speed, or deployability compared with the relevant baseline methods.