TopFormer: Topology-Aware Transformer for Point Cloud Registration

The problem

Point cloud registration estimates the rotation and translation that align two partially overlapping scans. Most learned methods describe each point by its local geometry, match the descriptors, reject outliers with RANSAC and solve for the transformation. Recent Transformer-based methods work on a sparse, sub-sampled set of points and let every sparse point attend to all the others, and some add the points’ relative positions so that attention knows where each point sits.

Sub-sampling throws away fine-grained structure, though, and relative position does not help when two sparse points are close to each other in space but far apart along the surface. The paper’s example is the top of a chair next to the corner of a desk: the two points have similar local geometry because they lie in the same neighbourhood, and their relative position barely separates them because they are close in Cartesian space. Their features become ambiguous and lead to wrong matches. In the dense point cloud the difference is easy to see, since the route from one to the other along the surface is long; in the sparse points it is hard to observe.

How it works

TopFormer follows a coarse-to-fine registration pipeline and adds one ingredient: a topological structure encoding that tells the Transformer how each pair of sparse points is connected along the surface. Because the sparse points are a subset of the dense points, this surface information can be measured on the dense cloud and handed to the sparse points, which the authors call a sparse-through-dense strategy.

1. Close in space, far along the surface. A KPConv encoder extracts local features at a sparse and a dense level. Two sparse points such as a chair top and a desk corner can look alike to it, and their relative position does little to separate them.
2. Sparse points inside the dense graph. Down-sampling uses furthest point sampling, so every sparse point is also a dense point. On the dense points, TopFormer builds a K-nearest-neighbour graph, keeping only edges shorter than a radius τ.
3. Geodesic distance by shortest path. For two sparse points p and q, Dijkstra’s algorithm on the dense graph gives the geodesic distance from p to q, the length of the shortest route along the surface.
4. Direction from normals, independent of their sign. Normals estimated by PCA are unoriented, and scans can be in arbitrary poses, so the normals are hard to re-orient consistently. TopFormer therefore uses the sines of the angle between the normal at p and the vector from p to q, and of the angle between the two normals; flipping a normal turns an angle α into π − α and leaves the sine unchanged.
5. Encoding inside attention, then matching. The geodesic distance, the Euclidean distance and the two sines are mapped by a sinusoidal function into the topological structure encoding, which is added to the keys of self-attention. Cross-attention then lets the two clouds exchange information; sparse points are matched from a confidence matrix, the matches are refined at the dense level with an optimal transport layer, and RANSAC estimates the final rotation and translation.

Results

The indoor experiments use 3DMatch and its low-overlap split 3DLoMatch, which holds the scan pairs with 10–30 % overlap. Registration recall is the share of pairs whose estimated transformation has an RMSE below 0.2 m, computed with 50K RANSAC iterations. TopFormer’s registration recall is the highest, or tied for highest, at every correspondence number tested (5,000 down to 250) on both splits, and its feature matching recall on 3DMatch stays between 98.4 % and 98.7 %.

Its inlier ratio is below GeoTransformer’s at 5,000 correspondences (71.7 % against 71.9 % on 3DMatch, 37.2 % against 43.5 % on 3DLoMatch), but higher at smaller numbers, for example 82.8 % against 76.0 % on 3DMatch and 53.1 % against 46.2 % on 3DLoMatch at 1,000. On the outdoor KITTI odometry data, most of the compared methods already reach 99.8 % recall; TopFormer matches that, with a rotation error of 0.27°.

The ablation study shows that both parts of the encoding contribute. Without the geodesic distance, registration recall falls from 92.4 % to 90.3 % on 3DMatch and from 75.1 % to 72.3 % on 3DLoMatch; without the normal-based directional terms it falls to 91.9 % and 73.9 %. Dropping the encoding entirely, which leaves a vanilla Transformer, costs 2.7 and 7.7 points of recall, and the inlier ratio falls by 8.6 and 5.6 points.