SuP: Sub-cloud Driven Point Cloud Registration

The problem

Point cloud registration finds the rotation and translation that line up two 3D scans of the same scene. Modern methods do this precisely when the two scans share a lot of geometry. In applications such as large-scale mapping and autonomous navigation, though, the views often share only a small part of the scene. Once the shared surface falls below about 30 %, classic ICP and learned methods alike struggle to find reliable correspondences, and registration slows, drifts or fails.

The paper traces these failures to the regions that are not shared: they can still be geometrically or semantically similar to each other, and matching the two whole clouds then produces wrong correspondences that obscure the correct ones. Earlier work tried to predict which points lie in the overlap, but that prediction depends on consistent features and becomes less effective on low-overlap pairs. Later methods strengthen the features with attention, geometric encodings, semantics or colour, yet their accuracy still drops as overlap falls.

How it works

SuP changes what it looks for. Two scans may overlap only a little overall, but if each is cut into smaller pieces and the pieces are matched pairwise, a subset of these sub-cloud pairs has high overlap. SuP calls these high-overlap pairs anchor pairs. It finds them in two phases, then builds its matches only from inside them.

1. Features for the whole scan. A KPConv backbone extracts point features at several scales, and attention layers exchange context within and between the two scans. By default SuP uses the colour-enhanced backbone and attention of ColorPCR.
2. Subdivide. Each scan is split into k sub-clouds. Farthest point sampling picks well-spread centroids, and each sub-cloud collects a centroid’s nearest neighbours within a radius.
3. Phase I: overlap-guided prior weighting (OPS). For every source–target pair of sub-clouds, SuP measures how strongly their point features agree, counting only mutual top-ranked correlations, and keeps the best-scoring pairs as candidates. This is an efficient first filter against pairs that do not overlap.
4. Phase II: multi-scale post-weighting (MPN). Each candidate pair is aligned with a preliminary pose from a weighted SVD. A small learned network then checks whether points that now sit close together also have similar features, at two scales. The pairs with the highest feature-consensus weights become anchor pairs. During training, the network is supervised by how well each candidate actually aligns (an alignment-aware weighting loss).
5. Merge-to-match. Inlier correspondences from all anchor pairs are merged, refined on the denser points, and passed to a standard estimator (RANSAC or the RANSAC-free LGR) to compute the final rotation and translation.

In the reported setup, each scan is cut into 6 sub-clouds, giving 36 possible pairs. OPS keeps 24 of them, and the top 8 by feature consensus become anchor pairs.

Results

The evaluation uses colour-enhanced versions of the indoor 3DMatch benchmark: Color3DMatch, with pairs overlapping by more than 30 %, and Color3DLoMatch, the harder split with 10–30 % overlap. Registration recall is the share of scan pairs aligned to within 0.2 m RMSE. SuP uses ColorPCR’s colour-enhanced backbone by default; compared with ColorPCR alone, the strongest baseline, it raises recall from 96.5 % to 97.8 % on Color3DMatch and from 88.3 % to 90.2 % on Color3DLoMatch under LGR. The share of correct matches rises as well: with 250 sampled correspondences on Color3DLoMatch, 76.0 % of SuP’s matches are correct (inlier ratio), against 68.0 % for ColorPCR.

The sub-cloud step also works as an add-on to other methods. Added to GeoTransformer, it raises Color3DLoMatch recall from 74.0 % to 76.7 %; added to PEAL, from 81.2 % to 83.4 % (Table 4). In the ablation study, adding the four components one at a time (OPS, the feature-consensus weights, MPN and the alignment-aware loss) raises recall at every step on both benchmarks (Table 5). Sub-cloud size matters as well: smaller sub-clouds may lower recall because of insufficient overlap, larger ones miss local detail, and the default radius setting τ = 0.35 gives the best balance (Table 6). The paper’s limitation section notes that features from scans captured with very different sensors could weaken OPS and MPN, and suggests feature normalisation or domain adaptation as a next step.