TTA-MAC: Per-Instance Test-Time CAD-Anchored Refinement of Monocular Depth Calibration and 6D Object Pose

The problem

Zero-shot 6D object pose from a single RGB image has advanced quickly thanks to foundation detectors and monocular depth models. A central bottleneck remains, according to the authors: metric grounding. Monocular depth gives rich relative geometry, but its scale and shift are biased for each object instance. A global CAD-guided affine calibration such as CAD-MAC reduces this bias, yet the remaining residual can be strongly non-Gaussian: on LineMod-Occlusion with the DAv2 depth model, its skewness is +4.5 and its excess kurtosis +36. A Gaussian prior for per-instance refinement therefore fits the actual errors poorly.

How it works

TTA-MAC is a CAD-anchored refinement layer applied at test time. It takes a frozen detector, a frozen monocular depth backbone, a warm-start pose and the object’s CAD model, and refines only eight geometric variables: the two calibration scalars of the monocular depth and the 6D pose.

• Per-instance MAP problem. The depth-derived point cloud is aligned to a signed distance field computed from the CAD model.
• Heavy-tailed prior. A Student-t prior models the heavy-tailed calibration residuals.
• Trust region. The refinement runs inside a trust region with three gates and restores the best state found so far. The key safeguard is a signed-SDF gate on the fraction of the pseudo-depth cloud that lies inside the CAD volume. This detects wrong-basin states where the cloud penetrates the object, a failure that ordinary ICP or unsigned-distance alignment cannot see.

Results

Compared with a CAD-MAC+ICP baseline, the median ADD error improves from 44.0 mm to 31.3 mm on LineMod-Occlusion, and the success rate (ADD below 10% of the object diameter) rises from 10.4% to 14.3%. On LineMOD, across 18,273 instances, the median ADD improves from 43.5 mm to 30.5 mm and the success rate from 17.3% to 23.3%. The transfer to other datasets is partial: the median ADD improves from 41.1 mm to 38.3 mm on YCB-V and from 59.5 mm to 55.6 mm on TUD-L, and these runs also expose remaining threshold and over-refinement failures. The authors name visibility-aware, symmetry-aware and occlusion-robust trust regions as future directions.

This page summarises the abstract of the manuscript; details will be added once the paper is published.