Robust Object 6D Pose Estimation Under High Dynamic Ambient Light

The problem

Estimating an object’s 6D pose means recovering its 3D rotation and translation relative to the camera, here from a single RGB image. Gen6D is a model-free pose estimator: at test time it needs no high-quality object model, depth map or object mask, and works from reference images of the object instead. It runs in three stages: a detector finds the object and its size, a selector picks the most similar reference image for a rough rotation, and a refiner corrects the estimate.

Under high dynamic ambient light, with stark contrast between highlights and shadows, this pipeline breaks down. Reflections on the object’s surface change with the light, shadows alter its brightness and texture, and colours are distorted. The authors tested Gen6D in scenes where projected light sources make the object’s surface reflect unevenly, and its pose estimates were poor. The refiner often cannot repair this on its own, because it depends heavily on the rough estimates from the earlier stages, which the lighting has already affected.

How it works

The method keeps Gen6D’s detector–selector–refiner pipeline and changes the feature extractor that all three stages share. An efficient channel attention (ECA) module is embedded in the VGG backbone. The resulting VGG-ECA network weights informative feature channels up and noisy ones down, so the features are less disturbed by glare and shadow.

1. Glare confuses the original model. On objects with metal parts, such as a knife or scissors, Gen6D often finds only the weakly reflective plastic handle and misses the highly reflective metal.
2. Channel attention in the backbone. ECA modules are added after the VGG layers with 64, 128 and 256 filters. Each averages every channel to a single value, applies a 1D convolution across neighbouring channels (its kernel size is set adaptively from the number of channels), maps the result to weights between 0 and 1 with a sigmoid, and multiplies the channels by these weights. Unlike the squeeze-and-excitation (SE) module it builds on, ECA does not reduce the channel dimension first.
3. Detector. VGG-ECA features of the reference images serve as convolution kernels on the query image’s features, at several predefined scales. The peak of the resulting heat map is the 2D projection of the object centre, and the scale map at that point gives the bounding-box size, from which the depth and the initial translation follow.
4. Selector. The object region is compared with the reference images, and the most similar one gives the initial rotation.
5. Refiner. A 3D CNN regresses the residual between the current pose and the true pose and updates the estimate.

Results

The model is trained on GenMOP, the official Gen6D dataset of 10 objects, and for testing the authors chose objects with highly reflective surfaces. ADD-0.1d counts poses whose average model-point distance from the ground truth is within 10 % of the object diameter; Prj-5 counts poses whose 2D projection error is within 5 pixels. On every one of the seven objects in Table 1, the improved model scores higher than Gen6D on both metrics.

Separately captured RGB images under high dynamic ambient light show the effect qualitatively. Where Gen6D boxes only the plastic handles of the knife and scissors, the improved model captures the whole object, and it also detects a transparent glass cup on which Gen6D fails. Across five light levels, from dark (L1) to bright (L5), Gen6D is much less accurate on the knife and scissors in both low (L1, L2) and high (L4, L5) light, while the improved model is more accurate, most visibly on the glass cup. The authors also report failures: strong light makes part of the knife blade appear black, so the improved model cannot detect the pose of those blade sections, and the metal scissors are still misaligned at L1 and L2. They name more objects and datasets, other attention mechanisms and tests in real industrial applications as future work.