Publications / 2026 / Few-Shot 6D Object Pose Estimation via Decoupled Rotation and Translation with Viewpoint Encoding

Few-Shot 6D Object Pose Estimation via Decoupled Rotation and Translation with Viewpoint Encoding

Lei Lu, Peng Cao, Wei Pan, Zhilong Su, Haojun Zhang, Wangxing Zheng, Ge Gao, Peng Li
Electronics, 15(3):561
↓ PDF { } BibTeX DOI ↗ Code

In one sentenceRotation is learned from synthetic renders alone and translation from just 600 real images per object, so this decoupled network estimates 6D object pose from one RGB image while relying much less on annotated real images.

The problem

6D pose estimation recovers an object’s 3D rotation and 3D translation relative to the camera, a core step for robotic grasping, autonomous driving and augmented reality. Depth sensors are affected by noise, occlusion and cost, and struggle with reflective or transparent objects, which motivates methods that use a single RGB image. Monocular methods based on keypoints, templates or direct regression have made steady progress, but they rely heavily on large annotated real datasets, and the scarcity of real pose annotations limits further gains.

This work decouples rotation from translation. Rotation is learned from synthetic images alone, so only the translation part is trained on real images, and the translation part does not have to learn rotation from them, which reduces the number of real samples needed.

How it works

An object detector first crops the object from the image. Rotation and translation are then estimated by two independent branches. The rotation branch factorises the full rotation R into an out-of-plane part Rγ (the object’s viewpoint) and an in-plane part Rθ (a rotation about the camera’s optical axis), R = RθRγ, and handles each with modules trained on synthetic images only. The translation branch builds on GDR-Net and is the only part trained on real images.

  1. Detect and crop. An object detector extracts a 128×128 region of interest (ROI) around the object from the RGB image; both branches work on this crop.
  2. Retrieve candidate viewpoints. A viewpoint encoder, trained on images rendered from ShapeNet, maps the ROI to a 64-dimensional vector that is sensitive to viewpoint but invariant to in-plane rotation. For the target object, 4,000 viewpoints are sampled uniformly on a sphere around its mesh and rendered to build a codebook; cosine similarity against the codebook gives the top-K candidate viewpoints.
  3. Regress the in-plane rotation. For each candidate, a regression head compares the ROI with the view rendered at that viewpoint and predicts the in-plane rotation, giving a complete 3D rotation hypothesis.
  4. Verify the orientation. A verification head scores how consistent each hypothesis is with the ROI, and the highest-scoring hypothesis is taken as the rotation estimate.
  5. Regress the translation. Following GDR-Net, a ResNet-34 predicts three 64×64 maps from the ROI (a dense 2D–3D correspondence map, a surface-region attention map and a visible-object mask), and a lightweight network regresses the scale-invariant translation from the correspondence and surface-region maps. This branch is trained with 600 real images per object.
Pipeline diagram: an RGB image passes through an object detector to an ROI; a rotation branch with viewpoint encoder, codebook, in-plane rotation regression and orientation verification outputs R; a translation branch with geometric feature regression maps and translation regression outputs T; together they give the 6-DoF object pose drawn on the image.
Network overview: (A) object detection, (B) rotation estimation by viewpoint retrieval, in-plane rotation regression and orientation verification, and (C) translation estimation from dense geometric features. Fig. 1 of the paper.
A hemisphere of grey viewpoint dots around a small yellow duck model; two highlighted viewpoints a and b each show the duck rendered at three in-plane rotations.
Viewpoints sampled uniformly on a sphere around the object. For two viewpoints, a and b, the object is rendered at three in-plane rotations, illustrating the split of a rotation into a viewpoint Rγ and an in-plane rotation Rθ. Fig. 2 of the paper.

Results

The evaluation uses LINEMOD (13 objects in cluttered scenes), LM-O (8 partly occluded objects, 1,214 test images) and YCB-V (21 objects, with severe occlusion and several symmetric objects). A pose counts as correct when its ADD(-S) error is below 10 % of the object’s diameter; for YCB-V the area under the ADD(-S) curve up to 10 cm is reported as well. The rotation network is trained on synthetic images only and the translation network on 600 real images per object; for YCB-V, physically based rendered images are also used in training.

On LM-O the method is ahead of the compared methods on most objects, with the largest gains over GDR-Net on Holepuncher (74.5 % against 60.1 %), Duck and Ape; GDR-Net remains slightly better on Can and Eggbox. On YCB-V it also leads in AUC of ADD(-S), 86.9 % against 84.4 % for GDR-Net. The few-shot comparison shows the largest gap: when ZebraPose and CheckerPose are retrained with the same 600 real images, their YCB-V ADD(-S) falls to 35.7 % and 37.8 %, against 65.9 % here (Table 6). The rotation modules were also evaluated on their own on LINEMOD: on real images with predicted ROIs, viewpoint retrieval is within 10° in 72 % of cases, and in-plane rotation is within 10° in 74 % of cases with predicted viewpoints and ROIs and in over 90 % with ground-truth viewpoints (Figs. 5 and 6), so in-plane accuracy depends strongly on retrieval quality. The authors note that accuracy partly depends on the reliability of the object detector, and that generalisation to unseen object categories remains limited.

Six cluttered tabletop scenes from LM-O, each with a green 3D bounding box drawn around one estimated object.
Qualitative results on LM-O; estimated poses are drawn as green 3D boxes. Fig. 7(a) of the paper.
Six YCB-Video scenes with household objects such as a sugar box, a bowl, a mustard bottle and a pitcher, each enclosed by a green 3D bounding box.
Qualitative results on YCB-Video. Fig. 7(b) of the paper.

Cite

@article{lu2026few,
  title={Few-Shot 6D Object Pose Estimation via Decoupled Rotation and Translation with Viewpoint Encoding},
  author={Lu, Lei and Cao, Peng and Pan, Wei and Su, Zhilong and Zhang, Haojun and Zheng, Wangxing and Gao, Ge and Li, Peng},
  journal={Electronics},
  volume={15},
  number={3},
  pages={561},
  year={2026},
  publisher={MDPI},
  doi={10.3390/electronics15030561}
}
Type
Article Journal
Topic
6D Pose & Robotics
Venue
Electronics, 15(3):561
Year
2026
DOI
10.3390/electronics15030561