VeinTr: Robust end-to-end full-hand vein identification with transformer

The problem

Hand vein identification recognises a person from the vein pattern of the hand, imaged under near-infrared light. Typical systems follow four steps: locate the hand, extract a region of interest (ROI), extract features from it, and match the result against a database of enrolled hands. Earlier work built hand-crafted geometric or statistical descriptors, which suffer under perturbations such as illumination changes and noise; more recent methods learn the features with convolutional neural networks.

Even the learned methods still work on an ROI, usually cut from the palm. That brings two drawbacks. The preprocessing is complex, and accuracy depends on how robustly the ROI is extracted, which is hard in real applications. And the crop throws information away: the vein pattern across the whole hand is unique to each person, so a representation built from only part of it is potentially inferior to one that uses the full pattern.

How it works

VeinTr skips ROI extraction and takes the full-hand image directly as input. Its identification has only two steps, feature extraction and probe-to-database matching. The feature extractor combines a convolutional local feature encoder, a lightweight transformer and an MLP-based global feature decoder, and the whole network is trained end to end as a metric-learning task.

1. No ROI crop. Conventional pipelines locate the hand and crop a palm ROI before extracting features, so veins outside the crop are lost. VeinTr feeds in the whole hand image instead.
2. Local features. Convolution blocks (convolution, batch normalisation, PReLU) reduce the size of the 256×256 input while extracting local features, and three residual blocks refine them into a 64×64×64 local feature map.
3. Global aggregation with attention. The feature map is cut into 16 patches of 16×16, which are flattened and projected to a lower dimension. Three multi-head attention layers with 8 heads relate every patch to every other one, enlarging the receptive field from local to the whole hand.
4. One feature per hand. The global feature decoder flattens the attention output and applies a two-layer MLP with batch normalisation, giving a 512-dimensional hand vein feature. Training uses the ArcFace loss, which pulls features of the same hand together and pushes different hands apart.
5. Probe-to-database matching. The normalised features of the enrolled hands form a database matrix D. A probe hand is assigned to the identity C with the highest similarity score, C = argmax(D · ‖VeinTr(p)‖), where ‖·‖ denotes normalisation.

Results

All methods in the comparison work on a palm or finger ROI; VeinTr is the only one that uses the full hand. On CASIA, all hand-crafted methods already exceed 90 % correct recognition and the earlier method NAF is close to saturation at 99.83 % and 99.50 %, so VeinTr’s lead in recognition rate is small (0.02 and 0.42 points); in equal error rate it is 0.41 points below NMRT and 3.61 below PVSNet (Table 1). For inter-dataset tests, VeinTr is trained on one dataset and tested on all three. Models trained on CASIA or TPV stay above 90 % correct recognition on every test set. PLUSVein is much smaller (420 images of 42 volunteers), and the model trained on it generalises less well, reaching 89.76 % on CASIA and 81.43 % on TPV.

The ablation study (all models trained on CASIA) shows that both parts of the network matter. Without the attention layers, the correct recognition rate drops to 83.43 % on CASIA, 65.14 % on TPV and 55.10 % on PLUSVein; without the residual blocks, it drops to 95.12 %, 75.07 % and 83.67 % (Table 3). With three attention layers the rate saturates on CASIA and peaks on TPV and PLUSVein; a fourth and fifth layer lower it, which the authors attribute to overfitting. Three residual blocks are chosen as a trade-off between recognition rate and computational cost. As future work, the authors plan to fuse full-hand vein information with full-hand print data.