Deep learning-enabled 3D multimodal fusion of cone-beam CT and intraoral mesh scans for clinically applicable tooth-bone reconstruction

Planning orthodontic treatment and dental implant surgery demands precise 3D models of both tooth crowns and their underlying root and bone structure. Cone-beam CT (CBCT) captures the bone and root anatomy but has limited surface resolution and requires manual, time-intensive segmentation. Intraoral optical scanners produce high-fidelity digital impressions of the visible tooth surfaces but cannot image beneath the gum line. This paper bridges that gap with a deep learning framework for multimodal 3D fusion. A learned registration network rigidly aligns the CBCT volume with the intraoral mesh, after which a multimodal segmentation network jointly processes both data sources to produce individual tooth-root and alveolar bone labels. The framework eliminates manual segmentation for the majority of clinical cases and produces reconstructions with accuracy validated against expert annotations on a large retrospective CBCT dataset. Published in Patterns (Cell Press, 2023), a high-impact data science journal, the work directly addresses a significant clinical bottleneck in digital dentistry and provides an open, reproducible benchmark for dental CBCT segmentation. The combined tooth-bone model is immediately usable in surgical planning software, enabling same-day patient consultations with accurate 3D visualisation.

Problem setting

Accurate 3D reconstruction of teeth and surrounding bone is essential for orthodontic and implant surgery planning, but current workflows require laborious manual segmentation of cone-beam CT (CBCT) images, which is time-consuming and operator-dependent. Intraoral optical scanners provide high-resolution surface geometry of visible dental structures but cannot capture subsurface bone. This work presents a deep learning framework that fuses CBCT volumetric data with intraoral mesh scans, combining the complementary strengths of both modalities to generate complete tooth-bone 3D reconstructions.

In the broader publication record, this work appears in Patterns (Cell Press), 4(9). The visual notes below pair the paper’s original figures with a concise reading of the method, experimental setup, and reported results.

Method and visual evidence

The method operates on discrete geometry and is designed to preserve meaningful shape structure while filtering noise, estimating features, or improving downstream geometric processing.

The extracted figures below show the geometry-processing pipeline, representative shapes, and visual or numerical comparisons.

Method overview. This image is extracted from an embedded PDF image object on page 4, then recomposed for web display.

Representation and setup. This image is extracted from an embedded PDF image object on page 8, then recomposed for web display.

Results and impact

The evaluation reported in Patterns (Cell Press), 4(9) uses the extracted figures above to show the method’s measurement, reconstruction, segmentation, matching, or diagnostic behavior on representative experiments. These visuals are paired with the paper’s quantitative or qualitative analysis to make the workflow easier to inspect from the homepage.

Source handling

I extracted 2 candidate image objects from paper.pdf and generated the compressed WebP figures used on this page.