New Sketch2CT framework uses AI diffusion models to generate anatomically accurate 3D medical volumes from 2D sketches and text descriptions

Sketch2CT combines multimodal guidance—user-drawn 2D sketches and textual descriptions—to create 3D medical volumes with consistent anatomical structures

The system uses diffusion probabilistic models to generate 3D segmentation masks from random noise, conditioned on both sketch and text inputs

Custom modules align and fuse sketch and text data, with a capsule-attention backbone architecture to improve feature refinement and integration

Addresses the critical challenge of medical image data scarcity by enabling synthetic generation of high-quality, realistic 3D medical imaging data