HGR-DiffSeg: Hybrid-guided robust diffusion model for medical image segmentation

Abstract

Medical image segmentation plays a critical role in clinical treatment by enabling accurate disease diagnosis and effective therapy. Recently, diffusion probabilistic models, as a class of generative models, have demonstrated strong capability in generating fine-grained details and preserving image diversity through modeling the underlying data distribution. Despite their success, directly applying diffusion models conditioned solely on the original image to medical image segmentation can lead to loss of segmentation-relevant information due to fundamental differences between generation and segmentation tasks. To address this, we propose HGR-DiffSeg, a novel framework for medical image segmentation that integrates hybrid guidance to enhance segmentation accuracy and robustness. Specifically, the contributions of this work are mainly as follows: (i) We propose a Coarse-label and Image Condition-constrained strategy, combined with a Dense Conditional Random Field, to mitigate the loss of critical information during coarse-label prediction; (ii) We design a Frequency-Spatial Dynamic Weight Adjustment module to adaptively weight semantic features and enhance the model's focus on regions of interest; (iii) We incorporate a Multi-Prediction Ensemble Post-Processing method during sampling to improve segmentation through integration and refinement. Extensive experiments on the ISIC2018, Kvasir-SEG, and REFUGE-2 datasets, which cover different imaging modalities, demonstrate that HGR-DiffSeg consistently outperforms state-of-the-art segmentation methods across various evaluation metrics, particularly excelling in gastrointestinal polyp segmentation tasks.