IEEE Transactions On Image Processing[JOURNAL]
Sun
200 papers
Time-Variant Image Inpainting via Interactive Distribution Transition Estimation.
Xing Y, Guo Q, Li X … +6 more , Huang Y, Cao X, Gong L, Lin D, Tsang I, Ma L
IEEE Trans Image Process · 2026 · PMID 42060431 · Publisher ↗
In this work, we focus on a novel and practical task, i.e., Time-vAriant iMage inPainting (TAMP). The aim of TAMP is to restore a damaged target image by leveraging the complementary information from a reference image, w...
In this work, we focus on a novel and practical task, i.e., Time-vAriant iMage inPainting (TAMP). The aim of TAMP is to restore a damaged target image by leveraging the complementary information from a reference image, where both images capture the same scene but with a significant time gap in between, i.e., time-variant images. Different from conventional reference-guided image inpainting, the reference image under TAMP setup presents significant content distinction to the target image and potentially also suffers from damages. Such an application frequently happens in our daily life to restore a damaged image by referring to another reference image, where there is no guarantee of the reference image's source and quality. In particular, our study finds that even SOTA reference-guided image inpainting methods fail to achieve plausible results due to the chaotic image complementation. To address such an ill-posed problem, we propose a novel Interactive Distribution Transition Estimation (InDiTE) module which interactively complements the time-variant images with appropriate semantics thus facilitate the restoration of damaged regions. To further boost the performance, we propose our TAMP solution, namely Interactive Distribution Transition Estimation-driven Diffusion (InDiTE-Diff), which integrates InDiTE with SOTA diffusion model and conducts latent cross-reference during sampling. Moreover, considering the lack of benchmarks for TAMP task, we newly assembled a dataset, i.e., TAMP-Street, based on existing image and mask datasets. We conduct experiments on the TAMP-Street datasets under two different time-variant image inpainting settings, which show our method consistently outperform SOTA reference-guided image inpainting methods for solving TAMP.
LSGNet: A Local-Pattern Separation and Global-Aware Network for Temporal Action Detection.
Gao Z, Yang W, Zhao Y … +3 more , Ma C, Li C, Wang R
IEEE Trans Image Process · 2026 · PMID 42060430 · Publisher ↗
Temporal Action Detection aims to localize and classify action instances within untrimmed videos, yet it remains challenging due to background clutter, high intra-class similarity, and varied temporal scales in real-worl...
Temporal Action Detection aims to localize and classify action instances within untrimmed videos, yet it remains challenging due to background clutter, high intra-class similarity, and varied temporal scales in real-world scenarios. To address these issues, we propose the Local-Pattern Separation and Global-Aware Network (LSGNet) tailored for temporal action localization. Specifically, the core of LSGNet is the Local Pattern Separation Module (LPSM), which explicitly models both consistency and variation patterns of action segments within local temporal windows. Additionally, to capture comprehensive contextual information, we introduce the Global Context-Aware Representation Module (GCRM), which decouples temporal features across multiple granularities and enables robust modeling of long-range dependencies. Finally, we design the Multi-scale Feature Refinement Module (MFRM) to mitigate the degradation of fine-grained information by performing iterative reconstruction across temporal scales, thereby enriching semantic representations and preserving temporal details. Extensive experiments on THUMOS14, ActivityNet1.3, HACS, and EPIC-Kitchens-100 demonstrate the effectiveness of the proposed LSGNet method. Additional ablation studies on the QVHighlights dataset further confirm the generalization capability of LPSM module in video moment retrieval and highlight detection, achieving consistent improvements in retrieval accuracy and localization precision.
Enhanced Query Attention Constrained by Bi-Directional Graphs for Human Pose Estimation Networks.
Yang Y, Fu H, Qian W … +2 more , Wang T, Lv Y
IEEE Trans Image Process · 2026 · PMID 42060429 · Publisher ↗
In human pose estimation, formulating keypoint localization as a classification task over discretized coordinate grids has proven effective. Essentially, the 2D features of the keypoints are reduced to 1D coordinate repr...
In human pose estimation, formulating keypoint localization as a classification task over discretized coordinate grids has proven effective. Essentially, the 2D features of the keypoints are reduced to 1D coordinate representations. This process leads to the loss of spatial constraints among keypoints and increases the difficulty for the model to capture their structural relationships. To address this issue, we propose an enhanced query attention mechanism constrained by bidirectional graphs. The core idea is to establish the topological constraints on the 1D coordinate representations. First, two fundamental connection directions of the skeleton are defined and encoded as a pair of adjacency matrices to enhance the feature interaction capability of the graph convolutional network (GCN). Second, a GCN-guided multi-scale feature fusion framework is designed to effectively combine multi-scale visual features with structural priors, thereby enhancing the representation of keypoint spatial distributions. Finally, a dual-gate module is incorporated into a GCN-guided attention unit to construct a structured query matrix constrained by the bidirectional skeleton graphs, which helps filter out spurious joint interactions and emphasize plausible ones. Extensive experiments on Tai Chi Chuan-Pose, Animal-Pose, AP-10K, MPII, COCO, and COCO-WholeBody datasets demonstrate that the proposed method outperforms existing methods in terms of both accuracy and robustness, particularly in balancing precise local keypoint localization with global pose consistency.
FAST-GOAL: Fast and Efficient Global-Local Object Alignment Learning.
Choi H, Jang YK, Eom C
IEEE Trans Image Process · 2026 · PMID 42055994 · Publisher ↗
Vision-language models such as CLIP have shown impressive capabilities in aligning images and text, but they often struggle with lengthy and detailed text descriptions due to pre-training on short and concise captions. W...
Vision-language models such as CLIP have shown impressive capabilities in aligning images and text, but they often struggle with lengthy and detailed text descriptions due to pre-training on short and concise captions. We present FAST-GOAL (Fast and Efficient Global-local Object Alignment Learning), an efficient fine-tuning method that enhances ability of CLIP to handle lengthy text through global-local semantic alignment. Our method consists of two key components. First, Fast Local Image-Sentence Matching (FLISM) efficiently extracts local image regions through object detection and spatial division, then matches them with corresponding sentences. Second, Token Similarity-based Learning (TSL) maximizes the similarity between patch tokens from specific regions in the image and their corresponding region embeddings, applying the same principle to text, which enhances the ability of the model to capture detailed correspondences. Additionally, we introduce GLIT100k, a dataset that provides both global image-lengthy caption pairs and context-derived local pairs, where local descriptions are extracted from global captions to maintain semantic coherence. Through extensive experiments on long caption datasets (DOCCI, DCI) and short caption datasets (MSCOCO, Flickr30k), we demonstrate that FAST-GOAL achieves significant improvements over baselines, enabling effective adaptation of CLIP to detailed textual descriptions while maintaining computational efficiency.
LSRNet: A Novel Interpretable Low-rank Sparse Representation Guided Fusion Network for Polarization and Intensity Images.
Yang B, Hu Y, Liu L … +2 more , Liu Y, Li J
IEEE Trans Image Process · 2026 · PMID 42055993 · Publisher ↗
Polarization and intensity images fusion (PIF) has extracted extensive attentions as it can generate images with clear scene information and salient texture details of the object surface that are important for downstream...
Polarization and intensity images fusion (PIF) has extracted extensive attentions as it can generate images with clear scene information and salient texture details of the object surface that are important for downstream applications. However, existing deep learning-based PIF methods usually lack interpretability and ignore the interactions among multi-modal features. To this end, we propose a novel interpretable low-rank sparse representation guided fusion network for polarization and intensity images (termed LSRNet). Specifically, a low-rank sparse representation deep unfolding module is designed to acquire the base and detail features of the source images, with the ability of improving the interpretability of the network. In addition, a cross-modal connection complementary feature extraction module is proposed, which aims to establish dependency among features of multi-modalities to fully extract complementary features of the source images. In order to demonstrate the validity of our LSRNet and take into account shortcomings of existing datasets for PIF, a multi-scene polarization and intensity image dataset, named MSPI dataset, is constructed, which includes 1034 high-resolution aligned image pairs. According to the best of our knowledge, this is the most comprehensive dataset for PIF that with a large number of image pairs, high resolution and multiple scene types. Extensive experiments on our MSPI dataset and two publicly available datasets (i.e., 12CFC and HCP) demonstrate the superior fusion performance, generalization ability, and desirable running efficiency of our LSRNet. Our codes and dataset will be publicly available at https://github.com/thebinyang/LSRNet.
Continuous Shape-to-Texture Face Aging With Flow-Based Prior Latent Age Modulation and Attentional Alignment StyleGAN.
Hu X, Qu J, Chen C … +2 more , Liang Y, Zhou Y
IEEE Trans Image Process · 2026 · PMID 42055992 · Publisher ↗
Albeit recent Generative Models have achieved notable progress in synthesizing realistic facial aging images, many of them, e.g., GAN-based methods, cannot accurately capture the continuous progression of age-related sha...
Albeit recent Generative Models have achieved notable progress in synthesizing realistic facial aging images, many of them, e.g., GAN-based methods, cannot accurately capture the continuous progression of age-related shape-to-texture changes over time. In this paper, we propose an innovative facial age transformation framework that enables the generation of continuous shape-to-texture aging facial images. Firstly, the Prior Latent Age Modulation (PLAM) is designed to leverage the advantages of continuous sampling in high-dimensional space by normalizing flows to achieve precise and reversible mapping between the age attribute variable distributions and the prior latent space, ensuring smooth transitions along with facial aging. Secondly, we introduce the Attentional Feature Fusion (AFF), which dynamically allocates weights to effectively fuse the age attribute features by the latent space manipulation with the content features in StyleGAN, thereby generating facial images that accurately depict facial characteristics from shape to texture corresponding to specific ages. Finally, through quantitative and qualitative analysis of existing datasets, we validate the effectiveness and superiority of our proposed method in facial aging tasks.
Real-World Nighttime Image Dehazing via Bayesian-Based Fractional-Order Variational Model.
Liu Y, Li T, Zhou Z … +2 more , Ren W, Lin W
IEEE Trans Image Process · 2026 · PMID 42055991 · Publisher ↗
Images captured under real-world nighttime haze conditions often suffer from severe degradations, including low visibility, color distortion, and reduced contrast, which not only impair visual perception but also degrade...
Images captured under real-world nighttime haze conditions often suffer from severe degradations, including low visibility, color distortion, and reduced contrast, which not only impair visual perception but also degrade the performance of vision-based tasks. However, existing dehazing methods are mainly designed for daytime scenarios and struggle to cope with the complex illumination and scattering characteristics of nighttime hazy images. In this paper, we propose a novel Bayesian-based variational framework with fractional-order constraints for real-world nighttime image dehazing. First, a simplified physical model is constructed to characterize nighttime hazy images, accounting for haze, low-light conditions, Poisson noise, and glow degradations. An anisotropic pre-processing strategy is iteratively applied in the Lab color space to remove glow effects. Subsequently, illumination and reflectance estimation within our constructed physical model is formulated as a maximum a-posteriori (MAP) problem, which is then approximated as a unified variational optimization function. To impose prior constraints, two fractional-order terms are introduced as priors to regulate the illumination and reflectance, promoting piecewise smoothness in illumination and preserving sharp edges and fine textures in reflectance. The resulting variational model is efficiently solved using the alternating direction minimization method. Finally, the estimated illumination and reflectance are enhanced via spatial-domain gamma correction for brightness adjustment and frequency-domain processing for texture detail enhancement. Extensive experiments on real-world datasets demonstrate that the proposed framework outperforms state-of-the-art dehazing methods in both qualitative and quantitative evaluations. Besides, our algorithm generalizes effectively to both other degraded scenes and high-level vision tasks.
Training-Free Open-Set Domain Adaptation With Vision-Language Models.
Yu Z, Lu K, Wu K … +3 more , Chen H, Li F, Li J
IEEE Trans Image Process · 2026 · PMID 42055990 · Publisher ↗
With the prevalence of pre-trained vision-language models like CLIP, leveraging the generic knowledge embedded in CLIP for domain adaptation has proved to be a promising direction. However, most existing CLIP-based metho...
With the prevalence of pre-trained vision-language models like CLIP, leveraging the generic knowledge embedded in CLIP for domain adaptation has proved to be a promising direction. However, most existing CLIP-based methods are limited to closed-set settings. This is primarily because CLIP needs the semantic labels of unknown classes for inference, thus making it not applicable to Open-Set Domain Adaptation (OSDA). To utilize the complementary roles of CLIP and the source model, our paper proposes a novel Semantic-guided Target Adaptation (SemTA) framework for OSDA in a training-free manner. Specifically, we introduce an unknown semantic discovery module. It uses the cluster centroids of the target data to obtain the semantic labels of unknown classes from the worldwide corpus. Then, the semantic-based inference can be performed with CLIP. Additionally, the dual sample attention mechanism is implemented to output sample-based inference. Representative features from both the source model and CLIP serve as the key to improve task specificity. Compared to previous OSDA methods which reject unknown data by confidence threshold, the proposed approach is more practical and offers better interpretability. Comprehensive evaluations on four benchmarks reveal our method sets a new state-of-the-art even without training. Our code will be publicly available soon.
Soft Supervision Guided Spatial-Temporal Refinement Network For Video-based Visible-Infrared Person Re-Identification.
Li J, Zhou C, Li R … +5 more , Li H, Lin X, Lu G, Xu Y, Zhang D
IEEE Trans Image Process · 2026 Apr · PMID 42055989 · Publisher ↗
Thanks to automatic switch between visible and infrared modes, person re-identification (Re-ID) in 24-hour has been possible through cross-modal retrieval. Instead of exploiting still images, video-based cross-modal pers...
Thanks to automatic switch between visible and infrared modes, person re-identification (Re-ID) in 24-hour has been possible through cross-modal retrieval. Instead of exploiting still images, video-based cross-modal person Re-ID is studied in this paper. Specifically, a large-scale dataset 'HITSZ-PVCM' is first collected, consisting of as many as 1,681 identities and 839,632 frames. Generally, videos contain much richer pedestrian appearances. However, most existing works only generate temporal representations by whole frames, inevitably losing fine-grained details. Furthermore, training a network by metric losses (e.g., center loss) is a common strategy, while such point-to-point constraints are too strong and limit model generalization due to existing diversity among intra-class samples. Here, we propose a Soft Supervision guided Spatial-Temporal Refinement (STR) network to tackle these problems. Specifically, STR refines each frame guided by a coarse temporal feature, so that more discriminative features are extracted and transformed to a sequential representation. Followed by a global-local mutual learning module, the modality gap is then erased without losing fine-grained details. Furthermore, we propose a novel soft-clustering center loss to measure intra-/inter-class similarity/dissimilarity in a group-to-group way, efficiently improving model generalization. To the best of our knowledge, HITSZ-PVCM is the largest dataset and STR achieves superior performances compared with state-of-the-arts.
Bias Alleviation Through Network Pruning for Sparse and Debiased Models.
Hong S, Kim S, Joo H … +4 more , Han H, Shin J, Wald Y, Lee J
IEEE Trans Image Process · 2026 · PMID 42055988 · Publisher ↗
Pruning is a highly effective method for reducing the size of neural networks with negligible impact on their average performance. However, recent studies have revealed that pruning actually amplifies the bias in the mod...
Pruning is a highly effective method for reducing the size of neural networks with negligible impact on their average performance. However, recent studies have revealed that pruning actually amplifies the bias in the models, leading to decreased performance for underrepresented groups. To address this issue, we first analyze the impact of pruning on the confidence of each sample and introduce Accumulated Confidence (AC). AC is a proxy that facilitates the identification of bias-conflicting and bias-aligned samples without relying on group annotations. We then propose a debiasing algorithm, which is called DEbiasing Network through Pruning (DENP). DENP utilizes AC to mitigate bias within the network. Even without bias information, DENP exhibits remarkable debiasing performance on varying levels of sparsity, effectively mitigating the bias-exacerbating property of pruning and resulting in both sparse and debiased neural networks. Moreover, even when compared with state-of-the-art debiasing baselines under identical conditions, the DENP still achieves the best performance on multiple benchmark datasets, demonstrating its superior debiasing capabilities.
FTGID: Fine-Grained Text-Driven Framework for Universal Generative Image Detection.
Huang Z, Lin L, Chen B … +2 more , Wang Y, Zhao T
IEEE Trans Image Process · 2026 · PMID 42048223 · Publisher ↗
The rapid progress of generative models has made detecting realistic forgeries a critical challenge for security and trust. Existing image and frequency-based methods depend on dataset-specific artifacts with poor genera...
The rapid progress of generative models has made detecting realistic forgeries a critical challenge for security and trust. Existing image and frequency-based methods depend on dataset-specific artifacts with poor generalization, while Vision-Language Model (VLM)-based methods remain limited by coarse prompts and underused cross-modal alignment. To address these issues, we propose a Fine-grained Text-driven Generative Image Detection (FTGID) framework, which enables comprehensive detection through multi-modal cues. First, we design a Layer-wise Adaptive Global Extractor (LAGE) that stabilizes multi-level global representations through adaptive CLS token fusion with lightweight calibration and parameter-efficient tuning. Second, we propose a Fine-grained Text-guided Local Enhancer (FTLE) that performs patch-level text-visual interaction to enhance the localization of forgery-relevant regions. Third, we introduce a High-frequency Artifact Feature Extractor (HAFE) that adaptively captures discriminative high-frequency cues, enabling more reliable detection of subtle generative artifacts. Extensive experiments demonstrate that FTGID consistently outperforms state-of-the-art GID methods across diverse generative models and unseen datasets, achieving superior performance, thereby enhancing both robustness and interpretability in open-world generative image detection. Our codes will be made publicly available after the peer review process.
Rethinking the Importance of High-Frequency Components in Transformers for Image Restoration.
Zhou S, Pan J, Chen D … +2 more , Dong Y, Yang J
IEEE Trans Image Process · 2026 · PMID 42043995 · Publisher ↗
Transformer-based approaches have achieved superior performance in image restoration, since they can model long-term dependencies well. However, the limitation in capturing local information restricts their capacity to r...
Transformer-based approaches have achieved superior performance in image restoration, since they can model long-term dependencies well. However, the limitation in capturing local information restricts their capacity to remove degradations. While existing approaches attempt to mitigate this issue by incorporating convolutional operations, the core component in Transformer, i.e., self-attention, which serves as a low-pass filter, could unintentionally dilute or even eliminate the acquired local patterns. In this paper, we propose HIT, a simple yet effective High-frequency Injected Transformer for image restoration. Specifically, we design a window-wise injection module (WIM), which incorporates abundant high-frequency details into the feature map, to provide reliable references for restoring high-quality images. We also develop a bidirectional interaction module (BIM) to aggregate features at different scales using a mutually reinforced paradigm, resulting in spatially and contextually improved representations. In addition, we introduce a spatial enhancement unit (SEU) to preserve essential spatial relationships that may be lost due to the computations carried out across channel dimensions in the BIM. Extensive experiments on 6 tasks (real noise, rain streak, blur, flare, underwater conditions, and low-light conditions) demonstrate that HIT with linear computational complexity performs favorably against the state-of-the-art methods. The source code is available at https://github.com/joshyZhou/HIT_.
Harnessing Diffusion Models for Image Manipulation With Partial Sketches.
Li T, Tu S, Xu L
IEEE Trans Image Process · 2026 · PMID 42043994 · Publisher ↗
Controllable image structure editing has attracted increasing attention. While recent interactive point-based methods are convenient and realistic, they often lack fine-grained control over localized content. Partial ske...
Controllable image structure editing has attracted increasing attention. While recent interactive point-based methods are convenient and realistic, they often lack fine-grained control over localized content. Partial sketches provide a simple yet expressive interface for local structure manipulation. However, existing partial-sketch-based manipulation methods relying on generative adversarial networks (GANs) suffer from limited generalization and fidelity. Moreover, although diffusion-based adapters excel at global conditioning (e.g., edge maps), localized editing with partial strokes remains challenging due to two key issues: effectively injecting sparse stroke conditions during denoising and preserving non-edited regions to avoid unintended changes. To address these challenges, we propose DiffStroke, a mask-free framework for localized image manipulation with partial sketches. We introduce trainable Image-Stroke Fusion (ISF) blocks to fuse source images and strokes at the feature level, enabling precise local shape control while maintaining appearance consistency. We further develop a self-supervised mask estimator to protect irrelevant regions without manual input. Specifically, we leverage Tweedie's formula to estimate a clean latent image from noisy latents, blend the denoised result with the source, and train the mask estimator by minimizing the error between the blended latent and the target latent. Experiments on natural and facial images demonstrate that DiffStroke outperforms state-of-the-art methods on both simple and complex stroke-based editing tasks. DiffStroke can also be combined with text prompts to produce diverse and creative results. Code is available at https://github.com/CMACH508/DiffStroke.
SMFormer: Empowering Self-Supervised Stereo Matching via Foundation Models and Data Augmentation.
Wang Y, Yang Z, Zheng J … +3 more , Zhang Z, Wu DO, Guo Y
IEEE Trans Image Process · 2026 · PMID 42043993 · Publisher ↗
Recent self-supervised stereo matching methods have made significant progress. They typically rely on the photometric consistency assumption, which presumes corresponding points across views share the same appearance. Ho...
Recent self-supervised stereo matching methods have made significant progress. They typically rely on the photometric consistency assumption, which presumes corresponding points across views share the same appearance. However, this assumption could be compromised by real-world disturbances, resulting in invalid supervisory signals and a significant accuracy gap compared to supervised methods. To address this issue, we propose SMFormer, a framework integrating more reliable self-supervision guided by the Vision Foundation Model (VFM) and data augmentation. We first incorporate the VFM with the Feature Pyramid Network (FPN), providing a discriminative and robust feature representation against disturbance in various scenarios. We then devise an effective data augmentation mechanism that ensures robustness to various transformations. The data augmentation mechanism explicitly enforces consistency between learned features and those influenced by illumination variations. Additionally, it regularizes the output consistency between disparity predictions of strong augmented samples and those generated from standard samples. Experiments on multiple mainstream benchmarks demonstrate that our SMFormer achieves state-of-the-art (SOTA) performance among self-supervised methods and even competes on par with supervised ones. Remarkably, in the challenging Booster benchmark, SMFormer even outperforms some SOTA supervised methods, such as CFNet.
INNFusion: A Diffusion-Based Blind Image Super Resolution Scheme Using Reversible Degradation Process With Invertible Neural Networks.
Poudineh M, Esmaeilzehi A, Ahmad MO
IEEE Trans Image Process · 2026 · PMID 42030189 · Publisher ↗
Deep neural networks using generative diffusion prior have provided the state-of-the-art performances for the task of blind image super resolution. Thanks to their powerful image generation capability, these deep network...
Deep neural networks using generative diffusion prior have provided the state-of-the-art performances for the task of blind image super resolution. Thanks to their powerful image generation capability, these deep networks are able to produce high-quality visual signals with realistic textures and structures. However, since these schemes employ a very large number of parameters, their training process is often difficult, and therefore, their performances can be limited. In order to address this, in this paper, we propose a diffusion-based blind image super resolution scheme, which by using a novel learning algorithm with invertible neural networks, is able to provide superior results. Specifically, we argue that because of the reversibility property of invertible neural networks, they are able to generate degraded low-quality images, whose super resolved versions are the upper bound of the image super resolution function space. The inclusion of such visual signals in the training process of our blind image super resolution network leads to facilitating the learning paradigm and achieving higher performances. We show that our proposed blind image super resolution scheme is able to outperform the state-of-the-art methods.
Unsupervised Domain Adaptation-Based Cross-Type Deepfake Image Detection.
Wang Q, Wang X, Liu Z … +3 more , Bai N, Zhao M, Pang S
IEEE Trans Image Process · 2026 · PMID 42030188 · Publisher ↗
In practical applications of social media and the Internet, deepfake face images involve a plethora of unlabeled samples. To effectively identify unlabeled deepfake images, the domain adaptation technique has gained sign...
In practical applications of social media and the Internet, deepfake face images involve a plethora of unlabeled samples. To effectively identify unlabeled deepfake images, the domain adaptation technique has gained significant attention. It applies the knowledge learned from labeled samples (source domain) to unlabeled samples (target domain) in a cross-domain manner. However, the existing domain adaptation-based deepfake detection methods primarily focus on intra-type cross-domain scenarios. In this study, we propose an unsupervised domain adaptation-based deepfake face image detection method for extra-type cross-domain scenarios. The core idea of our approach lies in the development of a domain adaptation model that consists of Domain Tag Adversarial (DTA) and Domain Feature Alignment (DFA) algorithms, called DTA-DFA, which empowers the proposed method with strong cross-domain capability. The DTA is utilized to weaken the specificity within each domain, while DFA aligns the distribution between the source and target domains. Compared with the existing deepfake detection methods, the experimental results demonstrate that the proposed method dramatically enhances the extra-type cross-domain detection performance. Moreover, the DTA-DFA model also exhibits a remarkable ability to perform cross-domain detection from large-shot labeled samples to few-shot labeled samples, further verifying its powerful cross-domain capability. Code is released at https://github.com/QinQin741/DTA-DFA-DA-model.
Mask-Guided Proxy Mining Network for Few-Shot Medical Image Segmentation.
Huang W, Hu J, Wang Y … +5 more , Bi X, Shu Y, Yang X, Zhang Y, Xiao B
IEEE Trans Image Process · 2026 · PMID 42030187 · Publisher ↗
Few-shot medical image segmentation (FSMIS) has attracted increasing attention as a promising technique for solving medical image segmentation tasks by relying on only a small amount of labeled data from new classes. Cur...
Few-shot medical image segmentation (FSMIS) has attracted increasing attention as a promising technique for solving medical image segmentation tasks by relying on only a small amount of labeled data from new classes. Current FSMIS methods typically employ pixel-level semantic correlations between support-query image pairs to guide the segmentation of query images. However, the class information gap between support and query images may induce severe mismatches, leading to semantic ambiguity between foreground and background pixels. To address this issue, we propose a novel mask-guided proxy mining network (MPMNet), which mines a set of representative reference features (termed proxies) from support and query images to rectify foreground-background ambiguity. Specifically, to eliminate false pairwise matches caused by excessive intra-class variations, we design a mask-guided proxy mining module to adaptively learn representative proxies that can perceive visual differences between objects with different scales and shapes. Moreover, we integrate a hierarchical prior generation module and a context-aware feature enrichment module into MPMNet to obtain multi-scale information and enhance the discriminability of features. With these well-designed components and structures, our MPMNet can effectively overcome the adverse effects of false pixel matches by establishing proxy-level semantic correlations. Extensive experiments on three standard medical segmentation benchmarks demonstrate that our MPMNet significantly outperforms previous state-of-the-art methods, with a mean gain of 2.71% in DSC across all datasets. The code is available at: https://github.com/donglongzi/MPMNet.
Multi-Granularity Topological Reasoning for Anatomically Consistent Vasculature Parsing.
Mou L, Liu Y, Xu Z … +5 more , Zhang H, Zheng Y, Liu J, Fu H, Zhao Y
IEEE Trans Image Process · 2026 · PMID 42024936 · Publisher ↗
Quantitative analysis of retinal vascular morphology is vital for clinical decision-making and the investigation of systemic diseases. Central to this process is the accurate segmentation of retinal arteries and veins (A...
Quantitative analysis of retinal vascular morphology is vital for clinical decision-making and the investigation of systemic diseases. Central to this process is the accurate segmentation of retinal arteries and veins (A/V) from the background, a task challenged by substantial variations in vessel calibers and the presence of low-contrast or ambiguous structures in fundus images, especially in ultra-wide field imaging where peripheral distortions and large-scale anatomical variability are pronounced. These factors often lead to fragmented semantic representations and topological inconsistencies in automated segmentation outputs. To address these limitations, we propose Ultra, a multi-granularity topological reasoning network designed for precise A/V segmentation. Ultra adopts a cascaded two-stage architecture: PriorNet generates coarse, multi-scale vascular priors that provide structural guidance, while RefineNet performs topology-aware segmentation refinement. To further enforce topological coherence, we propose the neighboring pixel connectivity regularization (NICER) layer, which selectively integrates local connectivity information predicted by the proposed connectivity prediction union (CPU) module. This connectivity is employed as auxiliary supervision through a pixel-wise local connectivity loss, reinforcing structural reasoning and promoting anatomically consistent vascular topology inference. Extensive experiments on ultra-wide field fundus imaging (UWF) datasets demonstrate that Ultra achieves state-of-the-art performance in A/V segmentation and topological preservation. Moreover, Ultra generalizes well to conventional color fundus photography (CFP) datasets, underscoring its robustness and broad applicability. Code is publicly available at: https://github.com/iMED-Lab/Ultra.
Riemannian Acceleration for Sparse PCA With Separable Structure and Second-Order Information Exploration.
Chen GY, Xu HL, Su XX … +3 more , Gan M, Chen X, Chen CLP
IEEE Trans Image Process · 2026 · PMID 42024935 · Publisher ↗
Sparse Principal Component Analysis (SPCA) is a powerful technique for dimensionality reduction and feature extraction in high-dimensional data, with applications spanning various fields such as computer vision, pattern...
Sparse Principal Component Analysis (SPCA) is a powerful technique for dimensionality reduction and feature extraction in high-dimensional data, with applications spanning various fields such as computer vision, pattern recognition, and data mining. However, the computational intensity of SPCA presents a significant challenge, necessitating the development of efficient and robust algorithms. In this paper, we shed light on the SPCA problem and uncover intriguing structures that enable us to design an efficient algorithm, which we have named SPCA_ACC. Firstly, we identify a separable structure in this problem, which prompts us to draw on the Variable Projection (VP) strategy and generalize it to separable nonlinear problem in Stiefel manifold. This strategy projects out part of the parameters to obtain a reduced problems, allowing the SPCA_ACC algorithm to optimize in a lower-dimensional parameter space. Secondly, we resolve the coupling between different parameters of the SPCA problem in the optimization process on a fixed coordinate-sparsity manifold, which opens the way to the use of second-order Riemannian accelerated VP strategy. Moreover, we systematically analyze the advantages of using VP to solve the SPCA problem from a theoretical perspective, and confirm the local quadratic convergence of our algorithm. Numerical experiments on datasets of different sizes and types demonstrate that our method achieves rapid convergence and significantly reduces computational costs.
Semantic-Aware Multimodal Collaborative Learning for Unsupervised Visible-Infrared Person Re-Identification.
Wang X, Luo S, Liu M … +3 more , Srivastava G, Liu S, Wang Y
IEEE Trans Image Process · 2026 · PMID 42024934 · Publisher ↗
Unsupervised visible-infrared person re-identification (VI-ReID) is challenging due to the significant modality gap between visible and infrared images. Most existing methods rely on one-hot clustering pseudo-labels as s...
Unsupervised visible-infrared person re-identification (VI-ReID) is challenging due to the significant modality gap between visible and infrared images. Most existing methods rely on one-hot clustering pseudo-labels as supervision signals, which often fail to capture the full semantic relationships among samples and are highly susceptible to noise. To address these limitations, we propose a Semantic-aware Multimodal Collaborative Learning (SAMCL) framework for unsupervised VI-ReID. Specifically, a Modality-aware Semantic Fusion (MSF) module is designed to bridge the inter-modality gap by integrating complementary semantic details from both visible and infrared modalities, generating enriched cross-modal supervision signals, for cross-modal collaborative learning. Meanwhile, we present a Dynamic Contrastive Learning (DCL) module to refine intra-modality feature learning by dynamically aligning samples with their neighboring centroids in the feature space, improving clustering reliability and intra-modality feature discrimination. By combining the two modules, SAMCL harnesses multimodal collaboration, minimizes dependence on noisy pseudo-labels, and provides a robust approach to unsupervised VI-ReID. Extensive experiments demonstrate the superiority of our proposed method. For instance, on the SYSU-MM01 dataset, our model achieves a Rank-1 accuracy of 68.68% in the All Search setting, surpassing the state-of-the-art (SOTA) by 3.48%. On the RegDB dataset, it achieves a Rank-1 accuracy of 94.47% in the Visible-to-Infrared setting, outperforming the SOTA by 3.57%. On the LLCM dataset, it achieves a Rank-1 accuracy of 50.6% in the Visible-to-Infrared setting, outperforming the SOTA by 3.7%. The code is available at https://github.com/luoshixi123/SAMCL.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe