IEEE Transactions On Image Processing[JOURNAL]
Sun
200 papers
Image Restoration Learning via Noisy Supervision in Fourier Domain.
Liu H, Liu J, Tan S … +1 more , Lam EY
IEEE Trans Image Process · 2026 · PMID 42275331 · Publisher ↗
Noisy supervision refers to supervising network learning with targets corrupted by noise, encompassing both weakly supervised learning with noisy targets and fully unsupervised denoising using unpaired noisy images. It a...
Noisy supervision refers to supervising network learning with targets corrupted by noise, encompassing both weakly supervised learning with noisy targets and fully unsupervised denoising using unpaired noisy images. It alleviates the data collection burden and enhances the practical applicability of deep learning techniques. Existing methods face two main limitations: they are ineffective at handling noise with long-range correlations, commonly found in real-world scenarios such as low-light imaging and remote sensing, and rely on pixel-wise loss functions that offer limited supervision for image deblurring and super-resolution. This work addresses these challenges by leveraging the Fourier domain, where spatially correlated noise exhibits sparsity and independence, and Fourier coefficients capture global information that enables stronger supervision. We prove that Fourier coefficients of a wide range of noise converge in distribution to the Gaussian distribution and establish a statistical equivalence between learning with clean and noisy targets in the Fourier domain. Based on these insights, we develop a weakly supervised framework for image restoration learning with noisy targets, and construct a fully unsupervised denoising method tailored to stripe-wise noise. Extensive experiments show that our approaches achieve superior performance in both quantitative metrics and perceptual quality.
Robust Discriminant Subspace Learning With α-Divergence for Image Classification.
Zheng H, Seghouane AK, Merad D
IEEE Trans Image Process · 2026 · PMID 42275330 · Publisher ↗
This paper proposes a novel robust Fisher Discriminant Analysis (FDA) for discriminative subspace learning in the presence of outliers. The proposed approach is motivated by the maximum-likelihood perspective on FDA and...
This paper proposes a novel robust Fisher Discriminant Analysis (FDA) for discriminative subspace learning in the presence of outliers. The proposed approach is motivated by the maximum-likelihood perspective on FDA and its connection to Kullback-Leibler (KL) divergence minimization. Within the probabilistic FDA framework, we develop a robust model by adopting the $\alpha $ -divergence as a flexible alternative to the KL divergence. The resulting method induces an adaptive redescending weighting scheme, in which each observation is weighted according to its statistical compatibility with the model, where the robustness level continuously controlled by $\alpha $ . As $\alpha $ decreases from 1, the influence of outliers is progressively suppressed, while classical FDA is recovered at $\alpha = 1$ . Combined with a two-fold iterative optimization procedure, the proposed method mitigates contamination at both the class-modeling stage and the projection-learning stage. We further provide theoretical analysis of the robustness mechanism, convergence, and computational complexity analysis to support the effectiveness and efficiency of the proposed method. Extensive experiments on synthetic data and multiple public image datasets under diverse contamination settings demonstrate that the proposed method consistently outperforms representative robust FDA variants and related approaches.
P3C-DNet: Pseudo-Groundtruth Contrastive Learning With Color Calibration Dehazing Network.
Zhao H, Ouyang Z, Meng W … +3 more , Rosin PL, Lai Y, Wang Y
IEEE Trans Image Process · 2026 · PMID 42275329 · Publisher ↗
Existing dehazing methods primarily rely on synthetic hazy images for supervised learning. While effective on synthetic datasets, these methods often struggle to generalize to real-world hazy images, leading to issues su...
Existing dehazing methods primarily rely on synthetic hazy images for supervised learning. While effective on synthetic datasets, these methods often struggle to generalize to real-world hazy images, leading to issues such as color distortion and incomplete haze removal. Moreover, their limited adaptability to real-world datasets and inability to handle complex haze scenarios remain significant challenges. To address these limitations, we propose a novel unsupervised framework P3C-DNet (Pseudo-groundtruth Contrastive learning with Color Calibration Dehazing Network). Our P3C-DNet introduces a Pseudo-groundtruth image generation strategy through the Pseudo-groundtruth Contrastive Supervision (PCS) module, which overcomes the lack of real haze-free training data by generating high-quality Pseudo-groundtruth images. To further refine the dehazing process, we incorporate a codebook-based image coding and matching mechanism that aligns Pseudo-groundtruth images with hazy inputs, enhancing the accuracy and detail of the dehazed outputs. To address the prevalent issue of color distortion, especially in complex environments, our P3C-DNet integrates a Dynamic Color Restoration Block (DCRB) to ensure visual quality and color consistency in the dehazed results. Experimental evaluations demonstrate that our P3C-DNet achieves superior performance in haze removal, color fidelity, and detail preservation, significantly outperforming existing methods and setting a new benchmark for real-world dehazing tasks.
Synergistic Prompting for Complementarity and Consistency in Incomplete Multi-View Clustering.
Hao X, Zhang Z, Tang Y … +5 more , Zhang L, Hao P, Diao Y, Jin G, Liu Y
IEEE Trans Image Process · 2026 · PMID 42275328 · Publisher ↗
Incomplete multi-view clustering (IMVC) aims to partition unlabeled multi-view data into semantically coherent groups, even when certain views are missing due to sensor failures, data collection constraints, or privacy c...
Incomplete multi-view clustering (IMVC) aims to partition unlabeled multi-view data into semantically coherent groups, even when certain views are missing due to sensor failures, data collection constraints, or privacy concerns. Despite advancements in deep IMVC methods, two critical challenges remain unresolved: (i) the lack of explicit mechanisms to model cross-view complementarity and (ii) the absence of principled strategies to ensure global semantic consistency across views. To address these challenges, we propose SP-IMVC, a novel Synergistic Prompting framework that jointly models complementarity and consistency under view incompleteness. Specifically, we introduce two types of learnable prompts: the Cross-View Complementary Prompt (CVCP), which aggregates auxiliary representations from available views to enrich the semantics of the current view and mitigate information loss; and the Latent Anchor Prompt (LAP), which utilizes a global anchor prompt pool to provide adaptive semantic priors that promote globally consistent representations. These prompts are optimized jointly within a unified architecture to achieve synergistic prompting of cross-view complementarity and global semantic consistency. Extensive experiments on six public benchmarks demonstrate that SP-IMVC consistently outperforms 14 state-of-the-art IMVC approaches, particularly in scenarios with high missing-view ratios, validating the effectiveness and robustness of our synergistic prompt-guided clustering framework. The code will be released to facilitate future research.
Global and Local Visual-Textual Alignment for Open Vocabulary Object Detection.
Wang H, Jia T, Deng S … +3 more , Chen D, Wang Q, Zuo W
IEEE Trans Image Process · 2026 · PMID 42275327 · Publisher ↗
Recently, with the development of the Vision-Language Model (VLM), adopting such VLM (e.g., CLIP) into object detection framework has gradually become a promising and attractive research direction, and the resulted open...
Recently, with the development of the Vision-Language Model (VLM), adopting such VLM (e.g., CLIP) into object detection framework has gradually become a promising and attractive research direction, and the resulted open vocabulary object detection methods can effectively alleviate the limitations in those close-set ones, making the detectors perceive the unseen world. The core issue in open vocabulary object detection is to design an effective and efficient alignment between the visual (e.g., image) and textual (e.g., caption) features in the semantic space, so that the detectors can capture more information around the open-set scene. Current approaches deploy extra uncurated image-text pairs to pre-train a detector for obtaining a better visual-textual alignment in the feature space. Besides, knowledge distillation technology is also adopted to design an appropriate information transferring flow for aligning the visual-textual knowledge. However, large-scale image-text pairs are not always available to obtain, and the pre-training process will inevitable introduce much more computation overhead. While knowledge distillation methods focus on aligning between the local region visual feature in RoI and the textual features of VLM, neglecting the global information alignment between the image and text. For addressing the dilemmas in these alignment manners, we propose a Global and Local Visual-Textual Alignment for Open Vocabulary Object Detection in this paper. Specifically, our proposed method integrates global image-caption and local region-prompt alignments into a unified learning paradigm. The global alignment takes the whole image and caption as the visual and textual inputs, respectively, and matches the image and caption representations from the detector and the text encoder in CLIP by contrastive learning from the overall perspective. Different from global alignment, the local one concentrates on the accordance between regions and prompts from the aspect of portion description. It extracts and aligns the embeddings for the visual patch RoIs from the image encoder in CLIP and discriminating textual token prompts from the text encoder. Moreover, we also design a prompt tuning strategy, which contains global and local components corresponding to the alignment procedure, for better adapting CLIP to downstream task object detection in a parameter-efficient learning manner. By implementation on Faster R-CNN, we conduct experiments on open vocabulary benchmarks OV-COCO and OV-LVIS, respectively. The results verify that our proposed method can achieve clear improvement over counterparts on novel categories, while performing favorably against state-of-the-arts.
VC-VTON: Toward Across-View and Multi-Posture-Driven Virtual Try-On via Spatiotemporal-Aware View-Consistency Training.
Ma Z, Wei J, Cai Z … +5 more , Yang C, Hua E, Xie S, Li J, Zhou B
IEEE Trans Image Process · 2026 · PMID 42275326 · Publisher ↗
Virtual try-on (VTON) aims to synthesize specific fashion images dressed in given garments, which possesses great potential in real-world scenarios. Existing methods generally stand on the shoulder of the single-view VTO...
Virtual try-on (VTON) aims to synthesize specific fashion images dressed in given garments, which possesses great potential in real-world scenarios. Existing methods generally stand on the shoulder of the single-view VTON to train a warping model and then fit the given garments onto the human body under a fixed posture and viewpoint, which often fails to preserve the consistent garment characteristics in across-view and multi-pose guided try-on scenarios due to the lack of both across-view data and effective view consistency training. To alleviate this dilemma, we propose a fresh view consistency-driven VTON task (VC-VTON) and release a multi-view virtual try-on dataset with complete annotation (e.g., viewpoint, text, posture, parsing maps, etc.) to encourage across-view training scenarios. Based on this hard-won dataset, we further propose VC-TwinNet, a Twin-UNet baseline based on spatiotemporal-aware View Consistency training, designed specifically for the challenging task. Specifically, to enable view-aware denoising and sparse-to-continuous view generalization, we introduce RoPE and circle embedding to represent the relative and continuous position relation across viewpoints, serving to distinguish their outfitting appearance and warping states. Afterwards, to implicitly learn the interactions across views under given multiple posture conditions, we further contribute a spatiotemporal-aware view attention module to capture the spatial and temporal details for across-view training. Moreover, we utilize an across-view consistency loss to supervise the model training, to ultimately improve the performance of our VC-VTON. Extensive experiments demonstrate the superiority of our approach and state-of-the-art results on various evaluations without declining single-view performance. And as for practicality and timeliness, our proposed components are essentially plug-and-play and remain effective in the new DiT-centered paradigm.
FACT: A Simple and Efficient Framework for Active Finetuning.
Xu W, Song Y, Cui Y … +3 more , Ren M, Liu Q, Hu Z
IEEE Trans Image Process · 2026 · PMID 42275325 · Publisher ↗
The main goal of active finetuning is to improve a pretrained model's performance on a specific task or domain by finetuning it with carefully selected informative or challenging data. Previous research has predominantly...
The main goal of active finetuning is to improve a pretrained model's performance on a specific task or domain by finetuning it with carefully selected informative or challenging data. Previous research has predominantly focused on the active aspect (i.e., data selection) while uniformly employing full finetuning for model adaptation, which inevitably distorts pretrained features due to distribution shift. This issue becomes particularly pronounced when the model size is large relative to the finetuning data quantity, leading to heightened overfitting risks. To address this critical gap, we formally outline the FiAF task that emphasizes systematic exploration of finetuning methodologies in active learning. We propose FACT, a three-phase hierarchical finetuning framework featuring both efficiency and simplicity, specifically designed for active finetuning scenarios. Our comprehensive experiments span: 1) Three major dataset categories encompassing classic (CIFAR10, CIFAR100, ImageNet-1k), imbalanced (CIFAR10-LT, CIFAR100-LT), and fine-grained (StanfordCars, FGVCAircraft) image classification datasets, each evaluated under 3-5 distinct sampling ratios; 2) Diverse pretrained architectures including Convolutional Neural Network (ConvNeXt), Vision Transformer (ViT), and Vision LSTM (ViL) networks; 3) A systematic investigation of frozen feature augmentation (FroFA) strategies. 4) A comprehensive and rigorous analysis of efficiency and generalizability. The results demonstrate significant improvements with strong generalization and robustness. Notably, under low sampling ratios, our framework achieves remarkable performance gains of over 20% on the ViT model for CIFAR10, CIFAR100, and ImageNet-1k benchmarks. This systematic approach establishes new state-of-the-art performance while maintaining parameter efficiency, proving particularly effective when labeled data is scarce.
Human-Structure-Aware Token Position Embedding for Tokenized Pose Estimation.
Gu Z, Zhao ZQ, Ding H … +4 more , Shen H, Tang Z, Zhang Z, Huang DS
IEEE Trans Image Process · 2026 · PMID 42275324 · Publisher ↗
Tokenized pose estimation (TPE) has demonstrated remarkable performance in lightweight human pose estimation (HPE) models. However, existing TPE methods typically initialize keypoint tokens randomly, without explicitly i...
Tokenized pose estimation (TPE) has demonstrated remarkable performance in lightweight human pose estimation (HPE) models. However, existing TPE methods typically initialize keypoint tokens randomly, without explicitly incorporating human structure priors. These priors play a vital role in HPE by effectively mitigating common challenges such as occlusion and ambiguity. To this end, we propose a Structure-Aware Keypoint Position Embedding (SAKPE). This embedding explicitly encodes inherent structural properties of the human body, such as symmetry and order, into the positional coordinates of keypoint tokens. It also employs learnable scale and offset factors to adapt to diverse human poses, thereby fully exploiting the geometric constraints among keypoints. Furthermore, to better leverage the positional relationships among patch tokens, we introduce a Layer-adaptive Hybrid Patch Position Embedding (LHPPE). It dynamically fuses absolute and relative position embeddings of patch tokens based on attention distributions across Transformer layers, enabling the model to learn both absolute and relative positional information adaptively. Taking the two together, we propose a novel position embedding method for pose estimation, named Human-structure-aware Token Position Embedding (HTPE). It significantly improves the performance of various TPE models. Extensive experiments on COCO, CrowdPose, and OCHuman show that HTPE achieves state-of-the-art (SOTA) performance among lightweight methods, with a negligible increase in parameters and FLOPs. Notably, it demonstrates consistent improvements under occlusion,, achieving up to 3.3 AP gains. The source code can be found in https://github.com/guzejungithub/HTPE.
AirDC: Adaptive Iterative Depth Refinement Framework for Full-Range Metric Depth Completion.
Shi H, Du Y, Zhang H … +4 more , Li W, Dong Z, Li B, Tang L
IEEE Trans Image Process · 2026 · PMID 42275323 · Publisher ↗
Accurate metric depth completion across wide depth ranges is critical for autonomous systems. However, existing methods often struggle to efficiently capture depth features at both close and long ranges, primarily due to...
Accurate metric depth completion across wide depth ranges is critical for autonomous systems. However, existing methods often struggle to efficiently capture depth features at both close and long ranges, primarily due to the inadequate modeling of fine-grained depth cues specific to different depth ranges. To address these limitations, we propose AirDC, an adaptive iterative depth refinement framework for full-range metric depth completion. The core contributions of our model lie in the design of two key modules. Specifically, we first construct an adaptive fine-grained stereo-LiDAR feature fusion module to fundamentally strengthen the model's capacity to preserve original full-range depth information. Built upon metric-aligned depth volumes (i.e., a 3D representation composed of cubic voxels uniformly partitioned in real-world metric space), this module employs an adaptive sub-voxel depth attention mechanism to enhance sensitivity to subtle depth variations across the full range, thereby both avoiding long-range accuracy degradation introduced by conventional disparity conversion and alleviating the coarse near-range granularity inherent in metric depth representations. Second, we introduce an iterative hypothesis-guided depth refinement module to improve prediction accuracy while maintaining memory efficiency. By integrating multi-scale multi-modal guidance information from depth hypotheses, this module enables explicit and progressive refinement of the initial depth estimation with a small parameter overhead. Experiments on multiple mainstream real-world and synthetic benchmarks demonstrate that AirDC achieves state-of-the-art performance, providing an effective solution for full-range metric depth completion. The code and data are available at https://github.com/yunqidu/AirDC.
DSPFusion: Image Fusion via Degradation and Semantic Dual-Prior Guidance.
Tang L, Li C, Wang Y … +3 more , Wang G, Yuan Y, Ma J
IEEE Trans Image Process · 2026 · PMID 42275322 · Publisher ↗
Existing infrared-visible image fusion methods are mainly tailored for high-quality source images. Although recent studies have begun to explore degradation-aware fusion, most existing methods still focus on specific deg...
Existing infrared-visible image fusion methods are mainly tailored for high-quality source images. Although recent studies have begun to explore degradation-aware fusion, most existing methods still focus on specific degradation types, while unified frameworks that aim to handle diverse degradations often depend on auxiliary textual prompts, which limits their practicality in automatic fusion scenarios. This work presents a Degradation and Semantic Prior dual-guided framework for degraded image Fusion (DSPFusion), which jointly performs degradation-aware restoration and complementary information aggregation in a unified architecture without relying on auxiliary prompts. Specifically, it first extracts modality-specific degradation priors from degraded infrared and visible images, while capturing compact semantic embeddings from paired source images as low-quality semantic priors to encode global scene context. Then, a semantic prior diffusion model is devised to restore high-quality scene semantic priors in a compact latent space, providing global scene guidance with low computational overhead and enabling over $30\times $ inference speedup compared with mainstream diffusion model-based image fusion schemes, such as DDFM. Guided by the restored semantic priors and degradation priors, the enhancement and fusion network adaptively suppresses degradations and aggregates complementary information. Extensive experiments under both degraded and normal scenarios demonstrate that DSPFusion effectively handles representative degradations, preserves complementary information, and achieves competitive performance with low computational cost, thereby broadening the practical application scope of image fusion. The source code is publicly available at https://github.com/Linfeng-Tang/DSPFusion.
HiSymGeo: Hierarchical Context Symbiosis for Cross-View Object-Level Image Geo-Localization.
Chen C, Chen Q, Ye M … +1 more , Zhang X
IEEE Trans Image Process · 2026 · PMID 42275321 · Publisher ↗
Cross-view object-level image geo-localization (CVOIGL) aims to locate ground/drone-view query objects in satellite imagery. This task confronts two obstacles, namely view differences from imaging platform viewpoint chan...
Cross-view object-level image geo-localization (CVOIGL) aims to locate ground/drone-view query objects in satellite imagery. This task confronts two obstacles, namely view differences from imaging platform viewpoint changes and detection ambiguities from similar objects in large-scale satellite views. Existing methods typically employ uniform feature processing across objects while overlooking query-reference cross-view differences, leading to compromised localization precision when handling structurally analogous objects with scale variations. In this paper, we propose HiSymGeo, a Hierarchical Context Symbiosis framework with dual cooperative learning, achieving cross-view representation alignment and structural ambiguity resolution. Specifically, to mitigate cross-view differences, the Diversified View Enhancer (DiVE) first incorporates context-aware query enhancement for ground/drone-view representation while constructing scale-agnostic reference enhancement in satellite views to handle scale variations. These view-specific features then undergo contrastive learning via semantic-aware matching to align query and reference representations. Furthermore, the Query-Gated Multi-Expert View Fusion (QG-MEVF) introduces dynamic expert routing via multi-scale pyramidal representations, in which a Mixture-of-Experts (MoE) inspired architecture employs query-driven gating to adaptively select scale-specific fusion expert. This differentiable routing mechanism boosts structural discrimination against analogous objects, enabling precise object localization. Extensive ablation experiments demonstrate HiSymGeo's superiority, achieving state-of-the-art effectiveness while ensuring high cross-dataset generalization. We have released our code at https://github.com/chenqi142/HiSymGeo.
Spectral State Fusion Tree Mamba for Hyperspectral Image Classification.
Tu B, Hu Z, Liu B … +1 more , He Y
IEEE Trans Image Process · 2026 · PMID 42275320 · Publisher ↗
Hyperspectral image (HSI) data possess complex spatial structures and high-dimensional spectral information. Mamba has been applied to address the limitations of general methods in HSI classification, including restricte...
Hyperspectral image (HSI) data possess complex spatial structures and high-dimensional spectral information. Mamba has been applied to address the limitations of general methods in HSI classification, including restricted receptive fields and high computational complexity. However, the scan mechanism of traditional Mamba unreasonably constructs spatial distance relationships between neighboring row pixels and fails to adaptively construct the optimal scanning path based on the spectral similarity of pixels. Additionally, the characteristic of traditional Mamba scanning each channel independently overlooks the feature extraction from high-dimensional spectral information. This work proposes a Spectral State Fusion Tree Mamba (SSFTM) architecture to resolve these limitations. The Tree Scan (TS) mechanism computes cosine distances among spatial neighboring pixels and spectral channels to construct adaptive minimum spanning trees in both the spatial and spectral domains, thereby establishing reasonable spatial-spectral relationships and enabling efficient joint feature extraction. The Spectral State Fusion (SSF) mechanism applies multi-layer one-dimensional dilated convolutions along the spectral dimension to the state space vectors, enabling inter-channel interaction and promoting multi-scale spectral feature extraction. The proposed SSFTM demonstrates superior classification accuracy across multiple datasets compared to SOTA methods and exhibits acceptable computational complexity. The code is available at https://github.com/copawloroous/SSFTM.
Learning Occlusion-Dynamic Invariant Representations for Multi-Object Tracking.
Li M, Hu H, Jain DK … +2 more , Niu B, Zhao X
IEEE Trans Image Process · 2026 · PMID 42275319 · Publisher ↗
Robust multi-object tracking (MOT) is hindered by the instability of appearance features under visual corruptions such as occlusion and motion blur. These perturbations introduce high-variance noise into feature trajecto...
Robust multi-object tracking (MOT) is hindered by the instability of appearance features under visual corruptions such as occlusion and motion blur. These perturbations introduce high-variance noise into feature trajectories, weakening temporal representations and causing identity switches. We address this challenge by learning more stable appearance representations under feature corruption. To this end, we propose the Causal Interaction Module (CIM), a causal architecture that follows a filter then reconstruct design for online tracking. A temporal filtering stage summarizes the historical feature trajectory into a stable anchor, and a contextual enhancement stage uses that anchor to refine frame-level features before association. Integrated into standard trackers, CIM improves association robustness while preserving the host tracking formulation. Experiments on multiple MOT benchmarks and corruption stress tests show consistent gains, especially on association-related metrics.
DDMPI: Diffusion Denoising for Magnetic Particle Imaging at the Low Concentration.
Guo L, An Y, Ma C … +4 more , Li J, Dong Z, Tian J, Liu J
IEEE Trans Image Process · 2026 · PMID 42275318 · Publisher ↗
Magnetic particle imaging (MPI) has demonstrated its advantages of high sensitivity and temporal resolution in various preclinical applications. However, during the imaging process, the signal is susceptible to different...
Magnetic particle imaging (MPI) has demonstrated its advantages of high sensitivity and temporal resolution in various preclinical applications. However, during the imaging process, the signal is susceptible to different noises, resulting in severe stripe artifacts in reconstructed MPI images. This phenomenon will be further aggravated in scenarios with low-concentration particles, which is a standard practice in biological applications, thereby seriously hindering the identification of key information. To solve this problem, we propose a joint optimization approach called diffusion denoising model for MPI (DDMPI) that integrates diffusion model with Transformer to remove the artifacts directly from MPI images obtained in the low-concentration scenarios. In DDMPI, a latent encoder generates prior features containing the relevance mapping between the contents and the artifacts within MPI images, and a conditional latent diffusion model optimizes these prior features. A U-shape Transformer module incorporates the prior features by a hierarchical integration module and utilizes a strip-self-attention module to capture the spatial distribution of the artifacts. Ablation experiments demonstrate the effectiveness of these modules in DDMPI. Extensive experiments, including simulation, phantom and in vivo experiments, demonstrate that DDMPI effectively removes artifacts and recovers fine details. Additionally, DDMPI is independent of the primary image reconstruction methods of various scanning devices. Thus, DDMPI can be not only practically applied to in vivo imaging but also flexibly combined with various existing MPI devices to effectively improve the imaging quality and provide critical information about diseases.
DRDFNet: A Degradation-Aware Restoration and Detail-Preserving Fusion Network for Infrared and Visible Image.
Liu P, Wei A, Zhang C … +5 more , Chen Z, Feng C, Lu F, Hu W, Lu K
IEEE Trans Image Process · 2026 · PMID 42275317 · Publisher ↗
Multi-source image fusion combines infrared and visible information to improve scene perception in applications such as drone reconnaissance and autonomous driving. However, most existing infrared-visible image fusion me...
Multi-source image fusion combines infrared and visible information to improve scene perception in applications such as drone reconnaissance and autonomous driving. However, most existing infrared-visible image fusion methods are developed under ideal imaging assumptions. In adverse environments, visible images often lose structural and textural details, whereas infrared images are affected by noise, stripe artifacts, and low contrast, leading to degraded fusion quality and weakened downstream perception performance. To address these limitations, we propose a unified Degradation-aware Restoration and Detail-preserving Fusion Network (DRDFNet), which consists of a Degradation-Aware Restoration Transformer and a Detail-Preserving Fusion Mamba. The restoration branch uses a Compound Degradation Restoration Module (CDRM) to remove complex degradations, while the fusion branch employs a Dynamic Feature Fusion Module (DFFM) to integrate local complementary cues and global correlations across modalities. A two-stage training strategy is further introduced to reduce the optimization conflict between restoration and fusion. In addition, we construct DIVIF, a large-scale degraded IVIF benchmark generated by a physics-based imaging simulator. Experiments on the DIVIF and AWMM-100k benchmarks demonstrate that DRDFNet achieves robust and competitive performance compared with SOTA methods. Both the dataset and source code will be made publicly available at https://github.com/Liupeng97/DRDFNet.
Selection, Aggregation, and Enhancement: Trajectory Consistent Diffusion Model for Image Super-Resolution.
Huang D, Guo Y, Huang Y … +3 more , Dai L, Shen F, Zeng H
IEEE Trans Image Process · 2026 · PMID 42268750 · Publisher ↗
Diffusion models have shown strong promise for image super-resolution (ISR). However, current approaches often underuse pretrained diffusion backbones and lack constraints on the sampling trajectory, which degrades struc...
Diffusion models have shown strong promise for image super-resolution (ISR). However, current approaches often underuse pretrained diffusion backbones and lack constraints on the sampling trajectory, which degrades structural consistency and fine details. For that, we introduce the trajectory consistent diffusion model (TCDM) for super-resolution, which jointly optimizes the sampling process through lightweight components and inference-time strategies while keeping the diffusion backbone frozen, yielding high-fidelity, detail-rich reconstructions. First, we propose a dynamic semantic selection (DSS) mechanism that records early intermediates, matches them to upsampled low-resolution features, and reconditions sampling with the best match to reduce the mismatch between conditioning and noise scale. Next, we design a cross-step aggregation guidance (CAG) strategy that aggregates features from the current state with the selected intermediate to enforce trajectory-level consistency in noise prediction. Finally, we present a plug-and-play frequency enhancement adapter (FE-Adapter) that injects different frequency-domain cues into the encoder during training, strengthening high-frequency perception while preserving global structures. Extensive experiments on multiple ISR benchmarks show that TCDM achieves strong structural fidelity and competitive no-reference perceptual quality, offering a favorable fidelity-perception trade-off.
NOTO: Noise-Tolerate Evidential Learning for Open-Set Cross-Modal Retrieval.
Pu R, Su C, Hu P … +3 more , Ren Z, Peng D, Sun Y
IEEE Trans Image Process · 2026 · PMID 42268749 · Publisher ↗
With the increasing accessibility of multimodal data, cross-modal retrieval (CMR) has gained significant attention in recent years. However, most existing CMR methods are built on clean annotations and closed-set label s...
With the increasing accessibility of multimodal data, cross-modal retrieval (CMR) has gained significant attention in recent years. However, most existing CMR methods are built on clean annotations and closed-set label space assumptions, which are often violated in practice. In realistic scenarios, annotations are often noisy due to machine-generated or non-expert labeling, while new categories may also emerge from heterogeneous data sources. The coexistence of label noise and open-set categories gives rise to open-set noisy labels (OSNL). Compared to closed-set noise, OSNL is more harmful because it arises from samples whose true categories lie outside the training label space. When such unknown-class samples are incorrectly assigned to known labels, the model cannot correct them through label relationships. Instead, the model is forced to learn erroneous semantic associations, embedding unknown semantics into incorrect categories. This bias gradually accumulates and disrupts the semantic structure of the shared representation space, ultimately causing existing CMR methods to struggle to maintain reliable performance. To address these challenges, this paper proposes NOise-TOlerate evidential learning (NOTO), a novel framework that robustly learns cross-modal representations under both closed-set and open-set noisy labels. Specifically, a Robust Evidential Learning (REL) module is proposed to detect clean, closed-set noisy, and open-set noisy instances by modeling the predictive distribution as Dirichlet evidence and inferring belief masses. Based on these inferred instance types, REL then assigns tailored optimization strategies to enhance semantic consistency and enlarge the discrimination margin between in-distribution data and open-set categories. An Adaptive Noise-aware Contrast (ANC) module is proposed to adaptively select reliable positive pairs according to the estimated noise states and maximize the mutual information between them to strengthen cross-modal alignment and mitigate the adverse effects of noisy supervision simultaneously. Extensive experiments and comparisons with ten state-of-the-art CMR methods on four benchmarks demonstrate that NOTO achieves superior retrieval performance and robustness against open-set noisy labels. The code is available at https://github.com/perquisite/NOTO.
Lens Privacy Sealing: A New Benchmark and Method for Physical Privacy-Preserving Action Recognition.
Liu M, Wang Z, Li P … +1 more , Yuan J
IEEE Trans Image Process · 2026 · PMID 42262946 · Publisher ↗
RGB camera-based surveillance systems enable human action recognition for public safety and healthcare, yet raise serious privacy concerns. Existing methods rely on post-capture algorithms, which fail to protect privacy...
RGB camera-based surveillance systems enable human action recognition for public safety and healthcare, yet raise serious privacy concerns. Existing methods rely on post-capture algorithms, which fail to protect privacy during data acquisition. We propose Lens Privacy Sealing (LPS), a simple hardware solution that physically obscures camera lenses with adjustable laminating film, providing pre-sensor privacy protection at minimal cost. Unlike software methods or expensive engineered optics, LPS achieves strong privacy through stochastic multi-layer scattering that is physically irreversible. We introduce the P3AR dataset for privacy-preserving action recognition, featuring both large-scale replay-captured (P3AR-NTU, 114K videos) and real-world collected (P3AR-PKU) subsets with privacy attribute annotations. To handle video degradation from LPS, we propose MSPNet, a single-stage framework incorporating Inter-Frame Noise Suppressor (IFNS) and Cross-Frame Semantic Aggregator (CFSA), enhanced by contrastive language-image pre-training for robust semantic extraction. Extensive experiments demonstrate that MSPNet with IFNS and CFSA nearly doubles action recognition accuracy compared to baseline methods while suppressing identity recognition to low levels. Comprehensive validation shows LPS achieves a superior privacy-utility trade-off compared to state-of-the-art hardware methods, resists reconstruction attacks including PSF inversion and data-driven recovery, and generalizes robustly across optical configurations and challenging environments. Code is available at https://github.com/wangzy01/MSPNet.
CondFoodGen: A Conditional Two-Stream Network for Controllable Food Image Generation.
Zhao M, Xiong H, Min W … +3 more , Hou S, Zhang M, Jiang S
IEEE Trans Image Process · 2026 · PMID 42258684 · Publisher ↗
Food image generation is an important research direction in food computing, aiming to produce highly realistic images that accurately capture the visual characteristics of various dishes while adhering to specified input...
Food image generation is an important research direction in food computing, aiming to produce highly realistic images that accurately capture the visual characteristics of various dishes while adhering to specified input conditions. Existing methods that rely solely on textual descriptions struggle to handle the large intra-class variability of food, often resulting in limited diversity and accuracy. Although some approaches incorporate additional conditions, they generally lack optimizations for food-specific challenges, leading to inconsistencies in texture, shape, and color fidelity. To address these limitations, we propose CondFoodGen, a diffusion-based two-stream network for controllable food image generation. The architecture consists of a control stream and a generation stream, where the control stream provides conditional guidance to regulate the generation process. To optimize bidirectional interactions between the two streams, we introduce the Bidirectional Adaptive Gating (BAG) mechanism, which not only guides synthesis but also adaptively refines control representations through feedback from the generation stream. In addition, we propose the Wavelet-Guided Hierarchical Attention (WGHA) module, which combines wavelet-based multi-frequency analysis with hierarchical attention to enhance fine-grained texture fidelity and structural realism. A progressive multi-stage training strategy further stabilizes optimization and enables seamless integration of conditional guidance with bidirectional interaction. Extensive experiments on three food image datasets demonstrate that CondFoodGen consistently generates high-quality and diverse images. Compared with the best existing food image generation methods, our approach achieves an average improvement of about 11.0% across three evaluation metrics and compared to the leading conditional generation approaches, the average improvement reaches 16.2%. The source code, trained models, and supplementary materials are publicly available at https://github.com/housujuan123/CondFoodGen.
Language Supervised Multi-Camera Multi-Object Tracking.
Mao K, Hong X, Fan X … +1 more , Zuo W
IEEE Trans Image Process · 2026 · PMID 42258683 · Publisher ↗
Recent multi-camera multi-object tracking (MCMOT) algorithms are primarily trained using per-detection identity annotations, which are complicated to obtain. In contrast, labeling a language description per-object is a m...
Recent multi-camera multi-object tracking (MCMOT) algorithms are primarily trained using per-detection identity annotations, which are complicated to obtain. In contrast, labeling a language description per-object is a more natural and human-friendly way. In this paper, we explore MCMOT in a language-supervised manner (LS-MCMOT) and propose a novel approach LaVST, which performs language-to-vision weakly-supervised learning based on reliable pseudo-labels generated via tracklet-level cross-modality matching. In addition, we design an ID-aware projection self-correction mechanism to correct inaccurate image-to-ground projection in a self-supervised manner. The models trained with our approach exhibit promising performance in LS-MCMOT. Surprisingly, they perform favorably against state-of-the-art identity-supervised methods, especially in cross-dataset evaluation (with an average gain by 20.0% in IDF1), underscoring the potential of language annotations in MCMOT. Codes and language annotations will be available here.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe