IEEE Transactions On Image Processing[JOURNAL]
Sun
200 papers
MaPPA: Multimodal Controllable Person Image Generation with Pose and Appearance Guidance.
Ye M, Dong Y, Wang T … +3 more , Zhou D, Tan Q, Du B
IEEE Trans Image Process · 2026 Jun · PMID 42228657 · Publisher ↗
Person image generation has become an increasingly important problem in computer vision with broad applications in virtual try-on, digital content creation, entertainment, and human-computer interaction. Despite recent a...
Person image generation has become an increasingly important problem in computer vision with broad applications in virtual try-on, digital content creation, entertainment, and human-computer interaction. Despite recent advances in diffusion-based generative models that can produce photorealistic results, existing pipelines are still constrained by rigid modality requirements. Most prior methods rely on fixed patterns such as pose-plus-appearance inputs or text-only descriptions, limiting their flexibility and controllability in practical scenarios where users may prefer different or mixed types of guidance. This lack of adaptability poses a clear barrier to deployment in real-world systems. To overcome these challenges, we propose MaPPA (Multimodal Controllable Person Image Generation with Pose and Appearance Guidance), a unified multimodal framework that leverages transformer-based latent diffusion models. The central idea is to provide composable control over multiple modalities, enabling person image generation conditioned on text prompts, reference appearance images, pose keypoints, or any combination thereof. To achieve this, we introduce a unified framework that incorporates dedicated control blocks for appearance and pose guidance, which are strategically interleaved with transformer base blocks. These control blocks are conditioned on a unified multimodal embedding that integrates heterogeneous inputs into a consistent representation, thereby supporting arbitrary modality combinations with a single unified pipeline. Another key contribution of MaPPA is a cumulative classifier-free guidance strategy that enables allowing users to independently adjust the strength of appearance and pose guidance via scalar weights from different control signals. This design allows users to adjust the relative strength of appearance versus pose guidance, providing fine-grained controllability during inference. Furthermore, to address the common problem of detail loss in latent-diffusion decoders, we propose a texture enhancement decoding (TED) strategy, which fine-tunes the VAE decoder with edge-aware reconstruction objectives. This refinement significantly alleviates texture distortion, preserving high-frequency details in clothing patterns, facial regions, and other fine structures. Extensive experiments confirm that MaPPA achieves competitive quantitative scores while providing superior perceptual quality and user preference. Unlike task-specific methods, our framework-though constrained by the capacity of the unified multimodal embedding-supports combinations of text, pose, and appearance modalities within a single pipeline, demonstrating both flexibility and practical value. The code and the corresponding model will be made publicly available to the research community.
Event-Aware Instructed Assistant for Referring Video Segmentation.
Liu J, Ding H, He S … +1 more , Jiang YG
IEEE Trans Image Process · 2026 · PMID 42228656 · Publisher ↗
Existing referring video segmentation methods often treat a video as a single event consisting of multiple images, overlooking the fact that a video typically contains multiple distinct events. Under such a mechanism, th...
Existing referring video segmentation methods often treat a video as a single event consisting of multiple images, overlooking the fact that a video typically contains multiple distinct events. Under such a mechanism, the model needs to directly understand all the complex content in the video and text, which can easily lead to confusion and hallucinations. To address this issue, we propose to decompose a video to a set of simple events by learnable Event Query, and understand complex video content in an event-by-event, easy-to-understand manner. This is based on the observation that natural language expressions often divide a video into distinct, text-related segments, each representing a separate event within a compound event. We introduce EVIS, an Event-Aware Video Instructed Segmentation Assistant, which utilizes text-guided Event Queries to partition a video into simple events, extracting event-aware visual-text features to achieve a hierarchical understanding of the video. Additionally, we propose Object-Pixel-Hybrid Learning, which enables the MLLMs to track targets in long-term videos by integrating fine-grained pixel features with prior object queries. Extensive experimental results on 5 public benchmarks demonstrate EVIS's strong performance in addressing the referring video segmentation task. Code and trained models will be publicly released.
Spatial-Temporal Self-Compensating Graph Convolutional Network for Skeleton-Based Action Recognition Under Data Constraints.
Li X, Geng Q, Huang Q … +3 more , Li X, Tang J, Ye Q
IEEE Trans Image Process · 2026 · PMID 42224318 · Publisher ↗
Skeleton-based human action recognition has emerged as a prominent research focus in computer vision, with significant progress achieved in recent years. However, existing methods often suffer substantial performance deg...
Skeleton-based human action recognition has emerged as a prominent research focus in computer vision, with significant progress achieved in recent years. However, existing methods often suffer substantial performance degradation under real-world data constraints, such as body occlusion, missing frames, and noise. These limitations critically undermine the robustness of related techniques in practical applications. To address these challenges, we propose a Spatial Temporal Self-compensating Graph Convolutional Network (STSc-GCN), which skillfully utilizes the systematic and regular nature of human movement to mitigate performance degradation caused by data constraints through a data self-compensation mechanism. Specifically, STSc-GCN comprises two key modules: 1) collaborative motion spatial compensation (CMSC). This module designs multiple distinct topological relationships, primarily including Walk-probability Generality Topology and Self-organizing Particularity Topology, respectively, to deeply explore the universal and personalized collaborative relationships between human joints. These relationships help compensate for the lack of information caused by spatial data constraints and 2) meta-action sharpening temporal Compensation (MSTC). This module introduces a novel motion sharpening mechanism that enhances key dynamic information within the meta-action sequences through cross-attention technology, thereby improving model adaptability to missing-frame scenarios. STSc-GCN achieves state-of-the-art performance on four constrained datasets and shows superior results on three widely used standard datasets, confirming its effectiveness in both constrained and general scenarios. Code will be available at https://github.com/XingLi1012/STSc-GCN.git.
Decouple-Then-Synergize: A Self-Paced Collaborative Learning Network for RGB-T Snowy Urban Scene Parsing.
Zhou W, Li Y, Jiang Q … +3 more , Liao L, Cong R, Lin W
IEEE Trans Image Process · 2026 · PMID 42213560 · Publisher ↗
Fusing RGB and thermal infrared images is essential for advancing urban scene analysis. However, both modalities exhibit severe performance degradation under snowy conditions. Although independent enhancement modules can...
Fusing RGB and thermal infrared images is essential for advancing urban scene analysis. However, both modalities exhibit severe performance degradation under snowy conditions. Although independent enhancement modules can partially mitigate this issue, stacking multiple modules with different functions increases model complexity and may cause intermodular interference. To address these limitations, we propose a "decouple-then-synergize" framework that decouples the task into frequency-oriented enhancement and spatial semantic fusion, implemented by FRENet (frequency restoration enhancement network) and SIFNet (spatial interactive fusion network), respectively. FRENet uses an asymmetric enhancement strategy that selectively sharpens RGB color gradients while amplifying faint thermal targets. It incorporates a precise spectral refinement module to restore high-frequency details. SIFNet introduces a Mamba zipper fusion module to achieve robust interaction of high-level semantics and performs a reconstruction task to implicitly integrate thermal features into the RGB stream. To ensure effective collaboration between the two networks, we design a self-paced curriculum that manages bidirectional knowledge exchange at both the sample and pixel levels. This approach enables the networks to evolve into their enhanced versions, namely FRENet-collaborative learning (CL) and SIFNet-CL. Extensive experiments on the SUS and PST900 datasets demonstrate that our framework outperforms state-of-the-art scene parsing methods. The code and associated results are available at https://github.com/Lyb-2001/SPCL.
HANeRV: Hierarchically Adaptive Neural Representation for Video Compression.
Tang L, Zhu J, Zhang X … +3 more , Zhang L, Ma S, Huang Q
IEEE Trans Image Process · 2026 · PMID 42213559 · Publisher ↗
Recent advances in video compression introduce implicit neural representation (INR) based methods, which effectively capture global dependencies and characteristics of entire video sequences. Unlike traditional and deep...
Recent advances in video compression introduce implicit neural representation (INR) based methods, which effectively capture global dependencies and characteristics of entire video sequences. Unlike traditional and deep learning based approaches, INR-based methods optimize network parameters from a global perspective, resulting in superior compression potential. However, most current INR methods utilize a fixed and uniform network architecture across all frames, limiting their adaptability to dynamic variations within and between video sequences. This often leads to suboptimal compression outcomes as these methods struggle to capture the distinct nuances and transitions in video content. To overcome these challenges, we propose Hierarchically Adaptive Neural Representation for Video Compression (HANeRV), an innovative INR-based video compression network that adaptively conducts structure optimisation based on the specific content of each video sequence. To better capture dynamic information across video sequences, we propose a dynamic architecture-level adjustment (DAA). Furthermore, to enhance the capture of dynamics between frames within a sequence, we implement a dynamic frame-level adjustment (DFA). Finally, to effectively capture spatial structural information within video frames, thereby enhancing the detail restoration capabilities of HANeRV, we devise a structure level hierarchical structural adaptation (HSA). Experimental results show that HANeRV achieves state-of-the-art performance among INR-based video compression methods and surpasses the H.266/VVC (x266, medium preset) anchor on diverse datasets.
Local Semantics Refinement of Adaptive Representations for Robust Noisy Label Learning.
Lin Y, Zhang Y, Hou J … +2 more , Wang Z, Zeng Z
IEEE Trans Image Process · 2026 · PMID 42213558 · Publisher ↗
The success of deep learning models heavily depends on high-quality labeled data, yet noisy labels are inevitable in large-scale datasets. Existing methods often suffer from confirmation bias and overlook the informative...
The success of deep learning models heavily depends on high-quality labeled data, yet noisy labels are inevitable in large-scale datasets. Existing methods often suffer from confirmation bias and overlook the informative value of hard but clean samples. To address these challenges, we propose Local Semantics Refinement of Adaptive Representations (LFDA), a novel framework that adaptively refines label quality by leveraging local feature consistency and representation alignment. LFDA introduces a Local Consistency Score module that evaluates the similarity among local samples in the latent space to distinguish clean from noisy labels. In addition, a confidence neighborhood is further constructed to provide local reference guidance, enabling more accurate identification and correction of noisy instances. To enhance semantic reliability, LFDA integrates a Reliability-Aware Representation Alignment (RRA) module that aligns high-confidence sample representations to implicitly refine low-confidence instances via soft supervision. Extensive experiments on both synthetic and real-world noisy datasets demonstrate that LFDA consistently outperforms state-of-the-art label noise learning methods. The results confirm its good robustness, generalization ability, and effectiveness in handling diverse and complex noise conditions.
Coarse Labels Matter: Revisiting the Role of Coarse-Grained Supervision in Fine-Grained Learning.
Zhao XY, Zhang P, Zhuang Q … +2 more , Yao Y, Wei XS
IEEE Trans Image Process · 2026 · PMID 42213557 · Publisher ↗
The prohibitive cost of acquiring high-quality fine-grained annotations has spurred significant interest in leveraging readily available coarse labels for fine-grained learning. However, prevailing approaches tend to rel...
The prohibitive cost of acquiring high-quality fine-grained annotations has spurred significant interest in leveraging readily available coarse labels for fine-grained learning. However, prevailing approaches tend to rely on increasingly sophisticated unsupervised methods to define fine-grained proxy tasks, with coarse labels often playing an auxiliary role. In this paper, we propose CSer, a framework designed to maximize the utility of coarse label information for Coarse-to-Fine learning. Specifically, to reconcile the conflict between preserving fine-grained feature diversity and maintaining strong coarse-grained supervision, our coarse-grained self-distillation strategy fortifies the backbone's discriminative power by distilling knowledge from the final classifier to intermediate layers. Concurrently, we introduce dense supervision on common component features within each coarse class, which are decoupled using Non-negative Matrix Factorization. This enhances responses to distinct components, thereby mitigating the simplicity bias in embeddings that can arise under coarse supervision. Moreover, we leverage relationships among intra-class samples to dynamically adjust the negative sampling strategy in contrastive learning, thereby constructing distinct fine-grained class relationships tailored to different coarse classes. Extensive experiments conducted on multiple benchmark datasets demonstrate the effectiveness of our method, yielding state-of-the-art results surpassing competing methods.
Bayesian Fully-Connected Tensor Network for Hyperspectral-Multispectral Image Fusion.
Shan L, Yang Z, Yang LT … +3 more , Li C, Zhao H, Nie X
IEEE Trans Image Process · 2026 · PMID 42213556 · Publisher ↗
Tensor decomposition is a powerful tool for data analysis and has been extensively employed in the field of hyperspectral-multispectral image fusion (HMF). Existing tensor decomposition-based fusion methods typically rel...
Tensor decomposition is a powerful tool for data analysis and has been extensively employed in the field of hyperspectral-multispectral image fusion (HMF). Existing tensor decomposition-based fusion methods typically rely on disruptive data vectorization/reshaping or impose rigid constraints on the arrangement of factor tensors, hindering the preservation of spatial-spectral structures and the modeling of cross-dimensional correlations. Although recent advances utilizing the Fully-Connected Tensor Network (FCTN) decomposition have partially alleviated these limitations, the process of reorganizing data into higher-order tensors still disrupts the intrinsic spatial-spectral structure. Furthermore, these methods necessitate extensive manual parameter tuning and exhibit limited robustness against noise and spatial degradation. To alleviate these issues, we propose the Bayesian FCTN (BFCTN) method. Within this probabilistic framework, a hierarchical sparse prior that characterizing the sparsity of physical elements, establishes connections between the factor tensors. This framework explicitly models the intrinsic physical coupling among spatial structures, spectral signatures, and local scene homogeneity. For model learning, we develop a parameter estimation method based on Variational Bayesian inference (VB) and the Expectation-Maximization (EM) algorithm, which significantly reduces the need for regularization parameter tuning. Extensive experiments demonstrate that BFCTN not only achieves state-of-the-art fusion accuracy and strong robustness but also exhibits practical applicability in complex real-world scenarios. The source code is available at: https://github.com/LinsongShan/BFCTN.
Improving 3D Gaussian Splatting Compression by Scene-Adaptive Lattice Vector Quantization.
Xu H, Wu X, Zhang X
IEEE Trans Image Process · 2026 · PMID 42213555 · Publisher ↗
3D Gaussian Splatting (3DGS) is rapidly gaining popularity for its photorealistic rendering quality and real-time performance, but it generates massive amounts of data. Hence compressing 3DGS data is necessary for the co...
3D Gaussian Splatting (3DGS) is rapidly gaining popularity for its photorealistic rendering quality and real-time performance, but it generates massive amounts of data. Hence compressing 3DGS data is necessary for the cost effectiveness of 3DGS models. Recently, several anchor-based neural compression methods have been proposed, achieving good 3DGS compression performance. However, they all rely on uniform scalar quantization (USQ) due to its simplicity. A tantalizing question is whether more sophisticated quantizers can improve the current 3DGS compression methods with very little extra overhead and minimal change to the system. The answer is yes by replacing USQ with lattice vector quantization (LVQ). To better capture scene-specific characteristics, we optimize the lattice basis for each scene, improving LVQ's adaptability and R-D efficiency. This scene-adaptive LVQ (SALVQ) strikes a balance between the R-D efficiency of vector quantization and the low complexity of USQ. SALVQ can be seamlessly integrated into existing 3DGS compression architectures, enhancing their R-D performance with minimal modifications and computational overhead. Moreover, by scaling the lattice basis vectors, SALVQ can dynamically adjust lattice density, enabling a single model to accommodate multiple bit rate targets. This flexibility eliminates the need to train separate models for different compression levels, significantly reducing training time and memory consumption.
Practical Lossless Volumetric Medical Image Compression via Tri-Plane Context Tree Learning.
Bai Y, Zhao Y, Wang K … +5 more , Du Y, Cheng J, Fang T, Liu X, Gao W
IEEE Trans Image Process · 2026 · PMID 42213554 · Publisher ↗
Lossless compression of volumetric medical images is of paramount importance for clinical and research applications where data fidelity is essential. Traditional compression methods are often limited in efficiency due to...
Lossless compression of volumetric medical images is of paramount importance for clinical and research applications where data fidelity is essential. Traditional compression methods are often limited in efficiency due to rigid, handcrafted models. Conversely, deep neural network (DNN)-based compression methods, while effective, demand substantial computational resources, hindering deployment in resource-constrained settings. To address these challenges, we propose a novel tri-plane context tree (TCT)-based method for lossless volumetric medical image compression that delivers high performance without relying on DNNs or external training data. To exploit intra-slice and inter-slice redundancies, we introduce a compact tri-plane context representation that decomposes complex 3D context modeling into efficient 2D modeling on three orthogonal planes. By integrating this representation with a context tree framework, we develop an input-specific TCT model employing an adaptive binary tree structure. At each tree node, the model dynamically selects from a suite of tri-plane based predictors and contextual feature extractors, enabling data-adaptive context modeling tailored to local structural characteristics. Instead of offline training, we sample a subset of the input volume to learn the TCT model by optimizing the minimum description length (MDL) through iterative construction and pruning. With the learned TCT model, each pixel retrieves its corresponding context, computes the prediction residual using the predictor dictated by the context, and performs entropy encoding based on the associated histograms. Experimental results demonstrate that the proposed method achieves compression performance on par with recent DNN-based methods on multiple datasets, while maintaining low computational cost and fast coding speeds, making it highly applicable in practice.
Exploiting Cross-Task Synergy via Frequency-Driven Hierarchical Learning for Multi-Task Dense Prediction.
Zhuge Y, Yu X, Zhang L … +3 more , Jia X, Zhan J, Lu H
IEEE Trans Image Process · 2026 · PMID 42208031 · Publisher ↗
Multi-task dense prediction improves pixel-level performance by leveraging shared representations and inter-task collaboration. However, existing approaches either rely on implicit task relationships or neglect frequency...
Multi-task dense prediction improves pixel-level performance by leveraging shared representations and inter-task collaboration. However, existing approaches either rely on implicit task relationships or neglect frequency-domain cues that are essential for preserving fine-grained details and enhancing cross-task feature learning at multiple scales. As a result, they face persistent challenges in multi-scale feature fusion, effective task interaction, and accurate decoding. To address these issues, we propose a hierarchical frequency-driven framework, termed Hierarchical Frequency-Adaptive Network (HiFAN), that facilitates cross-task collaborative optimization via frequency-domain analysis. Specifically, we first design a task-adaptive fusion module that exploits multi-scale frequency-domain information to enhance spatial details. This module generates dynamic convolutional kernels with task-specific parameters and positional biases to adaptively accommodate diverse task requirements. Next, we introduce an efficient cross-task interaction module that leverages compact low-frequency representations to enable global context exchange across tasks. Finally, we present a high-frequency-aware decoder that mitigates feature smoothing and detail loss commonly introduced by Transformer-based decoders. We demonstrate the effectiveness of HiFAN on two standard multi-task learning benchmarks, PASCAL-Context and NYUD-v2, achieving strong and competitive performance across multiple tasks. The code and model weights are available in HiFAN.
Stroke-Based Cyclic Amplifier: Image Super-Resolution at Arbitrary Ultra-Large Scales.
Guo W, Lu P, Peng X … +2 more , Zhao Z, Li S
IEEE Trans Image Process · 2026 · PMID 42208030 · Publisher ↗
Prior Arbitrary-Scale Image Super-Resolution (ASISR) methods often experience a significant performance decline when the upsampling factor exceeds the range covered by the training data, introducing substantial blurring....
Prior Arbitrary-Scale Image Super-Resolution (ASISR) methods often experience a significant performance decline when the upsampling factor exceeds the range covered by the training data, introducing substantial blurring. To address this issue, we propose a unified model, Stroke-based Cyclic Amplifier (SbCA), for ultra-large upsampling tasks. The key of SbCA is the stroke vector amplifier, which decomposes the image into a series of strokes represented as vector graphics for magnification. Then, the detail completion module also restores missing details, ensuring high-fidelity image reconstruction. Our cyclic strategy achieves ultra-large upsampling by iteratively refining details with this unified SbCA model, trained only once for all, while keeping sub-scales within the training range. Our approach effectively addresses the distribution drift issue and eliminates artifacts, noise and blurring, producing high-quality, high-resolution super-resolved images. Experimental validations on both synthetic and real-world datasets demonstrate that our approach significantly outperforms existing methods in ultra-large upsampling tasks (e.g. $\times 100$ ), delivering visual quality far superior to state-of-the-art techniques.
RCodSpace: A Robust Learned Coding Method for Deep Space Visual Transmission.
Yuan R, Cui C, Wang Y … +2 more , Ma S, Jia C
IEEE Trans Image Process · 2026 · PMID 42208029 · Publisher ↗
With advancement of deep space exploration, vast amounts of image data must be transmitted back to Earth for scientific research. Current deep-space image codecs rely on conventional progressive coding algorithms, but of...
With advancement of deep space exploration, vast amounts of image data must be transmitted back to Earth for scientific research. Current deep-space image codecs rely on conventional progressive coding algorithms, but offer limited compression performance. Despite great success on natural images achieved by learning-based compression methods, the high computational complexity restrains their application in the deep space missions and they are unable to cope with packet loss caused by severe noise interference during transmission. Motivated by the urgent need and technical challenges, we take Mars as a representative case and propose a novel image compression and transmission framework that innovatively incorporates learning-based strategies to deliver both low-complexity and error-resilient source coding. To adapt learning-based methods to the stringent constraints and high packet loss of deep space environment, we first establish a new Martian image dataset with high resolution and diversity, and analyze its characteristics to guide the network design. With heterogeneous textures yet synergistic structures as well as higher inter-channel similarity in the feature domain revealed for the Martian images, we develop a Martian Vision Adaptive Transformation Module (MVATM) with efficient low-complexity compression. Furthermore, unlike conventional one-stage training, a novel two-stage training strategy with Joint Channel Training (JCT) is proposed to enhance error resilience. Experimental results and hardware deployment strongly validate that our method achieves a better rate-distortion-complexity (RDC) trade-offs than other advanced learning-based models and significantly outperforms conventional methods in deep space simulation test. Also, the technical strategies proposed herein can offer methodological insights for deep space and other resource-constrained fields.
Enhancing Zero-Shot Adversarial Robustness of Vision-Language Models With Training-Free Adaptive Feature Movement.
Tong B, Lai H, Pan Y … +2 more , Yin J, Lin L
IEEE Trans Image Process · 2026 · PMID 42208028 · Publisher ↗
Pre-trained Vision-Language Models (VLMs) have demonstrated strong zero-shot generalization capabilities. Despite their effectiveness on various downstream tasks, they remain vulnerable to adversarial samples. Existing m...
Pre-trained Vision-Language Models (VLMs) have demonstrated strong zero-shot generalization capabilities. Despite their effectiveness on various downstream tasks, they remain vulnerable to adversarial samples. Existing methods fine-tune VLMs to improve their robust performance by performing adversarial training on a certain dataset. However, this can lead to model overfitting and is not a true zero-shot scenario. In this paper, we propose a truly zero-shot and training-free approach that can improve the zero-shot adversarial robustness of VLMs on the evaluated benchmarks. Specifically, we first discover that simply adding Gaussian noise can enhance the VLM's zero-shot robustness. Then, we treat the adversarial examples with added Gaussian noise as anchors and strive to find a path in the embedding space that leads from the adversarial examples to the cleaner samples. Furthermore, to avoid the overfitting issue caused by fixed hyperparameters, we propose an adaptive parameter adjustment method based on the distance between the anchors and adversarial samples in the embedding space. We largely preserve the original VLMs' zero-shot generalization abilities in a truly zero-shot and training-free manner on the evaluated benchmarks compared to previous methods. Extensive experiments on 16 datasets demonstrate that our method can achieve stronger zero-shot robust performance, improving the top-1 robust accuracy by an average of 10.83%.
Single-Image Reflection Removal via Iterative Prompt Learning of Reflection Level.
Song B, Zhou J, Xu S … +4 more , Liu X, Wu H, Fan X, Wen B
IEEE Trans Image Process · 2026 · PMID 42208027 · Publisher ↗
Single-image reflection removal (SIRR) aims to restore the latent background layer from a reflection-contaminated image. Despite the promising progress achieved by deep learning-based methods, the roles of negative train...
Single-image reflection removal (SIRR) aims to restore the latent background layer from a reflection-contaminated image. Despite the promising progress achieved by deep learning-based methods, the roles of negative training samples and descriptive prompts for the reflection severity are underexplored in most existing deep SIRR approaches, limiting their reflection removal performance and generalization capability. In this work, we introduce a novel training framework that synergistically leverages learnable prompts and image data to optimize the restoration network. To this end, we define reflection levels corresponding to varying degrees of reflection interference on the background content and learn reflection-level prompts to supervise the SIRR process. We propose an Iterative Reflection Level Reduction (IRLR) framework composed of a Restoration Network Training Module (RNTM) and a Reflection Level Learning Module (RLLM). Specifically, RNTM predicts the background layer under the guidance of prompts learned by RLLM, while RLLM in turn refines these prompts using outputs from RNTM. The two modules are trained iteratively to progressively reduce the reflection levels of estimated background layers. To initialize the prompts, we construct a dedicated reflection-level dataset for pretraining. For adaptively supervising RNTM, we design a new reflection-level-aware strategy to address the challenge of directly aligning the output background with the minimal reflection level. Comprehensive experimental results demonstrate that the proposed method significantly outperforms state-of-the-art methods on average performance across several released datasets, improving PSNR by 0.82 dB and SSIM by 0.0120, respectively. The source code and dataset are available at https://github.com/NamecantbeNULL/IRLR_SIRR.
SD-ReID: View-Aware Stable Diffusion for Aerial-Ground Person Re-Identification.
Wang Y, Hu X, Wang L … +2 more , Zhang P, Lu H
IEEE Trans Image Process · 2026 · PMID 42208026 · Publisher ↗
Aerial-Ground Person Re-IDentification (AG-ReID) aims to retrieve specific persons across cameras with different viewpoints. Previous works focus on designing discriminative models to maintain the identity consistency de...
Aerial-Ground Person Re-IDentification (AG-ReID) aims to retrieve specific persons across cameras with different viewpoints. Previous works focus on designing discriminative models to maintain the identity consistency despite drastic changes in camera viewpoints. The core idea behind these methods is quite natural, but designing a view-robust model is a very challenging task. Moreover, they overlook the contribution of view-specific features in enhancing the model's ability to represent persons. To address these issues, we propose a novel generative framework named SD-ReID for AG-ReID, which leverages generative models to mimic the feature distribution of different views while extracting robust identity representations. More specifically, we first train a ViT-based model to extract person representations along with controllable conditions, including identity and view conditions. We then fine-tune the Stable Diffusion (SD) model to enhance person representations guided by these controllable conditions. Furthermore, we introduce the View-Refined Decoder (VRD) to bridge the gap between instance-level and global-level features. Finally, both person representations and all-view features are employed to retrieve target persons. Extensive experiments on five AG-ReID benchmarks (i.e., CARGO, AG-ReIDv1, AG-ReIDv2, LAGPeR and G2APS-ReID) demonstrate the effectiveness of our proposed method. The source code and pre-trained models are available at https://github.com/924973292/SD-ReID.
Multi-Level Collaborative Distillation Meets Global Workspace Model: A Unified Framework for OCIL.
Su S, Liang G, Cheng D … +2 more , Zhang S, Ran L
IEEE Trans Image Process · 2026 · PMID 42202193 · Publisher ↗
Online Class-Incremental Learning (OCIL) enables models to learn continuously from non-i.i.d. data streams. Since samples of the data streams can be seen only once, it is more suitable for real-world scenarios compared t...
Online Class-Incremental Learning (OCIL) enables models to learn continuously from non-i.i.d. data streams. Since samples of the data streams can be seen only once, it is more suitable for real-world scenarios compared to offline learning. However, this constraint intensifies the challenge for OCIL in maintaining an appropriate balance between stability and plasticity. Moreover, under stricter memory buffer constraints in real world, current replay-based methods are less effective. While ensemble methods improve plasticity, they often struggle with stability. Inspired by the Global Workspace Theory (GWT), we propose a novel approach that enhances ensemble learning through a Global Workspace Model (GWM)-a shared, implicit memory that guides the learning of multiple student models. The GWM is formed by fusing the parameters of all students within each training batch, capturing the historical learning trajectory and serving as a dynamic anchor for knowledge consolidation. Like the broadcasting mechanism of GWT, the GWM is redistributed periodically to students, stabilizing learning and promoting cross-task consistency. In addition, we introduce a multi-level collaborative distillation mechanism. It enforces peer-to-peer consistency among students and preserves historical knowledge by aligning each student with the GWM. As a result, student models remain adaptable to new tasks while maintaining previously learned knowledge, striking a better balance between stability and plasticity. Extensive experiments on three standard OCIL benchmarks show that our method delivers significant performance improvement for several OCIL models across various memory budgets. The code is available at https://github.com/susususushi/GWM.
FDSNet: Frequency-Decoupled Stack Fusion Network for Light Field All-in-Focus Image Generation.
Sun M, Liu Y, Zhang L … +4 more , Lu L, Sun Y, Ai H, Brilakis I
IEEE Trans Image Process · 2026 · PMID 42202192 · Publisher ↗
All-in-focus(AIF) images, which contain comprehensive scene information with global sharpness, play a crucial role in high-precision light field (LF) measurement and computational imaging. However, generating AIF images...
All-in-focus(AIF) images, which contain comprehensive scene information with global sharpness, play a crucial role in high-precision light field (LF) measurement and computational imaging. However, generating AIF images from LF data typically requires accurate depth priors, which are often unavailable or unreliable in practice. To overcome this limitation, directly fusing a series of LF refocused images provides an effective alternative that eliminates the dependency on explicit depth estimation. Nevertheless, existing multi-focus image fusion(MFIF) methods are primarily designed for fusing image pairs with complementary focus, performing poorly when applied to stacks due to the error accumulation that occurs during iterative fusion. To this end, we propose a Frequency-Decoupled Stack Fusion Network (FDSNet) for high-precision depth-free LF AIF image generation. FDSNet incorporates a spatial-frequency joint feature extraction module that captures multi-scale spatial details while decoupling high- and low- frequency components to model textures and contextual information separately, thereby alleviating edge blurring caused by subtle focal variations and weak textures in transition regions. Moreover, a dual-stage cross-attention fusion module, following a coarse-to-fine strategy, suppresses artifacts, enhances edge fidelity, and enables simultaneous fusion of arbitrary numbers of refocused images, thereby avoiding error accumulation and computational redundancy. Extensive experiments on both synthetic and real LF datasets demonstrate that FDSNet achieves superior visual quality and quantitative performance. Additional experiments further demonstrate that FDSNet performs robustly under varying low-light and noisy conditions. These results validate that FDSNet delivers excellent fusion capability in terms of image clarity, detail preservation, noise resistance, and generalization, outperforming existing state-of-the-art methods.
Bi-Temporal Benefits: Progressive Spectral-Spatial-Temporal Feature Extraction for Hyperspectral Image Classification.
Liu W, Li S, Li X
IEEE Trans Image Process · 2026 · PMID 42202191 · Publisher ↗
Spectral-spatial feature extraction widely serves as the foundation for hyperspectral image classification (HSIC). However, its effectiveness decreases when applied to land covers type with temporal variations. This limi...
Spectral-spatial feature extraction widely serves as the foundation for hyperspectral image classification (HSIC). However, its effectiveness decreases when applied to land covers type with temporal variations. This limitation arises from the lack of the temporal dimension in existing HSIC methods, hindering their ability to model real-world surface dynamics. To tackle these problems, Bi-tEmporal HyperspectrAL image classiFication network (BehalfNet) employs the dual-branch stacked architecture to process bi-temporal images, learning spectral-spatial-temporal features. Within each stacked block, features undergo the sequential feature processing pipeline. Specifically, the progressive adaptive fusion (PAF) module first extracts foundational spectral-spatial features for each temporal phase through long-short term fusion. These features are then refined at an intra-temporal level by the gated spectral-spatial attention (GSSA) module. Subsequently, the bi-temporal self-cross attention (BTSCA) module effectively captures the complex dynamic changes between the bi-temporal features using a novel closed-loop attention mechanism. Furthermore, the Anji dataset is introduced as the first publicly available dataset for bi-temporal HSIC. Comprehensive experiments on the Anji dataset and public Viareggio dataset (originally used for anomaly change detection) demonstrate the competitiveness of the proposed BehalfNet over other state-of-the-art HSIC methods. The code and Anji dataset will be released at https://github.com/lixinghua5540/BehalfNet.
Self-Expressive High-Order Tensor Unrolling Network for Unsupervised Hyperspectral and Multispectral Image Fusion.
Wang H, Xu Y, Wei Z … +1 more , Wu Z
IEEE Trans Image Process · 2026 · PMID 42202190 · Publisher ↗
Hyperspectral and multispectral image fusion (HMF) enhances spatial-spectral quality by fusing low-resolution hyperspectral images (LR-HSI) with high-resolution multispectral images (HR-MSI). Although recent fusion metho...
Hyperspectral and multispectral image fusion (HMF) enhances spatial-spectral quality by fusing low-resolution hyperspectral images (LR-HSI) with high-resolution multispectral images (HR-MSI). Although recent fusion methods have shown promise in preserving the multi-mode structure of high-dimensional data, existing fusion methods still face some challenges. For tensor-based approaches, conventional mode-wise decomposition, such as order-3 CP or Tucker decomposition, may disrupt intrinsic spatial consistency. Furthermore, although deep learning exhibits powerful feature representation ability, existing deep fusion methods either rely on 'data-driven' deep fusion networks remain insufficiently interpretability with large training data. To address these issues, a novel Self-Expressive High-Order Tensor Unrolling Network (SHOTUN) is proposed for unsupervised HSI-MSI fusion. Within the sparse core tensor decomposition framework, we introduce the intrinsic self-expressive relationships among overlapping image patches as a form of high-order mode representation to preserve spatial structure of the fusion model. During optimization, we adopt an alternative optimizing strategy and design dedicated modules for each sub-problem, yielding an interpretable end-to-end training pipeline. Furthermore, to improve generalization across different sensors, we introduce a pre-training strategy into the unsupervised training for the more accurate estimation of unknown degraded parameters. Extensive experimental results on simulated and real datasets demonstrate the effectiveness of our proposed method. The source code is publicly available at https://github.com/Shawn-H-Wang/SHOTUN.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe