IEEE Transactions On Image Processing[JOURNAL]
Sun
200 papers
Mask-guided Asymmetric Contrastive and Semantic Alignment for Unsupervised Person Re-Identification.
Tan X, Wei R, Zhang Q … +3 more , Qi D, Miao D, Zhao C
IEEE Trans Image Process · 2026 Jul · PMID 42397992 · Publisher ↗
Unsupervised person re-identification (ReID) aims to learn identity-discriminative representations without manual annotations, which is challenging due to noisy pseudo labels, background clutter, and large appearance var...
Unsupervised person re-identification (ReID) aims to learn identity-discriminative representations without manual annotations, which is challenging due to noisy pseudo labels, background clutter, and large appearance variations. Recent studies have shown that exploiting fine-grained local cues is crucial for improving robustness in unsupervised ReID. In this context, random masking has emerged as a simple and annotation-free way to encourage the model to focus on informative regions. However, existing masking-based unsupervised ReID methods still suffer from two limitations: (1) Underused masked views: masked views are treated as degraded auxiliaries rather than exploited as fine-grained supervisory signals; (2) Weak cross-view alignment: feature alignment is restricted to mini-batch pairs, lacking explicit global alignment between masked and unmasked views across clusters. To address these issues, we propose the Mask-guided Asymmetric Contrastive and Semantic Alignment (ACSA) framework. Specifically, we introduce an Asymmetric Contrastive Learning (ACL) module with a dual-memory mechanism to separately encode masked and unmasked features, allowing masked views to serve as informative and discriminative supervision. In parallel, a Semantic Alignment Learning (SAL) module conducts multi-granularity distribution alignment by aligning both cluster-level prototypes and randomly sampled instance-level features, thereby preserving semantic consistency and intra-cluster diversity. Furthermore, to provide more reliable semantic anchors for SAL under noisy pseudo labels, we introduce a Progressive Refinement Module (PRM), which refines prototypes and features via exponential moving averaging for more stable semantic alignment. Extensive experiments validate the superiority of our method, even outperforming certain supervised counterparts. Code is available at https://github.com/Trangle12/ACSA.
Hyperbolic Cycle Alignment for Infrared-Visible Image Fusion.
Li Y, Cao B, Feng J … +3 more , Cao H, Hu Q, Zhu P
IEEE Trans Image Process · 2026 Jul · PMID 42391079 · Publisher ↗
Accurate alignment is a fundamental prerequisite for multi-modal image fusion, yet aligning multi-modal remains challenging due to nonlinear geometric distortions and substantial appearance discrepancies. This work prese...
Accurate alignment is a fundamental prerequisite for multi-modal image fusion, yet aligning multi-modal remains challenging due to nonlinear geometric distortions and substantial appearance discrepancies. This work presents the Hyperbolic Cycle Alignment Network (Hy-CycleAlign), a geometry-aware cyclic alignment framework formulated in hyperbolic space. By embedding multi-level representations into a negatively curved manifold, Hy-CycleAlign departs from conventional Euclidean-space paradigms and provides enhanced sensitivity to spatial perturbations, enabling more reliable modeling of cross-modal correspondences. The framework integrates a dual-path cyclic structure that enforces bidirectional deformation consistency and prevents accumulated alignment drift. In addition, a hyperbolic hierarchy contrastive alignment module jointly constrains semantic and structural representations within a unified hyperbolic embedding domain, promoting coherent alignment across global and local geometric scales. From a theoretical standpoint, we derive the sensitivity properties of the Poincaré model and show that its metric inherently amplifies positional variations, thereby strengthening the discriminability of subtle cross-modal misalignments compared with Euclidean geometry. Extensive experiments on diverse misaligned multi-modal datasets show that Hy-CycleAlign achieves strong overall results in alignment accuracy, structural fidelity, and downstream fusion quality. These results validate the effectiveness of hyperbolic geometric modeling for robust multi-modal image alignment.
Learning Gaze Synthesizer via 3D-eye Controlled Diffusion and Cross-domain Feature Alignment.
Chen M, Lin Y, Li Y … +4 more , Chen Y, Chen X, Ke B, Ni B
IEEE Trans Image Process · 2026 Jul · PMID 42391078 · Publisher ↗
Appearance-based supervised methods with full-face image input have made tremendous advances in recent gaze estimation tasks. However, intensive human annotation requirement inhibits current methods from achieving indust...
Appearance-based supervised methods with full-face image input have made tremendous advances in recent gaze estimation tasks. However, intensive human annotation requirement inhibits current methods from achieving industrial level accuracy and robustness. Although methods based on generative AI can synthesize eye images to expand self-annotated eye data, these methods usually have limited model capacity and cannot effectively inject gaze information, resulting in poor quality, monotonous texture, and inaccurate gaze direction of the generated eye images. To alleviate the above challenge, we propose a novel gaze data synthesizer framework, in which a 3D-eye model that can flexibly manipulate the gaze direction is used to finely control eye image synthesis based on a stable diffusion large generative model, so that high-quality eye images with arbitrary gaze angle can be synthesized. At the same time, when training the gaze feature extractor, we propose a cross-domain feature alignment module to minimize the feature distribution discrepancy between real samples and synthetic ones, to pursue domain-invariant (shape, texture, etc.) gaze representation. Both qualitative and quantitative experimental results demonstrate that the proposed scheme generates high-quality gaze images and also achieves superior gaze estimation performances over state-of-the-art.
Underlying Semantic Diffusion for Effective and Efficient In-Context Learning.
Ji Z, Cao W, Zhang Y … +2 more , Pang Y, Han J
IEEE Trans Image Process · 2026 Jul · PMID 42391077 · Publisher ↗
Diffusion models have emerged as a powerful framework for tasks like image controllable generation and dense prediction. However, existing models often struggle to capture underlying semantics (e.g., edges, textures, sha...
Diffusion models have emerged as a powerful framework for tasks like image controllable generation and dense prediction. However, existing models often struggle to capture underlying semantics (e.g., edges, textures, shapes) and effectively utilize in-context learning, limiting their contextual understanding and image generation quality. Furthermore, high computational costs and slow inference speeds hinder their real-time applications. To address these challenges, we propose Underlying Semantic Diffusion (US-Diffusion), an enhanced diffusion model that improves underlying semantics learning, computational efficiency, and in-context learning capabilities on multi-task scenarios. We introduce Separate & Gather Adapter (SGA), which decouples input conditions for different tasks while sharing the architecture, enabling better in-context learning and generalization across diverse visual domains. We also present a Feedback-Aided Learning (FAL) framework, which leverages feedback signals to guide the model in capturing semantic details and dynamically adapting to task-specific contextual cues. Furthermore, we propose a plug-and-play Efficient Sampling Strategy (ESS) for dense sampling at time steps with high-noise levels, which aims at optimizing training and inference efficiency while maintaining strong in-context learning performance. Experimental results demonstrate that US-Diffusion outperforms the state-of-the-art method, achieving an average reduction of 7.47 in FID on Map2Image tasks and an average reduction of 0.026 in RMSE on Image2Map tasks, while achieving approximately 9.45× faster inference speed. Our method also demonstrates superior training efficiency and in-context learning capabilities, excelling in new datasets and tasks, highlighting its robustness and adaptability across diverse visual domains. The source code will be released at https://github.com/dragon-cao/US-Diffusion.
DiffRES: Unleashing Text-to-Image Diffusion Models for Generative Referring Expression Segmentation without Information Leakage.
Zeng T, Wang S, Wu Y … +1 more , Du B
IEEE Trans Image Process · 2026 Jul · PMID 42391076 · Publisher ↗
Referring Expression Segmentation (RES) requires models not only to locate objects specified by referring expressions accurately but also to predict complete and precise masks. Existing methods primarily focus on complex...
Referring Expression Segmentation (RES) requires models not only to locate objects specified by referring expressions accurately but also to predict complete and precise masks. Existing methods primarily focus on complex multimodal alignment for object grounding, often neglecting mask quality, which results in incomplete foreground regions and imprecise boundaries. To address these challenges, we propose DiffRES, a mask-generating framework based on Stable Diffusion (SD), designed to tackle the RES problem with a focus on achieving high-quality masks. DiffRES effectively mitigates the information leakage issue prevalent in existing generative dense prediction diffusion models, which allows the model to infer the target's position directly from noisy masks during training without understanding the text condition, leading to severe overfitting. Specifically, DiffRES directly guides SD with visual and linguistic information to generate target binary masks, fundamentally bypassing the information leakage issue. This approach enables efficient knowledge transfer from SD to the RES task, resulting in precisely localized binary masks with sharp and precise boundaries. Extensive experiments show that DiffRES surpasses current state-of-the-art traditional methods on boundary precision (AP) which is sensitive to mask quality, while also significantly outperforming all existing SD-based RES models across all metrics. Our code is publicly available at https://github.com/charon517-517/DiffRES.
Location Matters: Frequency-Spatial Dual Space Adaptation for Cross-Domain Few-Shot Segmentation.
Zhang J, Sun G, Li Y … +3 more , Wang H, Shu X, Xie GS
IEEE Trans Image Process · 2026 Jul · PMID 42391075 · Publisher ↗
Current cross-domain few-shot semantic segmentation (CD-FSS) methods tend to overlook a fundamental yet domain-agnostic prior: the spatial correspondence between support and query images driven by the task itself. Unlike...
Current cross-domain few-shot semantic segmentation (CD-FSS) methods tend to overlook a fundamental yet domain-agnostic prior: the spatial correspondence between support and query images driven by the task itself. Unlike semantic similarity, this spatial correlation arises from the consistent structural layout of foreground objects across domains. To exploit this structural prior, we propose a novel frequency-spatial dual space adaptation (FDSA) framework, to learn domain-invariant structures and task-specific priors by jointly suppressing domain-specific redundancy in frequency domain and reinforcing geometric priors in spatial domain. Specifically, FDSA consists of two sequential modules, i.e., the frequency structural adapter (FSA) and the spatial geometry adapter (SGA). FSA performs image modulation in the frequency domain by emphasizing low-frequency foreground semantics and attenuating high-frequency noise, thus maintaining structural integrity of these input images. By contrast, SGA leverages handcrafted local descriptors to extract keypoints from both support and query images, generating Gaussian-based geometric priors that highlight desirable aligned regions. Additionally, we introduce spatial-guided SAM refinement (SSR) to extend our spatial geometric prior into the Segment Anything Model (SAM). SSR generates a soft Gaussian point prompt centered on the coarse mask, enabling SAM to refine segmentation masks without manual intervention. This integration effectively bridges task-specific localization with high-quality segmentation. Extensive experiments on four standard CD-FSS benchmarks demonstrate that our method achieves new state-of-the-art performance. Code is available at https://github.com/whales-zhang/FDSA.git.
BayeTopo: Bayesian-based Topology-guided Learning for Vascular Imaging Segmentation.
Xie B, Zhuang S, Zhang H … +3 more , Peng C, Xu L, Gao Z
IEEE Trans Image Process · 2026 Jul · PMID 42391074 · Publisher ↗
Vascular segmentation is a critical task in clinical medical image processing and a prerequisite for accurately diagnosing vascular-related diseases. The development of automated segmentation methods is challenged by int...
Vascular segmentation is a critical task in clinical medical image processing and a prerequisite for accurately diagnosing vascular-related diseases. The development of automated segmentation methods is challenged by internal variability in vessel representations. Recently, topology guidance has shown potential for capturing semantically consistent representations. However, current topology-guided methods lack modeling of global-to-local dependencies. This limitation forces latent representations subject to a trade-off between learning global topology and local geometries within the vascular network. In this paper, we propose a Bayesian-based topology-guided (BayeTopo) learning approach to capture global-to-local dependencies. It introduces a prior that explicitly models local geometry as a probability conditioned on global topology within topology-sensitive regions of the vascular network. We further implement a topology-guided diffusion model to optimize the conditional probability. It gradually infers local geometry from the restored global topology with multi-scale noise, enabling rich global-to-local representations. Then, an inhomogeneous diffusion process is involved, where noise initially accumulates in topology-sensitive regions before achieving uniformity. It ensures an orderly degradation of information from global topology to local geometry, thereby enabling effective global-to-local supervision. Extensive experiments on six datasets, involving three types of vascular networks under four imaging modalities, demonstrate the superior performance and generalization capability of our method compared to previous topology-guided learning and diffusion-based models. A series of case studies further validates the effectiveness of our designs in enhancing semantic consistency within local vascular regions, thereby improving topological accuracy.
vSTMD: Visual Motion Detection for Extremely Tiny Target at Various Velocities.
Xu M, Luan H, Hao ZD … +2 more , Peng J, Yue S
IEEE Trans Image Process · 2026 Jul · PMID 42391073 · Publisher ↗
Visual motion detection for extremely tiny (ET-) targets is challenging, due to their category-independent nature and the scarcity of visual cues, which often incapacitate mainstream feature-based models. Natural archite...
Visual motion detection for extremely tiny (ET-) targets is challenging, due to their category-independent nature and the scarcity of visual cues, which often incapacitate mainstream feature-based models. Natural architectures with rich interpretability offer a promising alternative, where STMD architectures derived from insect visual STMD (Small Target Motion Detector) pathways have demonstrated their effectiveness. However, previous STMD models are constrained to a narrow velocity range, hindering their efficacy in real-world scenarios where targets exhibit diverse and unstable dynamics. To address this limitation, we present vSTMD, a learning-free model for motion detection of ET-targets at various velocities. vSTMD proposes two key mechanisms: cross-Inhibition Dynamic Potential (cIDP) and Collaborative Directional Gradient Calculation (CDGC). Specifically, cIDP serves as a self-adaptive mechanism, efficiently capturing motion cues across a wide velocity spectrum. CDGC enhances orienting accuracy and robustness while reducing computational overhead to one-eighth of previously isolated strategies. Evaluated on the real-world dataset RIST, the proposed vSTMD and its feedback-facilitated variant vSTMD-F achieve relative F gains of 30% and 58% over state-of-the-art STMD approaches, respectively. Furthermore, both models demonstrate competitive orientation estimation performance compared to SOTA deep learning-driven methods. Experiments also reveal the superiority of the natural architecture for ET-object motion detection - vSTMD is 60× faster than contemporary data-driven methods, making it highly suitable for real-time applications in dynamic scenarios and complex backgrounds. Code is available at Project Repository.
Style-Aware Contrastive Test-Time Adaptation: A Dual-Cache Model for Robust Vision-Language Alignment.
Wang S, ALuSi, Yang X … +3 more , Wang P, Xu K, Zhang X
IEEE Trans Image Process · 2026 Jul · PMID 42384518 · Publisher ↗
Test-time adaptation (TTA) has emerged as a key strategy to enhance vision-language models (VLMs) under real-world distribution changes. However, existing methods always face two problems: (1) The fundamental trade-off d...
Test-time adaptation (TTA) has emerged as a key strategy to enhance vision-language models (VLMs) under real-world distribution changes. However, existing methods always face two problems: (1) The fundamental trade-off dilemma: parameter-free TTA retains inference efficiency but fails to correct modality misalignment, while prompt tuning adapts to shifts, incurs high computational costs, and lacks knowledge retention. (2) Discriminative collapse also exists in TTA when faced with fine-grained downstream tasks. To alleviate these two bottlenecks, we introduce Style-aware Contrastive Test-Time Adaptation (SCTTA), a novel framework that jointly addresses modality misalignment and discriminative collapse. Firstly, we introduce Style-aware Embedding Adaptation (SEA), which dynamically refines text embeddings by incorporating domain-specific style attributes, improving alignment between visual and textual modalities. Secondly, we propose Fine-grained Contrastive Adaptation (FCA), which enhances feature separation by enforcing contrastive learning with adaptive prototypes, reducing inter-class feature overlap in fine-grained tasks. In addition, we introduce Dual-Cache Model (DCM), which extends prior unimodal cache model to a multimodal cache for the first time. Eventually, it accumulates adaptation knowledge through a visual-cache (capturing evolving domain styles) and a textual-cache (retaining discriminative semantics), enabling long-term adaptation without additional overhead. Extensive experiments on 15 datasets demonstrate that our approach achieves state-of-the-art performance for both fine-grained and out-of-distribution dataset benchmarks. Furthermore, SCTTA continuously improves as more test samples accumulate, validating its sustainable adaptation capacity. Our code is available at https://github.com/alusi123/SCTTA.
Semantic Frame Interpolation.
Hong Y, Zhang J, Yi R … +4 more , Cao W, Hu X, Ma L, Yan S
IEEE Trans Image Process · 2026 Jul · PMID 42384517 · Publisher ↗
Generating intermediate video content of varying lengths based on given first and last frames, along with text prompt information, offers significant research and application potential. However, traditional frame interpo...
Generating intermediate video content of varying lengths based on given first and last frames, along with text prompt information, offers significant research and application potential. However, traditional frame interpolation tasks primarily focus on scenarios with a small number of frames, no text control, and minimal differences between the first and last frames. Recent community developers have utilized large video models represented byWan to endow frame-to-frame capabilities. However, these models can only generate a fixed number of frames and often fail to produce satisfactory results for certain frame lengths, while this setting lacks a clear official definition and a well-established benchmark. In this paper, we first propose a new practical Semantic Frame Interpolation (SFI) task from the perspective of academic definition, which covers the above two settings and supports inference at multiple frame rates. To achieve this goal, we propose a novel SemFi model building upon Wan2.1, which incorporates a Mixture-of-LoRA module to ensure the generation of high-consistency content that aligns with control conditions across various frame length limitations. Furthermore, we propose SFI-300K, the first general-purpose dataset and benchmark specifically designed for SFI. To support this, we collect and process data from the perspective of SFI, carefully designing evaluation metrics and methods to evaluate the performance of the model in multiple dimensions, including images and videos, and various aspects, including consistency and diversity. Through extensive experiments on SFI-300K, we demonstrate that our method is particularly well-suited to meet the requirements of the SFI task.
Physics-Guided Cross-Modal Decoupling with Test-Time Adaptation for Hyperspectral Image Restoration.
Liu Q, Xu Y, Wu Z … +1 more , Wei Z
IEEE Trans Image Process · 2026 Jul · PMID 42384516 · Publisher ↗
Hyperspectral image (HSI) restoration tasks including super-resolution, denoising, and inpainting, present significant challenges due to intrinsic spectral-spatial coupling and limited training data availability. Recent...
Hyperspectral image (HSI) restoration tasks including super-resolution, denoising, and inpainting, present significant challenges due to intrinsic spectral-spatial coupling and limited training data availability. Recent advances in RGB image restoration demonstrate that models pretrained on large-scale datasets acquire exceptional generalization capabilities, suggesting potential cross-modal knowledge transfer solutions for HSI recovery. However, existing approaches exhibit two critical limitations: (i) prohibitive computational costs from mandatory fine-tuning procedures, and (ii) inadequate cross-modal adaptation causing spectral distortions. To address these challenges, we propose a Two-Stage Cross-Modal Decoupling Network (CMDN) achieves spectral-faithful HSI restoration without fine-tuning the pretrained RGB prior; instead, we perform unsupervised test-time learning only on a lightweight spectral rectifier for sample-specific spectral calibration. Our methodology introduces two fundamental innovations: First, we develop a theoretically grounded framework using Singular Value Decomposition (SVD) to decouple HSIs into orthogonal spatial coefficients and spectral bases. This decomposition enables strategic reconfiguration of spatial coefficients into pseudo-RGB formats through band reorganization, facilitating direct deployment of frozen RGB-pretrained models for spatial textures recovery while preserving spectral integrity. Second, we propose a Physics-Motivated Spectral Rectifier (PMSR) that dynamically adjusts spectral reconstruction weights using spatial gradient priors, correcting spectral deviations through physics-consistent optimization rather than explicit error modeling, thereby achieving superior spectral fidelity. Comprehensive experiments confirm our method's superiority in both spatial reconstruction accuracy and spectral consistency over state-of-the-art techniques. Code is available at: https://github.com/QYo-Liu/CMDN.
Change-Prior-Guided Unsupervised Change Detection of Heterogeneous Remote Sensing Images.
Sun Y, Lei L, Kuang G
IEEE Trans Image Process · 2026 Jun · PMID 42378154 · Publisher ↗
Heterogeneous change detection (HeCD) enables the identification of land-cover changes using remote sensing imagery obtained from different sensors. Most existing methods overly emphasize modality transformation and shar...
Heterogeneous change detection (HeCD) enables the identification of land-cover changes using remote sensing imagery obtained from different sensors. Most existing methods overly emphasize modality transformation and shared feature extraction to bridge the gap between heterogeneous images. While these strategies facilitate comparable representations, they tend to neglect the intrinsic characteristics of the changes themselves, which limits their effectiveness in complex scenarios. To overcome this limitation, we propose a change prior-guided image transformation model (CPIT) for unsupervised HeCD. Specifically, starting from the definition of change detection, we analyze the connections among pairwise object relationships, change labels, and change semantics, and then derive change semantic consistency and inconsistency rules solely from the inherent nature of the change detection problem, without relying on data-specific assumptions. These rules are subsequently encoded as change semantic consistency and inconsistency constraints, which, from the perspective of graph signal processing, correspond to low-pass and high-pass spectral properties of the change signals. Finally, by integrating these semantic constraints with sparsity priors and image transformation constraints, we formulate a more precise transformation model for HeCD. Solving this model produces change detection results that conform to the change priors, thereby improving the detection performance. The derivation, formulation, and utilization of change priors in this work offer valuable insights for broader change detection research. Extensive experiments on five datasets validate the effectiveness of CPIT. The code will be released at https://github.com/yulisun/CPIT.
AgonicDreamer: Enhancing Multi-View Consistency in Text-to-3D Generation via Rectified Score Distillation.
Li Y, Tian F, Guan S … +5 more , Ge Y, Li W, Yan Y, Ma C, Yang X
IEEE Trans Image Process · 2026 Jun · PMID 42371881 · Publisher ↗
Score Distillation Sampling and its variants have shown strong potential in text-to-3D generation by leveraging scores estimated from pretrained text-to-image diffusion models to optimize 3D representations. However, due...
Score Distillation Sampling and its variants have shown strong potential in text-to-3D generation by leveraging scores estimated from pretrained text-to-image diffusion models to optimize 3D representations. However, due to the view-agnostic nature of these scores, existing methods often suffer from the multi-face Janus problem, leading to inconsistencies across different views. In this work, we propose Rectified Score Distillation, which addresses this issue by incorporating view-conditioned scores as priors. Specifically, we formulate a reverse ordinary differential equation (ODE) that is additionally conditioned on camera poses. Then we rectify the standard, view-irrelevant scores to approximate the desired gradients along this ODE. Building on these rectified scores, our full framework, named AgonicDreamer, enables the generation of photorealistic and multi-view consistent 3D content with fine-grained details, as validated by extensive experimental results.
BiCM-Prompt: Bidirectional Cross-Modal Prompt Tuning for Class-Incremental Learning on Multisource Remote Sensing Images.
Zou Y, Qu J, Dong W … +1 more , Li Y
IEEE Trans Image Process · 2026 Jun · PMID 42371880 · Publisher ↗
Class-incremental learning aims to continuously expand the category space while mitigating catastrophic forgetting of previously learned classes, and has recently attracted increasing attention in remote sensing image cl...
Class-incremental learning aims to continuously expand the category space while mitigating catastrophic forgetting of previously learned classes, and has recently attracted increasing attention in remote sensing image classification for long-term adaptive applications. However, most existing class-incremental learning methods are developed under single-source classification settings, where the complementary information across multiple data sources is not fully exploited, limiting their applicability in multi-source remote sensing scenarios classification. To address this issue, we propose a novel prompt-based framework for class-incremental learning on multi-source remote sensing images. Specifically, we introduce a Bidirectional Cross-Modal Prompt Tuning (BiCM-PT) module that dynamically selects modality-specific prompts while preserving historical cross-modal relationships by freezing modality relation projectors from previous tasks, thereby enhancing model stability. Furthermore, to improve plasticity for new-class learning, we design a Prompt-Guided Knowledge Aggregator (PGKA) that leverages learned prompts to guide decision-level feature aggregation and extract discriminative multi-modal representations. Together, these components enable effective and stable class-incremental learning in multi-source remote sensing environments. Extensive experiments on three real-world remote sensing benchmarks demonstrate the effectiveness of our approach in balancing stability and plasticity under multi-source incremental learning settings. The code is available at https://github.com/Jiahuiqu/BiCMPT.
GoP-based Quality Enhancement on Video Compression.
Lee S, Shin C, Kang HG … +1 more , Lee S
IEEE Trans Image Process · 2026 Jun · PMID 42371879 · Publisher ↗
With recent increases in the demand for high-resolution video content, it has become increasingly challenging to transmit video data within the constraints of limited band-width. Due to the time-consuming nature of devel...
With recent increases in the demand for high-resolution video content, it has become increasingly challenging to transmit video data within the constraints of limited band-width. Due to the time-consuming nature of developing and disseminating new standard codecs, a large body of research has addressed improving low-quality videos through post-processing techniques. Previous studies have primarily concentrated on enhancing the quality of compressed video by addressing the temporal consistency of adjacent frames over short durations. However, these approaches often overlook specific characteristics of the video coding framework, such as notable variations in codec artifact patterns occurring at the Group of Pictures (GoP) level, which can result in considerable viewer discomfort. In this paper, we propose GoP-based Quality Enhancement (GQE), which aims to improve the quality of compressed videos by addressing issues at the GoP level. First, we present a GoP Guided Feature Propagation (GGFP) module, which addresses the root cause of the GoP level issue by propagating features from the I-frame of a different GoP to the frames currently undergoing enhancement. Then, we introduce a Temporal Aggregation (TA) module to efficiently and effectively aggregate features from the I-frame and the current frame. We extensively evaluate our model using diverse test sequences across a range of codecs, including HEVC, VP9, and AV1. Our approach not only achieves a significant reduction in the pattern shifts of GoP-level artifacts, but also demonstrates a substantial improvement in overall video quality.
Align Then Tensorize: Multi-Level Consistent Anchor Graph Learning for Scalable Multi-View Clustering.
Tang C, Li M, Wang J … +5 more , Guan R, Wang S, Tang C, Zhu E, Liu X
IEEE Trans Image Process · 2026 · PMID 42371878 · Publisher ↗
Tensor-based multi-view clustering has been widely studied to capture high-order correlations among multiple views. Nevertheless, existing tensorial methods still exhibit several limitations. First, many approaches rely...
Tensor-based multi-view clustering has been widely studied to capture high-order correlations among multiple views. Nevertheless, existing tensorial methods still exhibit several limitations. First, many approaches rely on full similarity graphs, leading to quadratic or cubic complexity in the number of samples and poor scalability. Second, view-specific anchor graphs are often tensorized without cross-view anchor alignment, yielding structurally inconsistent tensor representations and reduced cross-view comparability. Third, low-rank regularization is typically imposed via the tensor nuclear norm (TNN), which uniformly shrinks singular values and may bias the estimation of the intrinsic tensor rank. To this end, we propose a novel framework, named Align then Tensorize: Multi-level Consistent Anchor Graph Learning for Scalable Multi-View Clustering (ATTMVC). It adopts an anchor-based graph learning framework in which each view is reconstructed from a small set of anchors with sample-wise sparse noise, substantially reducing computational complexity. Unlike existing tensor-based methods that directly tensorize unaligned view-wise anchor graphs, ATTMVC first aligns view-specific anchor graphs into a shared latent space, thereby enforcing structural consistency across views and enabling more reliable modeling of cross-view higher-order correlations. Furthermore, we introduce a Threshold Tensor Rank (TTR) surrogate on the aligned anchor graph tensor, which effectively promotes low-rank structure while mitigating the over-shrinking effect commonly caused by TNN-based regularization. Finally, extensive experiments demonstrate that ATTMVC outperforms state-of-the-art multi-view clustering methods. The code is publicly available at https://github.com/tangchuan2000/ATTMVC.
Beyond Fidelity: Diverse Image Synthesis via Retrieval-Augmented Diffusion.
Xia L, Wu B, Yang X … +4 more , Wu T, Zhao Y, Cai D, Liu W
IEEE Trans Image Process · 2026 Jun · PMID 42371877 · Publisher ↗
Image synthesis is a key application of generative AI. It can help reduce overfitting and the high cost of collecting real-world data for downstream discriminative models. However, current methods mainly focus on making...
Image synthesis is a key application of generative AI. It can help reduce overfitting and the high cost of collecting real-world data for downstream discriminative models. However, current methods mainly focus on making images look realistic and ignore their true goal: improving downstream model generalization and robustness. We find that existing approaches tend to produce high-fidelity synthetic images that closely resemble the original data. This limits their value for improving downstream task performance. To overcome this, we argue that diversity, not just fidelity, must guide synthetic data generation if it is to truly complement human-collected datasets. In this paper, we introduce a Retrieval-Augmented Generation framework for diverse diffusion-based image synthesis. At each generation step, we retrieve the top-K most similar samples in feature space from both real and previously generated images. We then apply an Anti-Attention mechanism that actively pushes the new image away from these retrieved samples in feature space, maximizing dissimilarity. We propose novel evaluation metrics to assess image synthesis diversity and demonstrate significant improvements over existing benchmarks. Moreover, downstream models trained with our synthetic data achieved a 1.9% absolute accuracy gain on standard benchmarks, outperforming existing synthesis techniques.
Multi-Branch Tree-based Fusion Neural Architecture Search with Zero-Cost Screen for Multi-Modal Classification.
Guo Q, Su Q, Liang X … +3 more , Qian Y, Li N, Cui Z
IEEE Trans Image Process · 2026 Jun · PMID 42371876 · Publisher ↗
Multi-modal classification (MMC) leverages effective fusion of information from diverse modalities to achieve superior classification performance. Existing fusion methods, however, rely on expert-designed architectures t...
Multi-modal classification (MMC) leverages effective fusion of information from diverse modalities to achieve superior classification performance. Existing fusion methods, however, rely on expert-designed architectures that demand substantial domain expertise and computational resources, with fixed topologies that offer limited structural flexibility across tasks and datasets. Although neural architecture search (NAS)-based fusion methods have been proposed to automatically discover high-performing architectures, these approaches are computationally expensive and predominantly rely on pairwise modality combinations, which fail to capture complex multi-variable correlations, limiting the architectures' expressiveness and flexibility. Consequently, there remains a lack of multi-modal fusion frameworks that can simultaneously achieve high efficiency and high accuracy. To break through these bottlenecks, we propose a multi-branch tree-based fusion neural architecture search framework (MBTF-NAS). For performance enhancement, MBTF-NAS employs a multi-branch tree-structured encoding strategy that enables dynamic and computationally efficient exploration of fusion topologies and substantially strengthens cross-modal interaction. A learnable model-level attention weighting mechanism further emphasizes informative modalities, improving the overall quality of multi-modal feature fusion. For efficiency improvement, MBTF-NAS leverages zero-cost proxy metrics for architecture evaluation, enabling rapid identification of high-potential candidates while dramatically reducing computational overhead. We conducted a comprehensive evaluation of MBTF-NAS on seven representative multi-modal benchmarks. The experimental results demonstrate that MBTF-NAS consistently outperforms state-of-the-art approaches, highlighting its effectiveness and generalizability.
Inter-LPCM: Learning-Based Inter-Frame Predictive Coding for LiDAR Point Cloud Compression.
Sun C, Yuan H, Jiang S … +3 more , Tian C, Zhang G, Hamzaoui R
IEEE Trans Image Process · 2026 · PMID 42371875 · Publisher ↗
Because LiDAR sensors acquire point clouds with a fixed angular resolution, the resulting data can be systematically parameterized and efficiently compressed in the spherical coordinate system. Traditional spherical coor...
Because LiDAR sensors acquire point clouds with a fixed angular resolution, the resulting data can be systematically parameterized and efficiently compressed in the spherical coordinate system. Traditional spherical coordinate-based point cloud compression methods have shown strong rate-distortion (RD) performance, with the predictive geometry coding (PredGeom) method in the geometry-based point cloud compression (G-PCC) standard being a prominent example. While PredGeom includes an inter-frame prediction mode, it relies on a simple linear model, which limits its ability to capture complex motion patterns or structural dependencies. On the other hand, existing learning-based compression methods in the spherical domain do not exploit inter-frame correlations to reduce geometry redundancy. To address these limitations, we propose a learning-based inter-frame predictive coding method (Inter-LPCM). For azimuth prediction, we use a delta coding strategy based on the predefined angular resolution. To improve compression for radii, we introduce an inter-frame radius predictive (Inter-RP) model that estimates the current point's radius using neighboring points from both the current frame and the registered reference frame. In addition, we design a lightweight attention-based prediction (LAEP) model to predict elevation angles by capturing long-range geometric correlations across different coordinates. For quantization, we propose an RD-optimized method to select the quantization steps in the spherical coordinate system. For entropy coding, we design distinct models for each spherical coordinate component. These models are adapted to the statistical priors of each coordinate, which enables more accurate probability estimation. Experimental results show that Inter-LPCM, in its best RD configuration, achieved a D1-PSNR BD-rate reduction of 26.1% compared with the G-PCC lossless octree-based coding mode on SemanticKITTI, and 8.3% compared with the inter-frame prediction mode of PredGeom on Ford, using the latest G-PCC test model TMC13 v31.0. Our source code is publicly available at https://github.com/SDUChangSun/Inter-LPCM.
CVDII: Enhancing One-Shot Skeleton Action Recognition Through Cross-View Dynamic Information Interaction.
Zhang Y, Chen J, Li T … +4 more , Zhang W, Li Z, Li B, Zhang W
IEEE Trans Image Process · 2026 · PMID 42371874 · Publisher ↗
One-shot 3D skeleton action recognition task struggles with diverse intra-class action execution styles, causing excessive discriminative information to obstruct obtaining separable feature space. We innovatively propose...
One-shot 3D skeleton action recognition task struggles with diverse intra-class action execution styles, causing excessive discriminative information to obstruct obtaining separable feature space. We innovatively propose leveraging shared information among intra-class action executions to mitigate the over-influence of discriminative information. To this end, we proposed dynamic information interaction module (DIIM) that enables shared information to effectively weaken excessive discriminative information. Specifically, DIIM facilitates effective information interaction by constructing a guided evolution pool to store execution-related shared information and ensure such information can be retrieved. We devise shared-discriminative projection strategy (SDPS) which adopts different feature extraction strategies for specific skeleton topologies to target mining discriminative and shared information from different views of skeleton data. In summary, our proposed Cross-View Dynamic Information Interaction (CVDII) framework integrates DIIM and SDPS, effectively tackles the problem of discriminative information redundancy caused by diverse intra-class action execution styles. Experiments conducted on NTU 60, NTU 120, PKU-MMD, and Kinetics datasets demonstrate that our proposed CVDII achieves remarkable performance.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe