Forskningsradar

Science Journals

Peer-reviewade publikationer — 63973 artiklar

Constrained MPC-Based Motion Planning for Morphing Quadrotors in Ultra-Narrow Passages under Limited Perception
arXiv:2605.15999v1 Announce Type: new Abstract: This paper introduces a motion planning framework to plan morphology and trajectory for morphing quadrotors under extremely constrained environments. We develop a novel obstacle avoidance cost function for nonlinear model predictive control (MPC) that enables navigation through extremely narrow gaps under limited perception from a 2D LiDAR. Classical artificial potential field-based costs typically have a high cost in narrow passages, artificially blocking the navigable path. In contrast, we propose a smooth exponential obstacle cost that preserves low traversal cost within narrow gaps while maintaining strong collision avoidance behavior. The formulation avoids hard activation thresholds and introduces a cost reduction factor to reduce the cost within narrow passages. Direct use of 2D LiDAR measurements in MPC allows navigation around arbitrarily shaped obstacles. The method is embedded within an acados-based nonlinear MPC framework. Simulation and experimental results demonstrate successful traversal of narrow corridors where typical repulsive cost functions would fail. The approach provides a computationally efficient and practical solution for navigating through tight spaces while maintaining safety from the obstacles. While we are implementing the framework on the morphing quadrotors, the cost function formulation is general-purpose for any mobile robot application, and is not limited to the morphing quadrotors. The implementation code is available at \href{https://github.com/harshjmodi1996/morphocopter_mpc}{Github Repo} and a short video is available at \href{https://zh.engr.tamu.edu/wp-content/uploads/sites/310/2026/03/MPC_MorphoCopter_video.mp4}{Video Link}.
Symplectic Neural Operators for Learning Infinite Dimensional Hamiltonian Systems
arXiv:2605.15881v1 Announce Type: cross Abstract: The modeling and simulation of infinite-dimensional Hamiltonian systems are central problems in mathematical physics and engineering, however they pose significant computational and structural challenges for standard data-driven architectures. In this work, we introduce the Symplectic Neural Operator, a neural operator architecture designed to preserve the symplectic structure intrinsic to Hamiltonian PDEs. We provide a theoretical characterization of their symplecticity and establish a rigorous long-term stability result based on the combination of symplectic structure preservation and learning accuracy. Numerical experiments on canonical Hamiltonian PDEs corroborate this theoretical result and show that SNOs exhibit improved energy behavior compared with non-structure-preserving neural operators.
From Flat Language Labels to Typological Priors: Structured Language Conditioning for Multilingual Speech-to-Speech Translation
arXiv:2605.16026v1 Announce Type: new Abstract: Compositional speech-to-speech translation (S2ST) systems built upon speech large language models (SpeechLLMs) have recently shown promising performance. However, existing S2ST systems often either neglect source-language information or encode it through a language-as-label paradigm, representing each source language as an independent flat embedding. Such a design overlooks systematic linguistic structure shared across languages, which may limit data-efficient multilingual adaptation when supervised S2ST data are scarce. To address this issue, we propose S2ST-Omni 2, a many-to-one compositional S2ST framework that systematically reformulates multilingual language conditioning from flat language labels to structured typological priors. Specifically, S2ST-Omni 2 revisits language conditioning at three levels: typology-informed hierarchical language encoding for structured source-language representation, dynamically-gated language-aware Dual-CTC for content-adaptive acoustic modulation, and typology-aware LLM prompting for decoder-side linguistic guidance. Experiments on CVSS-C show that S2ST-Omni 2 achieves superior average performance among representative S2ST approaches across BLEU, COMET, ASR-BLEU, and BLASER 2.0 under the adopted evaluation protocol. Ablation studies indicate that the proposed representation-level, acoustic-level, and decoding-level strategies provide complementary benefits. Moreover, controlled data-budget analyses and a Japanese-to-English evaluation using only approximately 3 hours of supervised training data suggest that explicit typological priors provide useful inductive biases for data-efficient multilingual S2ST.
On a linear DG approximation of chemotaxis models with damping gradient nonlinearities
arXiv:2501.13216v2 Announce Type: replace Abstract: In this work we present a novel linear and positivity preserving upwind discontinuous Galerkin (DG) approximation of a class of chemotaxis models with damping gradient nonlinearities. In particular, both a local and a nonlocal model including nonlinear diffusion, chemoattraction, chemorepulsion and logistic growth are considered. Some numerical experiments in the context of chemotactic collapse are presented, whose results are in accordance with the previous analysis of the approximation and show how the blow-up can be prevented by means of the damping gradient term.
Massively Degenerate Coherent Perfect Absorption in Gradient-Index Fibers
arXiv:2605.15998v1 Announce Type: new Abstract: Coherent perfect absorbers (CPAs) have recently attracted considerable attention due to their ability to enhance light--matter interaction. By exploiting interference, CPAs enable even weakly absorbing materials to achieve complete absorption under appropriate excitation conditions. Generalizing this concept to the simultaneous absorption of arbitrary multimode input states remains challenging, however, since conventional implementations typically operate only for a single or a very small number of input channels. Here, we propose a compact realization of a multimode coherent perfect absorber based on a gradient-index (GRIN) fiber. Using the self-imaging property of the fiber, the bulky free-space architecture of previous approaches is replaced by a monolithic waveguiding platform that supports near-degenerate rephasing of many spatial modes. We show that standard GRIN profiles optimized for minimal intermodal dispersion enable highly efficient absorption of complex multimode fields, with field-of-view reflectivities well below \(1\%\) for realistic parameters. This approach provides a practical and scalable route toward efficient multimode absorption in fiber-based and integrated photonic systems, with potential applications in light harvesting, optical control, and imaging.
Non-Markovian Light-Matter Dynamics in the Time Fractional Jaynes-Cummings Model with Modulated Coupling
arXiv:2507.16830v5 Announce Type: replace Abstract: We investigate the fractional time description of a generalized quantum light-matter system modeled by a time-dependent Jaynes-Cummings (JC) interaction, with different coupling types: constant, linear, exponential, and sinusoidal. Two formulations of the time fractional Schr\"odinger equation (TFSE) are examined, with a focus on their impact on population inversion and entanglement. Our findings highlight that the introduction of fractional order introduces memory effects, associated with damped oscillations and asymptotic decay. Furthermore, we find that the time-dependent couplings, combined with distinct fractional formulations, influence how these effects occur, ultimately resulting in high or low entanglement. A key finding of our work is that, under sinusoidal coupling, non-periodic dynamics is preserved for both formulations of the TFSE; however, within a certain range, the fractional order can act as a control mechanism for the non-periodic evolution.
MultiMat: Multimodal Program Synthesis for Procedural Materials using Large Multimodal Models
arXiv:2509.22151v3 Announce Type: cross Abstract: Material node graphs are programs that generate the 2D channels of procedural materials, including geometry such as roughness and displacement maps, and reflectance such as albedo and conductivity maps. They are essential in computer graphics for representing the appearance of virtual 3D objects parametrically and at arbitrary resolution. In particular, their directed acyclic graph structure and intermediate states enable a modular, interpretable workflow for interactive appearance modeling. However, creating such graphs remains challenging and typically requires professional training. While recent neural program synthesis approaches attempt to simplify this process, they solely represent graphs as textual programs, failing to capture the inherently visual-spatial nature of node graphs that makes them accessible to humans. To address this gap, we present MultiMat, a multimodal program synthesis framework that leverages large multimodal models to process both visual and textual graph representations for improved generation of procedural material graphs. We train our models on a new dataset of production-quality procedural materials and combine them with a constrained tree search inference algorithm that ensures static correctness while efficiently navigating the program space. Our experimental results show that our multimodal program synthesis method is more efficient in both unconditional and conditional graph synthesis with higher visual quality and fidelity than text-only baselines, establishing new state-of-the-art performance.
In-situ correlative SEM/KPFM for semiconductor devices and 2D heterostructures
arXiv:2605.16062v1 Announce Type: cross Abstract: Correlative nanoscale surface characterization benefits from simultaneously measuring electronic and structural properties in the same environment, a capability that is essential for modern-day materials science and semiconductor failure analysis. In-situ AFM-SEM measurements facilitated by self-sensing cantilevers offer great potential here; however, they are limited due to their inherent capacitive crosstalk. Here, we demonstrate for the first time the in-situ implementation of single-pass heterodyne Kelvin probe force microscopy inside a scanning electron microscope, using piezo-resistive cantilevers. We overcome the capacitive crosstalk prevalent in piezo-resistive cantilevers by demodulating excitation and detection to simultaneously map surface topography and contact potential difference for correlation with compositional analysis. We systematically compare different operational modes of this heterodyne technique, elucidating their spatial resolution, signal sensitivity, and signal-to-noise ratio. The integrated approach yields exceptional signal quality and reveals how electron beam scan parameters can directly influence surface potential contrast. We demonstrate this correlative analysis workflow on two-dimensional heterostructures and semiconductor circuits. This work establishes a robust and versatile correlative imaging mode for in-situ Kelvin force and topography imaging inside a scanning electron microscope for next-generation semiconductor device analysis and materials science.
Toward World Modeling of Physiological Signals with Chaos-Theoretic Balancing and Latent Dynamics
arXiv:2605.15465v1 Announce Type: new Abstract: Physiological time series signals reflect complex, multi-scale dynamical processes of the human body. Existing modeling studies focus on static tasks such as classification, event forecasting, or short-horizon next step prediction, while long-horizon signal-level forecasting and predictive nature of physiological signals remain underexplored. We introduce NormWear-2, a world model that encodes both multivariate physiological signals and clinical intervention variables into a shared latent space and models their joint temporal evolution as a dynamical system. Our approach combines inference from prior pre-trained knowledge (intuition) with instant non-parametric latent state transition adaptation (insight), enabling coherent forecasting across multiple temporal scales, conditioned on heterogeneous clinical interventions. During the pretraining phase, we find that chaos-theoretic balancing of dynamical regime diversity yields more robust representations, with a smaller balanced corpus outperforming one twice its size and capturing bifurcation regimes. We evaluate the world model performance across diverse real-world physiological datasets spanning heterogeneous temporal resolutions and intervention regimes, covering daily life, point-of-care, and clinical settings, including fitness planning, hemodialysis, diabetes management, and surgical monitoring. These evaluation datasets comprise records from 8,026 subjects, spanning study durations from 3.2 hours for high-resolution signal data to 2.3 years for longitudinal clinical biomarker tracking. NormWear-2 achieves the best overall forecasting performance across time, frequency, and latent representation domains, with significant improvements over state-of-the-art time series foundation models, while maintaining competitive downstream representation quality, providing a step toward general-purpose world models for physiological signals.
Whole-body motion planning and safety-critical control for aerial manipulation
arXiv:2511.02342v3 Announce Type: replace Abstract: Aerial manipulation combines the maneuverability of multirotors with the dexterity of robotic arms to perform complex tasks in cluttered spaces. Yet planning safe, dynamically feasible trajectories remains difficult due to whole-body collision avoidance and the conservativeness of common geometric abstractions such as bounding boxes or ellipsoids. We present a whole-body motion planning and safety-critical control framework for aerial manipulators built on superquadrics (SQs). Using an SQ-plus-proxy representation, we model both the vehicle and obstacles with differentiable, geometry-accurate surfaces. Leveraging this representation, we introduce a maximum-clearance planner that fuses Voronoi diagrams with an equilibrium-manifold formulation to generate smooth, collision-aware trajectories. We further design a safety-critical controller that jointly enforces thrust limits and collision avoidance via high-order control barrier functions. In simulation, our approach outperforms sampling-based planners in cluttered environments, producing faster, safer, and smoother trajectories and exceeding ellipsoid-based baselines in geometric fidelity. Actual experiments on a physical aerial-manipulation platform confirm feasibility and robustness, demonstrating consistent performance across simulation and hardware settings. The video can be found at https://youtu.be/hQYKwrWf1Ak.
ITGPT: Generative Pretraining on Irregular Timeseries
arXiv:2605.16069v1 Announce Type: new Abstract: Timeseries regression models often struggle to leverage large volumes of labeled multimodal data, particularly when the data are irregularly sampled or contain missing values. This is common in domains like healthcare and predictive maintenance, where data are collected from unreliable sources, and labeling requires expert knowledge or costly equipments. Transformer-based large language models have proven effective on structured data such as text through self-supervised learning (SSL) and generative pretraining (GPT) frameworks. However, such models lack the flexibility to efficiently process irregularly sampled multimodal timeseries data. In this paper, we introduce ITGPT, an attention-based architecture designed for handling multimodal, irregularly sampled timeseries by allowing training with both SSL losses and GPT-like objectives. We evaluate its performance on a healthcare task with the TIHM dataset, and a predictive maintenance task with the CompX dataset. Our results demonstrate that ITGPT achieves state-of-the-art performance without requiring resampling, feature fusion or explicit data imputation. Furthermore, when labels are scarce, ITGPT effectively leverages unlabeled data through SSL and GPT training, outperforming the purely supervised approach. This represents an important step towards efficiently using large and unstructured timeseries datasets for practical inference tasks.
Chladni states in Ising Spin Lattices
arXiv:2507.05961v2 Announce Type: replace-cross Abstract: Low-temperature spin dynamics can become trapped in long-lived patterns shaped by the geometry of the interaction network. Here we introduce Chladni states: spin configurations obtained by binarizing the eigenmodes of the interaction Laplacian. These graph-spectral patterns organize the metastable configurations reached by Ising systems under non-ergodic relaxation. The resulting Topological Mode Decomposition provides a compact way to monitor and reconstruct frozen spin configurations in ferromagnets, frustrated antiferromagnets, and spin glasses.
Scalable neuromorphic computing from autonomous spiking dynamics in a clockless reconfigurable chip
arXiv:2605.16114v1 Announce Type: new Abstract: We propose a scalable neuromorphic architecture based on spiking dynamics emerging from the autonomous time-continuous evolution of clockless (asynchronous) digital circuits. Implemented on commercially available field-programmable gate arrays (FPGAs), our system implements networks of interacting Boolean spiking neurons with configurable excitatory and inhibitory synaptic weights. A complete processing pipeline enables efficient handling of spike-encoded data for solving machine-learning tasks. We demonstrate competitive performance for an audio classification task with spike-based encoding and high-speed processing. Power consumption is significantly lower than traditional digital implementations; this makes our approach an efficient alternative that bridges the gap to dedicated analog neuromorphic systems without the need for specialized hardware design. More generally, our approach establishes clockless digital hardware as a viable platform for neuromorphic computing. It paves the way for reconfigurable chips to be turned into energy-efficient quasi-analog neuromorphic processors.
Ti-iLSTM: A TinyDL Approach for Logic-Level Anomaly Detection in Industrial Water Treatment Systems
arXiv:2605.15874v1 Announce Type: new Abstract: Industrial Water Treatment Systems (IWTS) are safety critical cyber-physical infrastructures and due to increased connectivity, these systems are exposed to cyber threats that can manipulate process behaviour without creating obvious devices outliers. In particular, logic-layer deception anomalies can preserve numerically plausible measurements while breaking expected cause-and-effect relationships in the control process. These attacks are difficult to detect using threshold-based monitoring or require heavy server-oriented anomaly detection models. This paper explores the potential of Tiny Deep Learning (TinyDL) to provide lightweight on-device logic-level anomaly detection for resource constrained Programmable Logic Controllers (PLCs). We propose a novel framework, TinyDL-based incremental LSTM (Ti-iLSTM) which optimises the memory and space foot print of Long Short-Term Memory (LSTM), to detect logic-layer inconsistencies in Programmable Logic Controller (PLC) based Industrial Water Treatment Systems (IWTS). Experiments on the publicly available SWaT dataset show that the optimised model achieves high detection performance (F1-score=0.983 and ROC-AUC=0.998). A deployment-style validation on the WADI dataset confirms that the proposed light-weight framework remains applicable beyond a single dataset. The research demonstrates that combining logic-aware supervision with Tiny Deep Learning (TinyDL) sequence learning creates an efficient and accurate anomaly detection suitable for resource constrained Programmable Logic Controllers (PLCs) in industrial environments.
FM-G-CAM: A Holistic Approach for Explainable AI in Computer Vision
arXiv:2312.05975v3 Announce Type: replace Abstract: Explainability is a vital aspect of modern AI for real-world impact and usability. The main objective of this paper is to emphasise the need to understand the predictions of Computer Vision models, specifically Convolutional Neural Network (CNN) models. Existing methods for explaining CNN predictions are largely based on Gradient-weighted Class Activation Maps (Grad-CAM) and focus solely on a single target class; this assumption about the target class selection neglects a large portion of the predictor CNN's prediction process. In this paper, we present an exhaustive methodology, called Fused Multi-class Gradient-weighted Class Activation Map (FM-G-CAM), that considers multiple top-predicted classes and provides a holistic explanation of the predictor CNN's rationale. We also provide a detailed mathematical and algorithmic description of our method. Furthermore, alongside a concise comparison of existing methods, we compare FM-G-CAM with Grad-CAM, quantitatively and qualitatively highlighting its benefits through real-world practical use cases. Finally, we present an open-source Python library with an FM-G-CAM implementation to conveniently generate saliency maps for CNN-based model predictions.
Skew-adaptive conformal prediction
arXiv:2605.16145v1 Announce Type: cross Abstract: We develop a skew-adaptive extension of split conformal prediction for regression. The method starts from an asymmetric interval family centered at a point prediction and uses the gauge approach to deduce the conformity score induced by this family. The inverse hyperbolic sine transform of signed scaled residuals provides the training target for an additional predictive model, whose role is to learn how predictive uncertainty should tilt across the feature space. The resulting procedure preserves the finite-sample marginal validity of split conformal prediction under exchangeability, while producing intervals that adapt to both local scale and local skewness. We also develop a calibration-sample-based estimator for comparing the expected relative future width of the skew-adaptive and classical scaled-score intervals. Experiments on a variety of datasets indicate gains in prediction interval efficiency over the scaled-score construction and conformalized quantile regression, and show that the proposed estimator closely matches the corresponding average width ratio observed on the test sample.
On the size of k-irreducible triangulations
arXiv:2603.20030v2 Announce Type: replace Abstract: A triangulation of a surface is k-irreducible if every non-contractible curve has length at least k and any edge contraction breaks this property. Equivalently, every edge belongs to a non-contractible curve of length k and there are no shorter non-contractible curves. We prove that a k-irreducible triangulation of an orientable surface of genus g has $O(k^2g)$ triangles, which is optimal. This is an improvement over the previous best bound $k^{O(k)} g^2$ of Gao, Richter and Seymour [Journal of Combinatorial Theory, Series B, 1996].
Locating nuclear-powered submarines with antineutrinos
arXiv:2605.15642v1 Announce Type: cross Abstract: Nuclear-powered submarines are difficult to track with conventional methods in congested waterways. We revisit antineutrino-based detection as a barrier concept, analogous to a neutrino-enabled SOSUS-style fence in strategic straits. Using analytic scaling relations and numerical estimates, we show that detectability depends primarily on closest approach, detector depth, and deployed mass. For representative assumptions, a 20\,kt detector in the Strait of Gibraltar reaches a local benchmark score $Z_A\simeq2.54$ for an assumed 100\,MW thermal-power sensitivity-study case in a conservative worst-case transit (with Poisson operating point $(P_\mathrm{FA},P_\mathrm{det})\simeq(5.5\times10^{-3},0.51)$ at threshold $k=2$), while a three-detector line raises the mapped score to $Z_A\simeq4.66$. For broad ocean passages such as GIUK, required detector counts are substantially larger; in the baseline maximum passing distance $\mathrm{PDD}_{\max}=5$\,km geometry, about 80 detectors yield only $Z_A\sim1.6$. The paper outlines detector technology choices, statistical assumptions, and deployment constraints for a first-generation feasibility assessment.
Large Language Models Could Be Rote Learners
arXiv:2504.08300v5 Announce Type: replace Abstract: Benchmark-based evaluation, e.g., multiple-choice questions (MCQs) and open-ended questions (OEQs), is widely used for evaluating Large Language Models (LLMs), yet their reliability is undermined by benchmark contamination. When pre-exposed to the testing benchmark during training, less capable LLMs have been found to achieve inflated performance, thereby yielding erroneous results in LLM evaluation. In this study, we reframe contamination as an inherent aspect of learning and seek to disentangle and expose genuine capability acquisition from superficial memorization in LLM evaluation. Following this, firstly, by analyzing model performance under different memorization conditions of MCQs, we uncover a counterintuitive trend: LLMs perform worse on memorized benchmarks than on non-memorized ones, indicating the coexistence of two learning phenomena, i.e., rote memorization and genuine capability learning. To disentangle them, we propose TrinEval, a novel evaluation framework that reformulates MCQs into an alternative knowledge-centric trinity format, reducing memorization while preserving inherent knowledge, enabling the evaluation of genuine capability in the presence of memorization. Extensive experiments validate the effectiveness and robustness of TrinEval in reformulating benchmarks, and the evaluation results further reveal that mainstream LLMs rely on rote memorization for an average of 19.6% of knowledge points across the MMLU and the GSM8K dataset.
MCAC: A Model Counting Algorithm for Exact Computation of Error Metrics of Approximate Circuits
arXiv:2411.10037v3 Announce Type: replace Abstract: Effective usage of approximate circuits for various performance trade-offs requires accurate computation of error. MCAC is a novel model counting framework for exact computation of several average and worst-case error metrics that are used to evaluate approximate circuits. Unlike other methods in the literature, our framework uses the same error miter for all metrics. It requires a single synthesis of the system consisting of the exact and approximate circuits followed by a subtractor that finds the difference of the two outputs. Existing miter-based methods require multiple calls to the model counter, one for each output of the miter. MCAC uses the CNF formula of the system to compute all metrics. Our algorithm converts the formula to a tree and uses message passing to compute all metrics. We propose data structures to efficiently store and perform sparse computations required for conversion to a tree and message passing. Results for all the error metrics for several benchmark instances show a significant speedup over using off-the-shelf model counters along with specialized miters for each metric.
Physics-Aware Machine-Learning-Driven Inverse Design of Broadband Ultra-Open Acoustic Metamaterials
arXiv:2605.16031v1 Announce Type: new Abstract: Ventilated acoustic silencers combing sound attenuation with high ventilation are pivotal for advanced noise control. However, balancing attenuation, bandwidth, openness, and thickness remains a high-dimensional challenge. Here, we report a physics-aware machine-learning-driven inverse design framework for ultra-open acoustic silencers (UAS). By leveraging Green's function-based parameterization, we physically decouple the design space into spectral and radial parameters, ensuring physical interpretability while reducing complexity. We introduce a two-stage forward prediction architecture that captures broadband envelopes and sharp resonant features via a coarse-to-fine strategy. Coupled with a population-based, hybrid-objective parallel (PHP) inverse strategy, our framework enables rapid exploration of non-convex landscapes, identifying hundreds of optimized candidates within seconds. Crucially, this framework uncovers hidden linear design rules that govern high-performance monolithic designs, acting as geometric proxies for optimal impedance-matching. We experimentally validate a family of prototypes: UAS-2 demonstrates the monolithic limit with high ventilation ratio, while UAS-3 demonstrates versatility in multi-mode interactions. To circumvent the trade-off ceiling of single-unit resonators, a parallel-composite architecture (UAS-4) is introduced to enhance performance through spatial interference distribution. Results confirm a broadband bandwidth exceeding 830 Hz achieved with an ultra-thin profile (0.1-0.2{\lambda}) and 80% ventilation. This work establishes a data-driven paradigm for discovering design principles in functional metamaterials.
ADS-IMC: Accelerating Data Sorting with In-Memory Computation
arXiv:2605.16213v1 Announce Type: new Abstract: Sorting is a fundamental operation across numerous computational domains. Traditionally, this process involves transferring data from main memory to a processing unit for sorting, followed by writing the sorted data back to memory. This conventional approach incurs substantial latency and energy overheads due to the extensive data movement between memory and processing components. To mitigate these overheads, this paper introduces novel architectures for executing sorting operations directly within the memory fabric, eliminating the need for off-chip data transfer. To our knowledge, this work represents the first exploration of in-memory sorting using 6T SRAM. The proposed architecture is designed to operate on data represented in the standard weighted binary radix format commonly used in digital systems. The proposed architecture achieves a significant 3.4x reduction in latency compared to memristor-based IMC sorting.
RaPD: Resolution-Agnostic Pixel Diffusion via Semantics-Enriched Implicit Representations
arXiv:2605.15908v1 Announce Type: new Abstract: Natural images are continuous, yet most generative models synthesize them on discrete grids, limiting resolution-flexible generation. Continuous neural fields enable resolution-free rendering, but prior methods introduce continuity only at the decoding stage as an interpolation module, leaving the generative latent space discretized and reconstruction-oriented. We propose RaPD (Resolution-agnostic Pixel Diffusion), which performs diffusion in a continuous Neural Image Field (NIF) latent space. RaPD bridges this reconstruction-generation gap with Semantic Representation Guidance for generation-aware latent learning and a Coordinate-Queried Attention Renderer for coordinate-conditioned, scale-aware rendering. A single denoised latent can be rendered at arbitrary resolutions by changing only the query coordinates, keeping diffusion cost fixed. Experiments demonstrate superior generation quality and resolution scalability.
Attention Dispersion in Dynamic Graph Transformers: Diagnosis and a Transferable Fix
arXiv:2605.16112v1 Announce Type: new Abstract: Transformer-based architectures have become the dominant paradigm for Continuous-Time Dynamic Graph (CTDG) learning, yet their performance remains limited on temporally shifted datasets. In this work, we identify attention dispersion as a shared failure mode of dynamic graph Transformers under temporal distribution shift. Through controlled ablation contrasting structurally and temporally distinguished historical neighbors against random ones, we show that prediction depends on a class of critical nodes that carry consistently more predictive signal than arbitrary neighbors. However, existing Transformers fail to focus on these nodes even when they are present in the input, as temporal shift weakens attention contrast and produces overly dispersed attention distributions. This diagnosis suggests a simple and transferable fix: replace standard attention with differential attention, which suppresses common-mode attention and amplifies distinctive token-level signals. When added to three representative CTDG Transformer baselines, differential attention consistently improves performance, with gains concentrated on high-shift datasets. Attention-level measurements further confirm the mechanism, showing reduced attention entropy and increased attention mass on critical nodes. Building on these findings, we introduce DiffDyG, a reference implementation combining differential attention with standard input encodings. Across 9 benchmarks and three negative sampling protocols, DiffDyG achieves SOTA performance, with especially large gains on the most shifted datasets.
Generative Long-term User Interest Modeling for Click-Through Rate Prediction
arXiv:2605.15905v1 Announce Type: new Abstract: Modeling long-term user interests with massive historical user behaviors enhances click-through rate (CTR) prediction performance in advertising and recommendation systems. Typically, a two-stage framework is widely adopted, where a general search unit (GSU) first retrieves top-$k$ relevant behaviors towards the target item, and an exact search unit (ESU) generates interest features via tailored attention. However, current target-centered GSU would ignore other latent user interests, leading to incomplete and biased interest features. Additionally, the matching-based retrieval process in GSUs depends on the pairwise similarity score between target item and each historical behavior, which not only becomes time-consuming for online services as user behaviors continue to grow, but also overlooks the interaction information among user behaviors. To combat these problems, we propose a \textbf{Gen}erative \textbf{L}ong-term user \textbf{I}nterest model named GenLI for CTR prediction. GenLI consists of an interest generation module (IGM), a behavior retrieval module (BRM), and an interest fusion module (IFM). The IGM generates multiple interest distributions to indicate different aspects of real-time user interests, which is target-independent and incorporates interaction information among behaviors, ensuring complete and diverse interest features. The BRM selects related behaviors via a simple lookup operation, reducing the time complexity for weighting each behavior to $O(1)$. Finally, the IFM uses delicate gating mechanisms to generate interest features. Based on the generation process, GenLI improves the diversity of user interests and avoids complex matching-based behavioral retrieval, achieving a better balance between accuracy and efficiency for CTR prediction.