arXiv:2601.08013v2 Announce Type: replace
Abstract: Accurate forecasts of segment-level sailing durations are fundamental to enhancing maritime schedule reliability and optimizing long-term port operations. However, conventional estimated time of arrival (ETA) models are primarily designed for the immediate next port of call and rely heavily on real-time automatic identification system (AIS) data, which is inherently unavailable for future voyage segments. To address this gap, the study reformulates future-port ETA prediction as a segment-level time-series forecasting problem. We develop a transformer-based architecture that integrates historical sailing durations, destination port congestion proxies, and static vessel descriptors. The proposed framework employs a causally masked attention mechanism to capture long-range temporal dependencies and a multi-task learning head to jointly predict segment sailing durations and port congestion states, leveraging shared latent signals to mitigate high uncertainty. Evaluation on a real-world global dataset from 2021 demonstrates the proposed model consistently outperforms a comprehensive suite of competitive baselines. The result shows a relative reduction of 4.70% in mean absolute error (MAE), 4.95% in mean absolute percentage error (MAPE) and 2.59% in root mean squared error (RMSE) compared with sequential deep learning models. The relative reductions compared with gradient boosting machines are 7.03% in MAE, 39.49% in MAPE and 4.37% in RMSE. The case study conducted on one major destination port further illustrates the model's superior accuracy.
Science Journals
arXiv:2605.17944v1 Announce Type: cross
Abstract: Rapid advancements in cloud based platforms providing access to quantum computing capabilities have opened up several challenges for efficient usage of these highly delicate and costly devices. Although most of the current systems use a priority based access protocol, they are unable to fully support reliable, efficient, and scalable execution of larger-scale applications. To overcome this limitation, we propose a comprehensive solution for efficient allocation of quantum programs to appropriate quantum devices, considering all the relevant cost metrics into account including, fidelity, execution time and communication overhead. We also formulate use-cases for distributed quantum workflow and propose modified graph based algorithms to solve for allocation of such use-cases, assuming a hybrid classical-quantum network. Since hardware advancements in large standalone devices is an ongoing process, it is critical to investigate such distributed workflows to maximize the best utilization of current NISQ devices. Our empirical study shows that the proposed techniques perform better than state-of-the-art methods for almost all evaluation parameters, with average improvements of approximately $5\%$ in execution time, $30\%$ in communication overhead, $40\%$ in wait time and $2\%$ in fidelity, providing better solutions to efficient allocation strategies.
arXiv:2605.18204v1 Announce Type: cross
Abstract: Discrete diffusion models are a powerful class of generative models with strong performance across many domains. For efficiency, however, discrete diffusion typically parameterizes the generative (reverse) process with factorized distributions, which makes it difficult for the model to learn the target process in a small number of steps and necessitates a long, computationally expensive sampling procedure. To reduce the gap between the target and model distributions and enable few-step generation, we propose Forward-Learned Discrete Diffusion (FLDD), which introduces discrete diffusion with a learnable forward (noising) process. Rather than fixing a Markovian forward chain, we adopt a non-Markovian formulation with learnable marginal and posterior distributions. This allows the generative process to remain factorized while matching the target defined by the noising process. We train all parameters end-to-end under the standard variational objective. Experiments on various benchmarks show that, for a given number of sampling steps, our approach produces a higher quality samples than conventional discrete diffusion models using the same reverse parameterization.
arXiv:2605.18223v1 Announce Type: new
Abstract: Mineral dissolution in porous media is classically partitioned into static regimes within the Pe-Da plane, but this framework fails to capture the dissolution behavior of structurally complex rocks. Using three-dimensional micro-continuum simulations on micro-CT images of three rock samples spanning a wide range of pore-space heterogeneity, we track the joint evolution of dissolution morphology, velocity distribution, and reaction rate. Our results reveal that initial flow heterogeneity controls accessibility of reactants, thereby controlling the dissolution regime,reshaping them as dynamic trajectories. Channeled dissolution emerges as a simultaneous reorganization of structure and flow, and the resulting permeability-porosity relationship cannot be captured by a single power-law. The effective power-law exponent increases with heterogeneity and changes over time, reaching a maximum of 9.8, 18.0, and 40.9 for the three samples. Consequently, the effective reaction rate falls one to three orders of magnitude below the uniform dissolution prediction, with the suppression scaling with flow heterogeneity due to mass transfer limitations in channeled dissolution.
arXiv:2605.18378v1 Announce Type: cross
Abstract: While video compression algorithms effectively reduce bitrate, aggressive quantization often compromises temporal coherence, introducing artifacts such as flicker, motion inconsistency, and unstable textures. Although spatial quality degradation is well-documented, the relationship between compression intensity and temporal stability remains insufficiently characterized. This paper systematically examines the progression of frame-to-frame coherence errors across different bitrate regimes, utilizing multiple codecs (AV1, HEVC, VP9, H.264) and content types. Our findings reveal that temporal consistency degrades non-linearly with increasing compression. Most critically, we identify a "Predictability anomaly" where sequences with unpredictable or irregular dynamics experience disproportionately higher instability than sequences with higher, but more predictable, motion magnitude. This challenges the conventional assumption that motion volume alone dictates encoding difficulty and highlights the necessity of temporal-aware metrics in compression pipelines.
arXiv:2605.18625v1 Announce Type: cross
Abstract: We study the near-kernel sector of the Hamiltonian constraint operator in the one-vertex model of quantum reduced loop gravity using variational Monte Carlo methods with neural quantum states. The analysis is based on the symmetric Hamiltonian containing both Euclidean and Lorentzian contributions, and on the variational minimization of the positive quadratic operator $\hat{\mathcal Q}=\hat C \hat C^\dagger$ in truncated Hilbert spaces with spin cutoff up to $j_{\mathrm{max}}=1001$. The resulting near-kernel states are found to organize into three qualitatively distinct classes. At low cutoffs, we find solutions that do not factorize across the three edge degrees of freedom. At larger cutoffs, we find two different factorized branches, both described to very high accuracy by products of one-edge wavefunctions but localized in different spin regimes. One of these branches is matched with near-unit fidelity by reduced Thiemann coherent states, providing evidence for an emergent semiclassical organization of the near-kernel sector. The other is likewise strongly factorized, but its one-edge factors are not well described by the same coherent-state family.
arXiv:2402.12705v5 Announce Type: replace
Abstract: Reconfiguration problems ask whether one feasible solution can be transformed into another by a sequence of local moves while maintaining feasibility throughout. For integers $d \geq 1$ and $k \geq d+1$, the Distance Coloring problem asks if a given graph $G$ has a $(d, k)$-coloring, i.e., a coloring of the vertices of $G$ by $k$ colors such that any two vertices within distance $d$ from each other have different colors.
For ordinary proper colorings ($d=1$), the $k$-Coloring Reconfiguration problem is polynomial-time solvable for $k\le 3$ [Cereceda, van den Heuvel, and Johnson, J. Graph Theory 67(1):69--82, 2011] but is $\mathsf{PSPACE}$-complete for every fixed $k\ge 4$, even on bipartite graphs [Bonsma and Cereceda, Theor. Comput. Sci. 410(50):5215--5226, 2009]. In this work, we initiate a study of the distance-$d$ analogue, for $d \geq 2$. We show that even for planar, bipartite, and $2$-degenerate graphs, $(d, k)$-Coloring Reconfiguration remains $\mathsf{PSPACE}$-complete for every $d \geq 3$ via a reduction from the well-known Sliding Tokens problem. Our construction uses $k = k_0 + 2 + n(\lceil d/2\rceil-1)$ colors on instances of size $n$, where $k_0\in\{3d+3,3d+6\}$ (depending on the parity of $d$). For $d = 2$, the same reduction scheme can be adapted to show that the problem is $\mathsf{PSPACE}$-complete on planar and $2$-degenerate graphs with same values of $k$. Additionally, on split graphs, there is an interesting dichotomy: the problem is $\mathsf{PSPACE}$-complete when $d = 2$ and $k$ is large but can be solved efficiently when $d \geq 3$ and $k \geq d+1$. For chordal graphs, we show that the problem is $\mathsf{PSPACE}$-complete for even values of $d \geq 2$. Finally, we design a quadratic-time algorithm to solve the problem on paths for any $d \geq 2$ and $k \geq d+1$.
arXiv:2411.10636v3 Announce Type: replace
Abstract: In this study, we investigate extrinsic gender bias in Bangla pretrained language models, a largely underexplored area in low-resource languages. To assess this bias, we construct four manually annotated, task-specific benchmark datasets for sentiment analysis, toxicity detection, hate speech detection, and sarcasm detection. Each dataset is augmented using nuanced gender perturbations, where we systematically swap gendered names and terms while preserving semantic content, enabling minimal-pair evaluation of gender-driven prediction shifts. We then propose RandSymKL, a randomized debiasing strategy integrated with symmetric KL divergence and cross-entropy loss to mitigate the bias across task-specific pretrained models. RandSymKL is a refined training approach to integrate these elements in a unified way for extrinsic gender bias mitigation focused on classification tasks. Our approach was evaluated against existing bias mitigation methods, with results showing that our technique not only effectively reduces bias but also maintains competitive accuracy compared to other baseline approaches. To promote further research, we have made both our implementation and datasets publicly available: https://github.com/sajib-kumar/Mitigating-Bangla-Extrinsic-Gender-Bias
arXiv:2602.07730v2 Announce Type: replace
Abstract: Across scientific disciplines, Laplacian eigenvectors serve as a fundamental basis for simplifying complex systems, from signal processing to quantum mechanics. In reinforcement learning (RL), they similarly form a basis over the state space, enabling reward functions to be approximated by projection onto a small set of eigenvectors. This projection makes zero-shot control possible, but it also imposes a fundamental limitation: the induced policies are only as expressive as the linear span of the chosen eigenvectors. We introduce the Laplacian Keyboard (LK), a hierarchical framework that goes beyond this linear span. LK constructs a task-agnostic library of behaviors from these eigenvectors, forming a behavior basis guaranteed to contain the optimal policy for any reward within the linear span. A meta-policy learns to stitch these behaviors dynamically, enabling efficient learning of policies outside the original linear constraints. We establish theoretical bounds on zero-shot approximation error and demonstrate empirically that LK improves over the zero-shot solution while achieving better sample efficiency compared to standard RL methods.
arXiv:2605.18257v1 Announce Type: new
Abstract: Multimodal representation alignment is pivotal for large language models and robotics. Traditional methods are often hindered by cross-modal information discrepancies and data scarcity, leading to suboptimal alignment spaces that overlook modality-unique features. We propose CodeBind, a framework that optimizes multimodal representation spaces through a modality-shared-specific codebook design. By incrementally aligning target and bridging modalities, CodeBind bypasses the need for fully paired data. Unlike traditional hard alignment, CodeBind decomposes features into shared components for semantic consistency and specific components for modality-unique details. This design utilizes a compositional vector quantization scheme, where a shared codebook bridges modality gaps and modality-specific codebooks mitigate representation bias by preventing dominant modalities from overshadowing others. Validated across nine modalities (text, image, video, audio, depth, thermal, tactile, 3D point cloud, EEG), CodeBind achieves state-of-the-art performance in multimodal classification and retrieval tasks.
arXiv:2605.18159v1 Announce Type: cross
Abstract: The Toffoli gate is a fundamental building block for quantum arithmetic and reversible logic, yet its efficient realization remains a major challenge in both near-term and fault-tolerant quantum architectures. Recent advances in dynamic quantum circuit capabilities, including mid-circuit measurement and classical feedforward, provide new opportunities for reducing the resource overhead of non-Clifford operations.
In this work, we propose a set of dynamic decomposition strategies for multi-controlled Toffoli gates that exploit adaptive circuit execution and ancilla-assisted constructions. Our methods systematically reduce entangling-gate count, T-count, and T-depth compared with conventional static decompositions, while preserving fault-tolerance guarantees. Through analytical cost models and experimental evaluation, we demonstrate that relative-phase primitives and measurement-conditioned corrections enable scalable implementations with improved depth and resource efficiency.
arXiv:2412.18158v2 Announce Type: replace
Abstract: Learned image compression methods have shown impressive performance but are often highly specialized for either human perception or specific machine vision tasks. This specialization limits their versatility and requires costly retraining for new applications. To address this, we introduce UniCodec, a universal codec built on a novel paradigm of semantic disentanglement at the encoder and compositional generation at the decoder. This framework is designed to simultaneously serve both human and machine needs, eliminating the need for task-specific retraining. At the encoder, UniCodec leverages pre-generated, task-specific label codebooks created by a Large Language Model (LLM). For any given task, a grounding model uses the corresponding codebook to perform task-aware disentanglement, compressing only the most relevant image regions. This mechanism not only saves significant bits but is also the key to our system's rapid, zero-retraining adaptation: switching to a new task is as simple as selecting a new codebook. The decoder then performs compositional generation: it combines the compact, disentangled components with powerful priors from a generative diffusion model. This process reconstructs a high-quality, complete image optimized with rich detail for human perception and precise features for machine vision tasks. Extensive experiments demonstrate that UniCodec consistently outperforms existing methods, effectively bridging the gap between human-centric and machine-centric compression.
arXiv:2605.18282v1 Announce Type: new
Abstract: Satellite Internet-of-Things (IoT) enables massive status-update services beyond terrestrial coverage, but grant-free uplink access creates a coupled freshness-control problem: increasing repetition and receiver-side diversity improves a device's capture-SIC opportunities, yet the resulting population congestion degrades network-wide freshness. Existing AoI-aware random-access models often rely on slot-synchronous collisions, fixed delivery probabilities, or scalar transmit-or-wait decisions and therefore cannot capture asynchronous satellite uplinks with capture and SIC. This paper develops a PHY-aware mean-field framework, termed ASTRA (Asynchronous Age-Aware Satellite Random Access), for freshness-driven satellite IoT random access. We build an access model that captures asynchronous arrivals, partial overlaps, capture, and SIC while preserving the dependence of delivery success on each device's repetition-diversity action. We then formulate the population interaction as a scalable mean-field MDP in which devices optimize access timing and intensity using only local AoI observations. The resulting system admits a mean-field equilibrium in which individual optimality and endogenous congestion are mutually consistent. We further prove that the optimal equilibrium policy admits an age-threshold structure. Numerical results show that the proposed policy reduces AoI relative to age-independent baselines.
arXiv:2503.13934v2 Announce Type: replace
Abstract: Mobile robot navigation in dynamic environments with pedestrian traffic is a key challenge in the development of autonomous mobile service robots. Recently, deep reinforcement learning-based methods have been actively studied and have outperformed traditional rule-based approaches owing to their optimization capabilities. Among these methods, those that assume continuous action spaces typically rely on Gaussian distributions, which limit the flexibility of the generated actions. In contrast, the application of diffusion models to reinforcement learning has advanced, enabling more flexible action distributions than Gaussian policy-based approaches. In this study, we apply a diffusion-based reinforcement learning approach to social navigation and validate its effectiveness. Furthermore, by exploiting the characteristics of diffusion models, we propose extensions that enable adaptation to previously unseen scenarios without additional training. As concrete scenario examples, we demonstrate adaptability to scenarios in which static obstacles exist in the environment that were not present during training, as well as scenarios in which the objective differs from training, such as accompanying target pedestrians while avoiding others to reach the destination.
arXiv:2602.20706v2 Announce Type: replace
Abstract: This paper introduces online algorithms with unreliable guidance (OAG), a model for ML-augmented online decision-making that cleanly separates the predictive and algorithmic components, thus offering a single, well-defined analysis framework that depends only on the problem at hand. Formulated through the lens of request-answer games, the OAG model brings multiple concepts (predictions from the answer space, guide, anytime competitiveness) which enable learning-augmented algorithms to be analyzed independently of predictor-specific choices - such as prediction semantics, error functions, or probing strategies - that would otherwise restrict the algorithm's generality and applicability. The clean framework of the OAG model allows to build the first generic compiler, the drop-or-trust-blindly (DTB) compiler, that turns almost any standard, prediction-free online algorithm into a learning-augmented one. Although simple, we show that the DTB compiler produces new learning-augmented algorithms with strong consistency-robustness guarantees for three classic online problems: we achieve new trade-offs for bipartite matching with adversarial arrival order, and obtain optimal solutions for caching and uniform metrical task systems.
arXiv:2603.01227v2 Announce Type: replace
Abstract: We propose the Lattice Representation Hypothesis of large language models: a symbolic backbone that grounds conceptual hierarchies and logical operations in embedding geometry. Our framework unifies the Linear Representation Hypothesis with Formal Concept Analysis (FCA), showing that linear attribute directions with separating thresholds induce a concept lattice via half-space intersections. This geometry enables symbolic reasoning through geometric meet (intersection) and join (union) operations, and admits a canonical form when attribute directions are linearly independent. Experiments on WordNet sub-hierarchies provide empirical evidence that LLM embeddings encode concept lattices and their logical structure, revealing a principled bridge between continuous geometry and symbolic abstraction.
arXiv:2605.18320v1 Announce Type: new
Abstract: Offline reinforcement learning methods typically enforce strict constraints to ensure safety; yet this rigidity often prevents the discovery of optimal behaviors outside the immediate support of the behavior policy. To address this, we propose Implicit Support Expansion via stochastic Policy optimization (ISEP), which leverages a value function interpolated between in-distribution data and policy samples to implicitly expand the feasible action support. This mechanism "densifies" high-reward regions, creating a navigable path for policy improvement while theoretically guaranteeing bounded value error. However, optimizing against this expanded support creates a multimodal landscape where standard deterministic averaging leads to mode collapse and invalid actions. ISEP mitigates this via a stochastic action selection strategy, optimizing the policy by stochastically alternating between conservative cloning and optimistic expansion signals. We instantiate this framework as ISEP-FM using Conditional Flow Matching utilizing classifier-free guidance to effectively capture the interpolated value signal.
arXiv:2509.22510v3 Announce Type: replace
Abstract: Alignment of Large Language Models (LLMs) is the ability to satisfy desired objectives during generation, which is critical for trustworthy deployment. In practice, alignment is often operationalized through multiple objectives such as Helpfulness, Harmlessness, and Honesty (HHH). Prior works study alignment via steering vectors in standard Transformer decoders but treat objectives in isolation, where optimizing a single objective can overwrite others, leading to interference. Recent works attempt to address this limitation by extending steering to a 1-to-N Transformer setting by replicating representations into objective-specific pathways, but apply transformations independently, resulting in inconsistent responses across objectives. Similarly, approaches such as safe RLHF and MoE-based designs study trade-offs across objectives but do not constrain objective-specific transformations within a shared representation during inference. As a result, even aligned State-of-the-Art (SOTA) LLMs can struggle to jointly satisfy HHH objectives in complex settings. To address this, we propose Adaptive Multi-Branch Steering (AMBS), a two-stage framework in a 1-to-N Transformer setting that parameterizes objective-specific transformations relative to a shared representation. In Stage I, a shared hidden representation is computed once. In Stage II, this representation is replicated into N pathways and updated relative to a shared reference, capturing objective-specific deviations while restricting divergence. This produces N objective-specific responses within a single forward pass, which can be combined at decoding to obtain a single response across objectives. Across multiple backbones, AMBS improves performance across HHH, with consistent gains in WR, TI, and SS (e.g., Avg 56.5% on LLaMA-2-7B) while maintaining efficiency (e.g., 189 Tok/s, 9 GPU-hrs).
arXiv:2605.18549v1 Announce Type: new
Abstract: Large Reasoning Models (LRMs) introduce new opportunities for safety monitoring through their Chain of Thought (CoT) reasoning. However, CoT is not always faithful to the model's final output, undermining its reliability as a monitoring tool. To address this, we investigate the hidden representations of LRMs to determine whether future behavior can be predicted from prompt and CoT representations. By evaluating a probe at each generated token, we construct a probe trajectory, the continuous evolution of a concept's probability across the reasoning process. We find that future model behavior is more distinguishable when examined over the full trajectory than from a single static prediction. To characterize these temporal dynamics, we extract signal-processing features that capture volatility, trend, and steady-state behavior, significantly improving the separation of future model states. We also present two methodological insights. First, template-based training data achieves near-parity with dynamically generated model responses, eliminating the need for a costly initial inference and labeling. Second, the choice of pooling operation is critical: average-pooling and last-token methods collapse to near-random performance, while max-pooling achieves up to 95% AUROC and yields stable probe trajectories. Using four datasets and four reasoning models across the domains of safety and mathematics, we demonstrate that trajectory features encode task-specific dynamics that improve outcome separability. These findings establish probe trajectories as a complementary framework for monitoring LRM behavior.
Warning: This article contains potentially harmful content.
arXiv:2512.14433v2 Announce Type: replace
Abstract: We report on the noise characterization of a free-running ring quantum cascade laser resonator emitting a single frequency mode around 7.7 $\mu$m. Using a gas cell filled with N$_2$O as a frequency-to-voltage discriminator, we measured the frequency noise power spectral density of the laser from which we extracted its linewidth. The results show a full width at half maximum close to 50 kHz at 1 s integration time, which represents at least a sixfold improvement compared to state-of-the-art quantum cascade lasers operating in a spectral region above 7 $\mu$m. We also demonstrate that such lasers can be efficiently used for frequency modulation spectroscopy, which opens up new possibilities for high resolution metrology and spectroscopic applications in the mid-infrared.
arXiv:2604.21937v2 Announce Type: replace
Abstract: Computational drug discovery, particularly the complex workflows of drug molecule screening and optimization, requires orchestrating dozens of specialized tools in multi-step workflows, yet current AI agents struggle to maintain robust performance and consistently underperform in these high-complexity scenarios. Here we present MolClaw, an autonomous agent that leads drug molecule evaluation, screening, and optimization. It unifies over 30 specialized domain resources through a three-tier hierarchical skill architecture (70 skills in total) that facilitates agent long-term interaction at runtime: tool-level skills standardize atomic operations, workflow-level skills compose them into validated pipelines with quality check and reflection, and a discipline-level skill supplies scientific principles governing planning and verification across all scenarios in the field. Additionally, we introduce MolBench, a benchmark comprising molecular screening, optimization, and end-to-end discovery challenges spanning 8 to 50+ sequential tool calls. MolClaw achieves state-of-the-art performance across all metrics, and ablation studies confirm that gains concentrate on tasks that demand structured workflows while vanishing on those solvable with ad hoc scripting, establishing workflow orchestration competence as the primary capability bottleneck for AI-driven drug discovery.
arXiv:2605.18553v1 Announce Type: new
Abstract: Recovering world space 4D motion of two interacting hands from egocentric video is a fundamental capability for supervising robot policy learning, where wrist trajectories track the end-effector and finger articulations specify the grasp pose. Two major challenges arise in this setting: hands frequently leave the camera view for extended periods due to head motion, and persistent hand-object interactions cause severe occlusions of one or both hands. Existing methods uniformly condition on noisy hand motion observations without accounting for their per-frame reliability, leading to substantial performance degradation. Our key insight is that accurate world space hand motion estimation is tightly coupled with the quality of per-frame hand observations. To this end, we decompose the quality of hand motion observations extracted from an off-the-shelf hand pose estimator into four channels: wrist global translation and finger articulations for both hands. We propose StableHand, a quality-aware flow-matching framework conditioned on these four-channel quality signals, which are predicted by a learned quality network. We naturally incorporate the quality signals into the flow-matching process through a per-channel forward schedule, a quality-adjusted velocity target, AdaLN modulation of the DiT denoiser, and a quality-aware ODE initialization. This unified generative process preserves high-quality observations while reconstructing unreliable ones using a learned bimanual motion prior. Experiments on HOT3D and ARCTIC, two egocentric benchmarks featuring long missing-hand spans and persistent hand-object occlusions, show that StableHand achieves state-of-the-art performance across all reported metrics, reducing W-MPJPE by 20-25% compared to the strongest baseline, with the largest gains on heavily occluded ARCTIC sequences.
arXiv:2605.10185v2 Announce Type: replace
Abstract: Ghost imaging reconstructs spatial information from a single-pixel bucket detector by correlating structured illumination patterns with scalar intensity measurements. While deep learning approaches have achieved promising results on static scenes, two critical limitations remain unaddressed: existing architectures fail to exploit temporal coherence across frames, leaving dynamic ghost imaging largely unsolved, and they assume additive Gaussian noise models that do not reflect the true Poissonian statistics of real single-photon hardware. We present DynGhost (Dynamic Ghost Imaging Transformer), a transformer architecture that addresses both limitations through alternating spatial and temporal attention blocks. Our quantum-aware training framework, based on physically accurate detector simulations (SNSPDs, SPADs, SiPMs) and Anscombe variance-stabilizing normalization, resolves the distribution shift that causes classical models to fail under realistic hardware constraints. Experiments across multiple benchmarks demonstrate that DynGhost outperforms both traditional reconstruction methods and existing deep learning architectures, with particular gains in dynamic and photon-starved settings.
arXiv:2605.10236v3 Announce Type: replace
Abstract: Modern off-policy reinforcement learning algorithms often rely on simple uniform replay sampling and it remains unclear when and why non-uniform replay improves over this strong baseline. Across diverse RL settings, we show that the effectiveness of non-uniform replay is governed by three factors: replay volume, the number of replayed transitions per environment step; expected recency, how recent sampled transitions are; and the entropy of the replay sampling distribution. Our main contribution is clarifying when non-uniform replay is beneficial and providing practical guidance for replay design in modern off-policy RL. Namely, we find that non-uniform replay is most beneficial when replay volume is low, and that high-entropy sampling is important even at comparable expected recency. Motivated by these findings, we adopt a simple Truncated Geometric replay that biases sampling toward recent experience while preserving high entropy and incurring negligible computational overhead. Across large-scale parallel simulation, single-task, and multi-task settings, including three modern algorithms evaluated on five RL benchmark suites, this replay sampling strategy improves sample efficiency in low-volume regimes while remaining competitive when replay volume is high.
arXiv:2605.18272v1 Announce Type: new
Abstract: Non-Hermitian lattices can host the non-Hermitian skin effect, a boundary-induced collapse of all bulk eigenstates into exponentially localized edge modes. This effect underlies anomalous bulk-boundary correspondence and remarkable enhancements in non-Hermitian sensing, yet direct energy-resolved access to the eigenmodes of non-Hermitian lattices has remained limited. Here we report band- and energy-resolved eigenmode spectroscopy of skin modes in a frequency synthetic dimension. By introducing strong frequency-domain boundaries in an electro-optically modulated ring resonator, we realize finite non-Hermitian lattices and use laser detuning as a spectroscopic axis for the eigenenergies of the effective Hamiltonian. Site-resolved heterodyne measurements then reconstruct the spatial profile of each mode, revealing boundary-localized skin states throughout the spectrum and their eigenenergy-dependent displacement from the edge. Beyond 1D, the same frequency-boundary architecture, upon incorporating long-range couplings between finite lattices, produces genuine 2D frequency lattices rather than the hitherto-realized folded 1D systems on twisted tubes. In these lattices we observe tunable directional transport and edge localization in two synthetic dimensions. Our results introduce eigenmode spectroscopy as a direct probe of non-Hermitian physics and establish strongly bounded frequency lattices as a flexible platform for Hamiltonian engineering.