arXiv:2605.15227v1 Announce Type: new
Abstract: Self-driving laboratories (SDLs) have attracted increasing attention as a means of accelerating scientific discovery; however, developing SDL software remains technically demanding. To improve accessibility, orchestration software frameworks have been proposed to coordinate SDL components. Nevertheless, existing frameworks are primarily designed for human interaction and do not provide standardized interfaces suitable for AI agents. In this work, we propose an SDL software architecture based on the Model Context Protocol (MCP), in which all SDL functionalities are exposed through MCP servers. Following this design principle, we introduce an MCP-based SDL orchestrator, named NIMO Controller. It provides a visual programming interface automatically generated through MCP-based tool discovery, allowing human users to design experimental workflows without writing code. The same MCP backend can also be accessed by AI agents, providing a unified interface for both human users and AI agents. We demonstrate the proposed system through a case study on a color-matching SDL. The results validate the usability of the proposed MCP-based SDL architecture.
Science Journals
arXiv:2511.21277v3 Announce Type: replace
Abstract: This paper presents LatencyScope, a mathematical framework for computing one-way uplink and downlink latency in fifth-generation radio access networks across diverse system configurations. LatencyScope models latency sources across the protocol stack, including radio interfaces, scheduling decisions, processing delays, frame structures, and hardware and software constraints, while capturing dependencies among configuration parameters and stochastic sources of delay. The framework also includes a configuration analyzer that uses these models to search billions of candidate settings and identify those that satisfy latency-reliability targets under user-specified constraints. We validate LatencyScope on two open-source fifth-generation radio access network testbeds, as well as on measurements from a public commercial fifth-generation network. The results show that LatencyScope closely matches empirical latency distributions, captures observed lower and upper latency bounds, and substantially outperforms prior analytical models and widely used fifth-generation network simulators. LatencyScope can determine whether ultra-reliable low-latency communication targets are feasible for a given deployment and, when they are feasible, efficiently find satisfying configurations, helping network operators reason about latency modeling, configuration analysis, and system-level bottlenecks.
arXiv:2603.29509v3 Announce Type: replace
Abstract: Molecular quantum sensors represent a promising frontier for the detection of nuclear magnetic resonance signals and alternating current magnetic fields at the nanoscale, potentially reaching single-proton sensitivity. Although the triplet states of molecular pentacene provide a viable sensing architecture, the triplet pair states produced by singlet fission of pentacene dimers could enable more flexible quantum manipulations through entanglement. In this work, we model the quantum sensing efficacy of a spin-polarized quintet manifold in a photoexcited pentacene dimer generated via intramolecular singlet fission. Using a Lindblad master equation approach, we simulate the evolution of the triplet pair state under standard dynamical decoupling sequences, including spin echo, XY4, and XY8 and provide a direct performance comparison to the traditional pentacene monomer benchmark. While both architectures exhibit comparable sensitivity for isolated single-spin detection, our findings indicate that the dimer architecture provides a superior interaction cross-section for detecting small ensembles of nuclear spins. Analytical expressions derived for fluorescence modulation demonstrate that sensitivity is optimized in the low-magnetic field regime and scales with the number of pulses in the sensing protocol. This study establishes a theoretical baseline for utilizing high-spin multi-excitonic states as chemically tunable, high-sensitivity quantum probes.
arXiv:2605.07523v1 Announce Type: cross
Abstract: In the imprecise geometry model, the input is an imprecise point set, which is a family of regions $F = (R_1, \ldots,R_n)$, where for each $R_i$ one may retrieve the true point $p_i \in R_i$. By preprocessing $F$, we can construct the output, in our case the Pareto front, on $P$ faster.
We efficiently construct the Pareto front of an imprecise point set in the plane. Efficiency is interpreted in two ways: minimizing (i) the number of retrievals, and (ii) the computation time used to determine the set of regions that must be retrieved and to construct the Pareto front.
We present an algorithm to construct the Pareto front for possibly overlapping rectangles that is \emph{instance-optimal} with respect to the number of retrievals, meaning that for every fixed input $(F, P)$, there is no algorithm that retrieves asymptotically fewer regions to compute the output. This is a strong algorithmic quality, as it means that our algorithm is competitive even to clairvoyant algorithms which know a correct guess of the output and only have to verify its correctness. In terms of algorithmic running time, instance-optimality is provably unobtainable. We instead present an algorithm which is within a $\log n$-factor of instance-optimality. This generalizes earlier results to overlapping input regions, at only a minor cost in running time.
For unit squares, we present an algorithm that is not only instance-optimal in the number of retrievals, but also \emph{universally} optimal in terms of running time, meaning that for any fixed set of regions $F$, no algorithm has a better worst-case running time for all possible point sets $P$. This is the first universally optimal algorithm for overlapping planar input. Compared to previous work, this result improves the degree of overlap, the preprocessing time, the number of retrievals, and the running time.
arXiv:2511.19399v3 Announce Type: replace
Abstract: Deep research agents perform multi-step research to produce long-form, well-attributed answers. However, most open deep research agents are trained on easily verifiable short-form QA tasks via reinforcement learning with verifiable rewards, which does not extend to realistic long-form tasks. We address this with Reinforcement Learning with Evolving Rubrics (RLER), where rubrics are constructed and maintained to co-evolve with the policy model during training. This allows the rubrics to incorporate newly explored information from search and contrasting model responses, enabling better fact checking and more discriminative on-policy feedback. Using RLER, we develop Deep Research Tulu (DR Tulu-8B), the first fully open model that is directly trained for open-ended, long-form deep research. Across four long-form deep research benchmarks in science, healthcare, and general domains, DR Tulu substantially outperforms existing open deep research agents (by 15.6% over Tongyi DR on average) and matches or exceeds proprietary deep research agents (by 0.7% over OpenAI DR on average), while being significantly smaller and cheaper per query (1000x cheaper than OpenAI DR per query).
arXiv:2605.15481v1 Announce Type: new
Abstract: InGaP-on-insulator, with its intrinsically high $\chi^{(2)}$ optical nonlinearity, has emerged as an efficient and bright integrated photonic platform for frequency conversion and on-chip entanglement generation, but high waveguide propagation loss in the visible wavelength range has limited its overall performance. Here, we identify the dominant loss mechanism through mode-profile analysis and effectively mitigate the loss using a surface treatment method. Statistical analysis of the resonator quality factor and propagation loss reveals the optimal ring radius that maintains a strong nonlinear interaction while suppressing significant bending related loss, resulting in loss as low as 0.49 dB/cm (4.31 dB/cm) at 1560 nm (780 nm). The method provides a 3.5--4$\times$ linear performance enhancement, enabling a second-harmonic generation efficiency of $3.01\times10^{5}$ %/W and a photon-pair generation rate of $11.7,\mathrm{MHz}/\mu\mathrm{W}$ and coincidence-to-accidental ratio as high as 10,000. The quasi-phase matching condition is experimentally verified, and nonlinear conversion is systematically characterized across the entire parameter space. This work establishes a scalable pathway for classical and quantum photonics in a low-loss, highly nonlinear, and wafer-scale integration platform.
arXiv:2511.18225v2 Announce Type: replace
Abstract: Quantum machine learning seeks to leverage quantum computers to improve upon classical machine learning algorithms. Currently, robust uncertainty quantification methods remain underdeveloped in the quantum domain, despite the critical need for reliable and trustworthy predictions. Recent work has introduced quantum conformal prediction, a framework that produces prediction sets that are guaranteed to contain the true outcome with a user-specified probability. In this work, we formalise how the time-varying noise inherent in quantum processors can undermine conformal guarantees, even when calibration and test data are exchangeable. To address this challenge, we draw on Adaptive Conformal Inference, a method which maintains validity over time via repeated recalibration. We introduce Adaptive Quantum Conformal Prediction (AQCP), an algorithm which provides asymptotic average coverage guarantees under arbitrary hardware noise conditions. Empirical studies on an IBM quantum processor demonstrate that AQCP achieves the target coverage level and exhibits greater stability than quantum conformal prediction.
arXiv:2604.02268v2 Announce Type: replace
Abstract: Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidance, injected skill content imposes substantial token overhead, and the model never truly acquires the knowledge it merely follows. We ask whether skills can instead be internalized into model parameters, enabling zero-shot autonomous behavior without any runtime skill retrieval. We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization. SKILL0 introduces a training-time curriculum that begins with full skill context and progressively withdraws it. Skills are grouped offline by category and rendered with interaction history into a compact visual context, teaching he model tool invocation and multi-turn task completion. A Dynamic Curriculum then evaluates each skill file's on-policy helpfulness, retaining only those from which the current policy still benefits within a linearly decaying budget, until the agent operates in a fully zero-shot setting. Extensive agentic experiments demonstrate that SKILL0 achieves substantial improvements over the standard RL baseline (+9.7\% for ALFWorld, +6.6\% for Search-QA, and+10.1\% for WebShop), while maintaining a highly efficient context of fewer than 0.5k tokens per step. Our code is available at https://github.com/ZJU-REAL/SkillZero.
arXiv:2605.15908v1 Announce Type: new
Abstract: Natural images are continuous, yet most generative models synthesize them on discrete grids, limiting resolution-flexible generation. Continuous neural fields enable resolution-free rendering, but prior methods introduce continuity only at the decoding stage as an interpolation module, leaving the generative latent space discretized and reconstruction-oriented. We propose RaPD (Resolution-agnostic Pixel Diffusion), which performs diffusion in a continuous Neural Image Field (NIF) latent space. RaPD bridges this reconstruction-generation gap with Semantic Representation Guidance for generation-aware latent learning and a Coordinate-Queried Attention Renderer for coordinate-conditioned, scale-aware rendering. A single denoised latent can be rendered at arbitrary resolutions by changing only the query coordinates, keeping diffusion cost fixed. Experiments demonstrate superior generation quality and resolution scalability.
arXiv:2605.15436v1 Announce Type: new
Abstract: This paper presents a comprehensive analysis of neural activation patterns across six distinct large language model (LLM) architectures, examining their performance on twelve cognitive task categories. Through systematic measurement of final activation values, attention entropy, and sparsity patterns, we reveal fundamental differences in how encoder and decoder architectures process diverse cognitive tasks. Our analysis of 144 task-model combinations demonstrates that mathematical reasoning consistently produces the highest attention entropy across all architectures, while decoder models exhibit significantly higher sparsity patterns compared to encoder models. The findings provide critical insights into the computational characteristics of modern language models and their task-specific neural behaviors, with implications for model selection and optimization in big data applications.
arXiv:2512.06655v3 Announce Type: replace
Abstract: Sparse autoencoders (SAEs) are increasingly used to extract activation directions for inference-time steering, but their standard sparsity objective treats latent features as independent. This prior can be poorly matched to high-level safety behaviors, where refusal and harmful compliance appear to depend on distributed structure in activation space. We introduce Graph-Regularized Sparse Autoencoders (GSAE), a dictionary-learning method that learns safety-steering directions by smoothing SAE decoder vectors over a neuron co-activation graph and applying the resulting direction bank through a two-gate runtime controller. Empirically, GSAE improves selective refusal across JailbreakBench, HarmBench, and XSTest, increasing harmful-request refusal while keeping benign-prompt refusals low. On Llama-3-8B, replacing the standard SAE with GSAE in an otherwise identical pipeline improves $\Delta_s$ by $20.1$ points on JailbreakBench and $16.8$ points on HarmBench. GSAE outperforms activation-steering baselines and black-box guardrails, preserves benign-task performance, generalizes across Llama-3, Mistral, Qwen 2.5, and Phi-4, and remains strong under black-box and gray-box jailbreak attacks.
arXiv:2505.19241v2 Announce Type: replace
Abstract: The recent success in using human preferences to align large language models (LLMs) has significantly improved their performance in various downstream tasks, such as question answering, mathematical reasoning, and code generation. However, achieving effective LLM alignment depends on high-quality datasets of human preferences. Collecting these datasets requires human preference annotation, which is costly and resource-intensive, necessitating efficient active data selection methods. Existing methods either lack a strong theoretical foundation or depend on restrictive assumptions about the reward function, such as linear latent reward functions. To this end, we propose an algorithm, ActiveDPO, that uses a theoretically grounded data selection criterion for non-linear reward functions while directly leveraging the LLM itself to parameterize the reward model used for active data selection. As a result, ActiveDPO explicitly accounts for the LLM's influence on data selection, unlike methods that select data without considering the LLM that is being aligned, thereby leading to more effective and efficient data collection. Our extensive experiments demonstrate that ActiveDPO outperforms existing methods across various models and real-world preference datasets.
arXiv:2605.15804v1 Announce Type: new
Abstract: This paper analyzes the security of the Message Queuing Telemetry Transport (MQTT) protocol in the context of the Internet of Things (IoT). The main objective consists of identifying vulnerabilities and proposing security improvements. Adopting a hybrid methodology, a theoretical review was combined with an experimental demonstration in a simulated Smart Home environment. Eavesdropping, Tampering, Denial of Service (DoS), and Brute Force attacks were executed and analyzed. The results evidenced critical risks due to the absence of robust encryption and authentication. Finally, mitigation strategies and best practices are proposed to strengthen MQTT implementations.
arXiv:2605.15437v1 Announce Type: new
Abstract: Extensive data processing is becoming commonplace in many fields of science. Distributing data to processing sites and providing methods to share the data with collaborators efficiently has become essential. The Open Science Data Federation (OSDF) builds upon the successful StashCache project to create a global data access network. The OSDF expands the StashCache project to add new data origins and caches, access methods, monitoring, and accounting mechanisms. Additionally, the OSDF has become an integral part of the U.S. national cyberinfrastructure landscape due to the sharing requirements of recent NSF solicitations, which the OSDF is uniquely positioned to enable. The OSDF continues to be utilized by many research collaborations and individual users, which pull the data to many research infrastructures and projects.
arXiv:2511.18127v2 Announce Type: replace
Abstract: Real-time 3D hand forecasting is a critical component for fluid human-computer interaction in applications like AR and assistive robotics. However, existing methods are ill-suited for these scenarios, as they typically require offline access to accumulated video sequences and cannot incorporate language guidance that conveys task intent. To overcome these limitations, we introduce SFHand, the first streaming framework for language-guided 3D hand forecasting. SFHand autoregressively predicts a comprehensive set of future 3D hand states, including hand type, 2D bounding box, 3D pose, and trajectory, from a continuous stream of video and language instructions. Our framework combines a streaming autoregressive architecture with an ROI-enhanced memory layer, capturing temporal context while focusing on salient hand-centric regions. To enable this research, we also introduce EgoHaFL, the first large-scale dataset featuring synchronized 3D hand poses and language instructions. We demonstrate that SFHand achieves new state-of-the-art results in 3D hand forecasting, outperforming prior work by a significant margin of up to 35.8%. Furthermore, we show the practical utility of our learned representations by transferring them to downstream embodied manipulation tasks, improving task success rates by up to 13.4% on multiple benchmarks. Dataset page: https://huggingface.co/datasets/ut-vision/EgoHaFL, project page: https://github.com/ut-vision/SFHand.
arXiv:2511.08103v4 Announce Type: replace
Abstract: Highly correlated photon sources can be realized through cooperative coupling among quantum systems, giving rise to superradiant collective emission. In solid-state ensembles, however, such collective behaviour is confined to subwavelength dimensions and is strongly suppressed at room temperature by inhomogeneous broadening and rapid dephasing, limiting practical implementations. Here, we show that molecular J-aggregates sustain room temperature superradiant emission and enter a highly collective regime when strongly coupled to delocalized photonic modes of a silicon bound-state-in-the-continuum (BIC) metasurface, extending J-exciton interactions far beyond the subwavelength limit. This enhanced cooperativity produces a photonic-fraction-dependent increase in emission rate and intensity and drives the system into a highly superbunched photon emission regime with g^((2))(0)>13. Spatial coherence measurements and stochastic modelling reveal that metasurface-mediated synchronization of ~10^3 J-excitons occurs within coupled superradiant domains spanning up to 6.7 um in diameter, corresponding to a 50-fold increase in inter-aggregate cooperative distance. These results establish exciton-polaritons in resonant dielectric metasurfaces as a platform to enhance superradiant emission and engineer temporally correlated light sources with picosecond-scale emission dynamics operating at room temperature.
arXiv:2605.15791v1 Announce Type: new
Abstract: The Shared Prosperity Internet (SPI) is a network-computing architecture that makes the benefits of automation and Artificial Intelligence (AI) broadly accessible to the society. To ground its design, this paper maps the physical constraints of Shannon, Landauer, Turing, and Einstein to three design principles: trustworthiness, sustainability, and technological sovereignty, and maps them into three technical pillars: i) post-Shannon, goal-oriented communication that transmits only what the task requires; ii) anticipatory decision-making ("negative latency") with confidence-bounded pre-action and correction; and iii) beyond-digital computing that selects energy-optimal substrates under deadline and computability constraints. The SPI is grounded in three societal use cases: remote teaching for pupils, remote teaching of robots and cyber-physical systems, and elder care. Furthermore, this paper defines measurable outcomes for an SPI, including latency decomposition, bits per event, energy and CO2 per task, safety and privacy indicators, and robustness.
arXiv:2605.06390v3 Announce Type: replace
Abstract: A leading proposal for aligning artificial superintelligence (ASI) is to use AI agents to automate an increasing fraction of alignment research as capabilities improve. We argue that, even when research agents are not scheming to deliberately sabotage alignment work, this plan could produce compelling but catastrophically misleading safety assessments resulting in the unintentional deployment of misaligned AI. This could happen because alignment research involves many hard-to-supervise fuzzy tasks (tasks without clear evaluation criteria, for which human judgement is systematically flawed). Consequently, research outputs will contain systematic, undetected errors, and even correct outputs could be incorrectly aggregated into overconfident safety assessments. This problem is likely to be worse for automated alignment research than for human-generated alignment research for several reasons: 1) optimisation pressure means agent-generated mistakes are concentrated among those that human reviewers are least likely to catch; 2) agents are likely to produce errors that do not resemble human mistakes; 3) AI-generated alignment solutions may involve arguments humans cannot evaluate; and 4) shared weights, data and training processes may make AI outputs more correlated than human equivalents. Therefore, agents must be trained to reliably perform hard-to-supervise fuzzy tasks. Generalisation and scalable oversight are the leading candidates for achieving this but both face novel challenges in the context of automated alignment.
arXiv:2511.04484v2 Announce Type: replace
Abstract: We study the repeated optimal stopping problem, in which the same optimal stopping instance with an unknown distribution is solved repeatedly over $T$ rounds. We aim to simultaneously achieve strong per-round performance guarantees relative to a given baseline and sublinear regret across all rounds.
Our primary contribution is a comprehensive theoretical characterization of whether and when these two objectives are compatible. First, under standard semi-bandit feedback, we prove that maintaining the per-round guarantee forces regret of $\Omega(T / \log T)$. Second, even under full feedback, we show that requiring almost-sure satisfaction of the per-round guarantee in every round is incompatible with sublinear regret. Third, under full feedback, we propose a general algorithmic framework that achieves both sublinear regret and the per-round guarantee with high probability.
Our framework applies to canonical problems, including the prophet inequality, the secretary problem, and their variants under adversarial, random, and i.i.d. input models. For example, in the repeated prophet inequality problem, our method guarantees that, with high probability in each round, its expected reward is at least that of the classical single-sample algorithm, which achieves a $1/2$ competitive ratio, while simultaneously ensuring $\tilde{O}(\sqrt{T})$ regret.
Furthermore, we establish a regret lower bound of $\Omega(\sqrt{T})$ even in the i.i.d. model, which is nearly tight with respect to the number of rounds.
arXiv:2605.15795v1 Announce Type: new
Abstract: We study real-time reconstruction and actuation for two binary Markov sources that share a wireless multi-packet reception (MPR) channel. Each sensor follows a stationary randomized sampling policy, and the receiver maintains source estimates using the most recently decoded updates. We derive closed-form expressions for the steady-state real-time reconstruction error (RTE) and the cost of actuation error (CAE) as functions of the source transition probabilities and the effective update probabilities. We then characterize these update probabilities under randomized sampling, linking the physical-layer MPR model to task-oriented reconstruction and actuation metrics. Using these expressions, we formulate a sampling-constrained optimization problem with a weighted-error objective. The resulting analysis reveals how source dynamics, semantic weights, and MPR coupling affect the allocation of sampling resources. Numerical results show that optimized randomized sampling outperforms random, greedy, and time-sharing baselines.
arXiv:2605.15498v1 Announce Type: new
Abstract: From a new perspective, this paper rederives Lagrange's equations. By applying the chain rule of differentiation, the intrinsic relationship between the momentum theorem and the kinetic energy theorem is first established. Subsequently, expressing the differential form of energy conservation in an arbitrary coordinate system and performing suitable differential operations yields Lagrange's equations. Generalized forces and generalized displacements are shown to be component representations of forces and displacements in a chosen coordinate system. Consequently, the essence of Lagrange's equations is identified as the transformation of the kinetic energy theorem into the momentum theorem via the chain rule for composite functions, thereby revealing how energy conservation constructs momentum conservation.
arXiv:2604.09631v2 Announce Type: replace
Abstract: As deep learning models are deployed on resource constrained edge platforms in autonomous driving systems, reli able knowledge of hardware behavior under resource degradation becomes an essential requirement. Therefore, we introduce a systematic characterization of CPU load, GPU utilization, RAM consumption, power draw, throughput, and thermal behaviour of TensorRT-optimized YOLOv10s, YOLOv11s and YOLO2026n pipelines running on NVIDIA Jetson Nano under a large-scale fault injection campaign targeting both lane-following and ob ject detection tasks. Faults are synthesized using a decoupled framework that leverages large language models (LLMs) and latent diffusion models (LDMs), based on original data from our JetBot platform data collection. Results show that across both tasks and both models the inference engines keep GPU occupancy stable, temperature rise under control, and power consumption within safe limits, while memory usage settles into a consistent release pattern after the initial warm-up phase. Object detection tends to show somewhat more variability in memory and thermal behavior, yet both tasks point to the same conclusion: the TensorRT pipelines hold up well even when the input data is heavily degraded. These findings offer a hardware-level view of model reliability that sits alongside, rather than against, the broader body of work focused on inference performance at the edge.
Don't Retrieve, Navigate: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG
arXiv:2604.14572v3 Announce Type: replace
Abstract: Retrieval-Augmented Generation (RAG) grounds LLM responses in external evidence but treats the model as a passive consumer of search results, with no view of how the corpus is organized or what it has not yet seen. We present Corpus2Skill, which distills a document corpus offline into a hierarchical skill directory and lets an LLM agent navigate it at serve time, drilling from a bird's-eye view through progressively finer summaries down to documents, and backtracking when a branch is unproductive. On an enterprise customer-support benchmark, Corpus2Skill improves both answer quality and grounding over single-shot dense, hybrid, hierarchical-retrieval, and agentic RAG baselines at a moderate cost tradeoff. A ten-subset generalization study further shows that corpus navigation is not a universal replacement for retrieval: it consistently helps on single-domain corpora with a recoverable topical taxonomy, but flat retrieval remains preferable on open-domain factoid pools or homogeneous-tabular corpora that defeat top-level clustering. We characterize this scope distinction and discuss it as a design guideline for knowledge-grounded systems. Code is available at https://github.com/dukesun99/Corpus2Skill.
Beyond the Training Data: Confidence-Guided Mixing of Parameterizations in a Hybrid AI-Climate Model
arXiv:2510.08107v5 Announce Type: replace
Abstract: Persistent systematic errors in Earth system models (ESMs) arise from difficulties in representing the full diversity of subgrid, multiscale atmospheric convection and turbulence. Machine learning (ML) parameterizations trained on short high-resolution simulations show strong potential to reduce these errors. However, stable long-term atmospheric simulations with hybrid (physics + ML) ESMs remain difficult, as neural networks (NNs) trained offline often destabilize online runs. Training convection parameterizations directly on coarse-grained data is challenging, notably because scales cannot be cleanly separated. This issue is mitigated using data from superparameterized simulations, which provide clearer scale separation. Yet, transferring a parameterization from one ESM to another remains difficult due to distribution shifts that induce large inference errors. Here, we present a proof-of-concept where a ClimSim-trained, physics-informed NN convection parameterization is successfully transferred to ICON-A. The scheme is (a) trained on adjusted ClimSim data with subtracted radiative tendencies, and (b) integrated into ICON-A. The NN parameterization predicts its own error, enabling mixing with a conventional convection scheme when confidence is low, thus making the hybrid AI-physics model tunable with respect to observations and reanalysis through mixing parameters. This improves process understanding by constraining convective tendencies across column water vapor, lower-tropospheric stability, and geographical conditions, yielding interpretable regime behavior. In AMIP-style setups, several hybrid configurations outperform the default convection scheme (e.g., improved precipitation statistics). With additive input noise during training, both hybrid and pure-ML schemes lead to stable simulations and remain physically consistent for at least 20 years.
arXiv:2510.06062v2 Announce Type: replace
Abstract: Reinforcement learning (RL) has shown great promise in large language models (LLMs) post-training, which typically rely on token-level clipping to maintain stability during optimization. Despite the empirical success of GRPO-style methods, we identify a fundamental and previously overlooked challenge in this popular Outcome-Supervised RL (OSRL) paradigm. We reveal that in OSRL, where advantages are shared across tokens within a response, importance sampling (IS) ratios deviate from their traditional purpose of distribution correction as in classic RL, which become token-level weights that allocate the shared advantage signal across tokens. We show that this hidden role shift induces a critical mismatch for positive-advantage tokens, leading to unbalanced token weighting between positive and negative tokens. Specifically, it suppresses the update of underrepresented tokens that are lagging behind, while over-amplifying already high-probability tokens. This mismatch results in rich-get-richer dynamics that over-reinforce confident tokens, weaken catch-up learning that drive entropy collapse, excessive repetition, and premature convergence. To address this, we propose Asymmetric Importance Sampling Policy Optimization (ASPO), a simple yet effective strategy that reverses the ratio-induced weighting of positive-advantage tokens, while stabilizing extreme updates and maintaining gradient flow. This mismatch correction aligns their update direction with the learning dynamics of negative ones. Comprehensive experiments across math reasoning and coding benchmarks demonstrate that ASPO significantly mitigates entropy collapse, improves training stability, and enhances performance over strong GRPO-based baselines. Our analysis provides new insights into the role of token-level weighting in OSRL and highlights the critical importance of correcting ratio-induced weighting in LLM RL.