arXiv:2504.08430v3 Announce Type: replace
Abstract: This paper presents a hybrid modeling approach that couples an Agent-Based Model (ABM) with a partial differential equation (PDE) model in an epidemic setting to simulate the spatial spread of infectious diseases using a compartmental structure with seven health states. The goal is to reduce the computational complexity of a full-ABM by introducing a coupled ABM-PDE model that offers significantly faster simulations while maintaining comparable accuracy. Our results demonstrate that the hybrid model not only reduces the overall simulation runtime (defined as the number of runs required for stable results multiplied by the duration of a single run) but also achieves smaller errors across both 25% and 100% population samples. The coupling mechanism ensures consistency at the model interface: agents crossing from the ABM into the PDE domain are removed and represented as density contributions, while surplus density in the PDE domain is used to generate agents with plausible trajectories derived from mobile phone data. We evaluate the hybrid model using real-world mobility and infection data for the Berlin-Brandenburg region in Germany, showing that it captures the core epidemiological dynamics while enabling efficient large-scale simulations. These results demonstrate that the proposed ABM-PDE framework provides a robust and computationally efficient alternative to full-scale agent-based simulations, making it suitable for realistic epidemic modeling and scenario analysis.
Science Journals
arXiv:2504.20583v3 Announce Type: replace
Abstract: We consider double ionization of negative bromine ion in intense low-frequency electromagnetic fields. By solving numerically the two-electron time-dependent Schr{\" o}dinger equation we demonstrate that while for pulses of a few tens of femtoseconds duration and longer the sequential single-electron approximation perfectly describes the ionization dynamics, for pulses as short as a few femtoseconds this picture breaks down entirely, and the electron-electron interaction suppresses the rate of ionization by roughly one order of magnitude. We also show clear signatures of the collective tunneling effect in the photoelectron density distribution. This counter-intuitive channel of ionization opens up due to the electron-electron repulsion in the direction lateral to the applied electric field.
arXiv:2607.07217v1 Announce Type: new
Abstract: Smart contract compilers are critical to ensuring the correctness of public blockchains whose defining characteristics are open-source and immutable code. We created SolSmith, a semantics-aware differential fuzz testing tool, to improve the quality of the Solidity compiler -- the most popular compiler for the Ethereum blockchain -- and spent over three years finding compiler defects that produce incorrect code. We call these defects miscompilation bugs. During this time period, we have discovered 25 miscompilation bugs that went unnoticed, some for multiple years. Our first contribution is to make compiler testing more rigorous. SolSmith achieves this goal by generating valid test programs that are likely to stress test code generation and optimization components. This helps SolSmith find bugs missed during routine testing that could potentially have serious implications for smart contracts and their users. Our second contribution is a qualitative and quantitative analysis of miscompilation bugs that we found in the Solidity compiler. We classify miscompilation bugs found by SolSmith based on their nature, root-causes, and impact on end-users. This sheds light on some pitfalls of optimizing compilers.
arXiv:2506.00473v4 Announce Type: replace
Abstract: The diurnal cycle of precipitation near New Guinea involves intricate land-ocean-atmosphere interactions, posing substantial challenges for tropical weather and climate simulations. Using over two decades of GPM satellite observations and convection-permitting Weather Research and Forecasting simulations, this study examines the physical mechanisms governing the pronounced offshore propagation of diurnal convection over New Guinea. We identify two distinct convective propagation modes: (a) a "ridge-to-coast" mode originated over elevated terrain and migrating toward the coastline, and (b) an "over-ocean" mode initiated near the coast, separated by a spatial gap of approximately 100 km. Our findings highlight the critical role of multi-scale thermally driven flow in shaping boundary-layer dynamics over warm ocean waters. Specifically, the afternoon sea-breeze front advects cooler air onshore, stabilizing the lower atmosphere and interrupting the continuous propagation of the first mode. At night, the hybrid land breeze (LB), strengthened by cold pools, generates offshore moist patches that facilitate the convective regeneration and propagation of the second mode. These offshore convective systems interact with monsoonal background winds, sustaining precipitation well beyond 200~600 km from the coast. Sensitivity experiments indicate that even a modest increase in sea surface temperature can enhance convective intensity and extend offshore propagation. These results shed light on the mechanisms that enable diurnal offshore convection to persist overnight and propagate far from the coastline, highlighting the importance of moist-boundary-layer density currents and offering insights for improving precipitation forecasts and global model performance over the Maritime Continent.
arXiv:2607.07422v1 Announce Type: new
Abstract: Logical Multi-Hop Query Answering over Knowledge Graphs (KGs) can be formulated as querying, with an implicit completeness assumption. Current works mainly focus on Existential First Order Logic (EFO) queries. These EFO queries contain conjunction, disjunction, and negation operators. Most existing works employ transductive reasoning, meaning they are not capable of reasoning over entities unseen during training. In the real world, there is a resource scarcity, and we cannot train a model with all the nodes of a large KG. Hence, we propose InductWave, a wavelet-based inductive embedding method for logical query answering on large KGs. Here, the training graph consists of fewer nodes than the test graph. Our model performs on par with the baseline models while having half the number of message-passing layers. It outperforms all of them in most cases, with 75% of the layers. These fewer resource requirements enable us to evaluate InductWave on massive graphs, such as Wiki-KG. We test our model using extensive experiments across varying train-test graph proportions of the FB15k-(237) dataset, comparing it with the state-of-the-art models. The code and datasets for the model are available at https://github.com/kracr/inductwave/.
arXiv:2607.06974v1 Announce Type: new
Abstract: Large language models (LLMs) increasingly improve their reasoning at test time via additional computation, yet most existing works treat each problem in isolation. When problems arrive sequentially, accumulating reusable experience across them can further improve performance. Existing memory-based methods either store whole-solution templates that generalize poorly to novel problems or use heuristic step-level selection that is not optimized for final-answer correctness. Learning selection policies requires large-scale training data and fixed action spaces, making such approaches unsuitable for test-time settings where memory expands incrementally and only limited supervision is available. We propose MILES (Modular Instruction Memory with LEarnable Selection for self-improving LLM reasoning), a framework that dynamically expands step-wise memory and applies correctness-optimized memory composition under realistic test-time constraints. MILES maintains modular memory units consisting of asymmetric pairs of sub-goal embeddings and sub-instructions, each associated with a learnable selection head. This memory structure enables a coarse-to-fine retrieval mechanism: The coarse level enables memory expansion and collects supervision for training selection heads from confident samples, while the fine stage applies learned selection heads to rerank coarse-level candidates and guide reasoning for uncertain samples. MILES consistently matches or outperforms prior methods while achieving superior accuracy-efficiency tradeoffs. Extensive experiments demonstrate its effectiveness, robustness, and transferability.
arXiv:2607.06976v1 Announce Type: new
Abstract: In this paper, we propose an efficient hybrid least squares/gradient descent (LSGD) method for MIONets to accelerate training. This method generalizes the LSGD method for DeepONets. Since MIONet is the sum of the entrywise product of multiple branch networks and a trunk network, it can be viewed as a multilinear function with respect to the last layer parameters of each branch network. These sets of parameters can be optimized using the alternating least squares method, where we solve the LS system for a single branch network in turn. To handle the large-sized system matrix, we introduce Kronecker and Khatri-Rao products and tensor permutation matrices to factor the large matrix into small ones. Our method is compatible with a general type of $L^2$ loss with regularization terms for the last layer parameters of each branch, where linear operators can be applied to the MIONet output in each loss term.
arXiv:2607.06989v1 Announce Type: new
Abstract: Table tennis, a dynamic, compact, and popular sport, has received significant attention as a robotics benchmark over the last decades. Most of the research has focused on the rally aspect - returning an incoming ball - requiring high-speed vision, agile motion planning, and tight closed-loop control. However, the other component of table tennis gameplay - the serve - is comparatively a quite unexplored research problem, that in fact requires pushing physics modeling and control to the extremes. Achieving competitive serves with a robot presents domain-specific challenges, such as high-spin generation from a spinless ball, precise aiming, or multi-objective optimization. In this work, we present a novel approach for generating official rule-compliant serves by combining motion primitives, Model Predictive Control, and Bayesian Optimization. Serves generated in this way offer a wide and controllable variation of spins of up to 550 rad/s, and speeds of up to 6.7 m/s, matching and even surpassing those of elite table tennis players.
arXiv:2607.07001v1 Announce Type: new
Abstract: Anticipating human motion from an egocentric perspective is fundamental for proactive assistance in AR/VR, human-robot collaboration, and embodied AI. While recent works incorporate language as a semantic prior to reduce the ill-posed nature of egocentric forecasting, they largely neglect the 3D spatial and semantic context that governs how motion unfolds, and treat pose and language prediction as separate inference streams. We introduce Ego3DLM, built on two core principles: accurate motion forecasting requires explicit spatial and semantic understanding of the 3D environment, and pose and language must be predicted holistically in a single pass, since motion is inherently tied to the semantic interpretation of actions being performed. Given three-point tracking, 3D scene features, and egocentric video, Ego3DLM simultaneously decodes past pose, future pose, past narration, and future narration in a single autoregressive pass, grounding predicted poses and descriptions in one another to enforce cross-modal and temporal consistency. We adopt a three-stage training scheme: (1) spatial-semantic scene awareness pretraining; (2) holistic instruction tuning over all four outputs in a single pass; and (3) GRPO-based reinforcement finetuning with intra- and inter-modal rewards that directly optimize pose-language fidelity. Experiments on the Nymeria benchmark demonstrate that Ego3DLM achieves state-of-the-art performance across future motion prediction, past motion tracking, and motion description, showing that 3D scene grounding and holistic cross-modal prediction yield physically plausible and semantically coherent motion forecasts. The project page is available at https://jaewoo97.github.io/Ego3DLM/.
arXiv:2506.12885v4 Announce Type: replace
Abstract: Crop type classification from optical satellite time series remains limited in its ability to generalize across growing seasons, particularly when crop phenology shifts due to inter-annual weather variability. This hampers deployment in operational settings where current-year labels are unavailable. In addition, uncertainty quantification is often overlooked, reducing the reliability of such approaches for practical crop monitoring. Inspired by ecophysiological principles, we introduce Thermal Time-based Temporal Sampling ($T^3S$), a simple, model-agnostic method that replaces calendar time with thermal time. By re-indexing satellite observations by cumulative growing degree days, $T^3S$ aligns phenologically equivalent growth stages across years, reducing temporal redundancy while concentrating on the most biologically informative periods. We evaluate $T^3S$ across three architecturally distinct backbones on (i) SwissCrop, a new country-scale, multi-year Sentinel-2 dataset with paired temperature data that we publicly release, and (ii) the cross-region TimeMatch benchmark spanning Denmark and France. Across these settings, $T^3S$ consistently improves cross-year and cross-region crop classification over several state-of-the-art baselines, including thermal positional encoding, with particularly strong gains in uncertainty calibration, robustness under label scarcity, and early-season prediction, while requiring no architectural modification.
arXiv:2607.07432v1 Announce Type: new
Abstract: Although one-sided spreading has been shown to improve the near-wall accuracy of diffuse-interface immersed boundary methods (DIBMs), the effect of its asymmetric kernel support on the effective boundary location remains insufficiently understood. In this work, a detailed analysis of the one-sided spreading operator reveals an inward displacement of the effective boundary relative to the geometric boundary. To compensate for this displacement, a target-value reconstruction strategy is developed to ensure consistency between the values imposed at the effective boundary and the prescribed conditions at the geometric boundary. The strategy is incorporated into the fully one-sided diffuse-interface immersed boundary method (FODIBM) and applies to both Dirichlet and Neumann boundary conditions. Although confined to the target-value evaluation step, the modification substantially improves boundary-condition enforcement with negligible additional computational cost. Coupled with a hybrid lattice Boltzmann solver, the improved method consistently reduces L_2 and L_{\infty} error norms across different grid resolutions while retaining approximately second-order grid convergence. The no-slip and isothermal boundary-condition errors are reduced by 77% and 85%, respectively. Simulations involving various two- and three-dimensional geometries further show improved predictions relative to both the conventional DIBM and the original FODIBM. The results agree well with body-fitted reference solutions and experimental data, demonstrating accurate and computationally efficient simulations of compressible flows around complex geometries.
arXiv:2607.07051v1 Announce Type: new
Abstract: Conversational image editing requires preserving not only visible content, but also content that temporarily disappears across turns. When newly added or modified content occludes a previously visible region, that region should reappear if it was never semantically changed. However, existing systems often fail to recover such occluded-but-unchanged content, producing inconsistent or hallucinated results. We introduce OCCUR-Bench, a diagnostic benchmark for temporal preservation in conversational image editing. OCCUR-Bench provides diverse occlusion-and-revelation scenarios with historical restoration references, enabling evaluation of faithful restoration rather than plausible regeneration. We also propose ReSpec, a training-free framework that makes implicit preservation explicit by pairing restoration-aware instructions with historical visual references. Given an editing history, ReSpec identifies what should persist, selects the historical image state that provides missing visual evidence, and conditions an in-context editor on the resulting instruction and reference image. Experiments show that ReSpec improves restoration fidelity and temporal consistency on OCCUR-Bench, highlighting the need to ground preservation in editing history rather than only the current image.
arXiv:2607.07083v1 Announce Type: new
Abstract: Subsampling significantly reduces the number of measurements, thereby streamlining data processing and transfer overhead, and shortening acquisition time across diverse real-world applications. The recently introduced Active Deep Probabilistic Subsampling (A-DPS) approach jointly optimizes both the subsampling pattern and the downstream task model, enabling instance- and subject-specific sampling trajectories and effective adaptation to new data at inference time. However, this approach does not fully leverage valuable dataset priors and relies on top-1 sampling, which can impede the optimization process. Herein, we enhance A-DPS by integrating a deterministic (fixed) prior-informed sampling pattern derived from the training dataset, along with group-based sampling via top-k sampling, to achieve more robust optimization, method we call Prior-aware and context-guided Group-based Active DPS (PGA-DPS). We also provide a theoretical analysis supporting improved optimization via group sampling, and validate this with empirical results. We evaluated PGA-DPS on three tasks: classification, image reconstruction, and segmentation, using the MNIST, CIFAR-10, fastMRI knee, and hyperspectral AeroRIT datasets, respectively. In every case, PGA-DPS outperformed A-DPS, DPS, and all other sampling methods.
arXiv:2607.07141v1 Announce Type: new
Abstract: Newly developed items must ordinarily be field tested before their psychometric properties are known, creating a cold start problem for item calibration. Predicting item parameters from features is a long standing measurement problem dating back to the Linear Logistic Test Model; modern text embeddings now automate the design matrices traditionally specified by hand. We propose an evaluation framework combining regularized regression on item text embeddings, repeated cross validated R squared reported with its resampling standard deviation, and two performance upper bounds: a reliability ceiling derived from parameter standard errors, and a design ceiling derived from simulation based power calibration. Applying this framework to a mathematics item bank (EEDI) and a medical licensure benchmark (BEA 2024), we find that item difficulty is highly predictable from text (repeated cross validated R squared = 0.53, or about 57% of its reliability ceiling), whereas discrimination and pseudo guessing appear less predictable. However, evaluating these results against our ceilings reveals that this apparent hierarchy stems from target reliability rather than text signal strength: text uniformly recovers 57 to 63% of the reliable variance across difficulty targets, whereas the 3PL pseudo guessing parameter has a reliability ceiling near zero, making it an unviable target at current precision. On BEA, embedding based regression matches leaderboard RMSE despite explaining almost no variance, highlighting the critical need for scale free metrics and explicit ceilings in benchmarking. Finally, we show that a single train and test split can inflate apparent accuracy by 0.1 to 0.15 in R squared, underscoring the necessity of repeated cross validation for calibration support applications and future benchmark construction.
arXiv:2607.06638v1 Announce Type: new
Abstract: Self-paced learning (SPL) is an effective learning paradigm that simulates the human learning process by progressing from easy to difficult samples based on the value of the loss function during the learning process. It has shown great potential in improving model performance and training efficiency. However, the prediction results of samples with smaller loss values are not necessarily reliable, indicating that such samples are not always simple samples for the model. Hence, this article proposes an uncertainty-aware self-paced learning based on evidential neural networks, termed UASPL, which integrates predictive reliability into sample selection through a general loss function within the Subjective Logic framework. This loss function incorporates uncertainty estimation and can be extended to different variants of SPL. Moreover, this loss function couples a sample selection preference, thereby ensuring the interpretability of the sample selection process. Finally, the experimental results on multiple datasets show that UASPL outperforms other SPL methods in terms of classification performance, interpretability, and generality. The source code is available at: https://github.com/treelife979/UASPL.
arXiv:2607.06699v1 Announce Type: new
Abstract: Recovering real-world scenes as interactive simulation environments can enable generalizable robot learning and reproducible policy evaluation. However, constructing scenes that are both physically stable and visually faithful remains slow and expensive. In this work, we present RoboSnap, a real-to-sim framework that turns a single RGB image into a simulation-ready scene. The key idea is a layered design that separates the physics-critical interaction area from the surrounding visual context: collision-aware foreground assets are refined for stable robot interaction, while a 3D Gaussian splatting visual layer preserves faithful background appearance under novel views. Experiments on DROID scenes and real-robot tasks show that RoboSnap achieves reliable trajectory replay in the recovered scenes, supports task-specific synthetic data generation for policy training, and yields meaningful sim-real correlation for policy evaluation. To further support real-to-sim research, we introduce DROID-Sim, a real-to-sim companion dataset constructed from 564 real-world scenes in DROID. Extensive experiments suggest that the value of real-to-sim methods lies not only in high-fidelity visual reconstruction, but in turning real environments into reusable infrastructure for robot learning and evaluation.
arXiv:2607.06954v1 Announce Type: new
Abstract: Wearable and mobile sensing technologies have demonstrated strong potential for health inference; however, most sensor models are designed for specific disease types, limiting their transferability across different health risks. Wearable foundation models offer a more generalizable approach in diverse health risk types. Nevertheless, most existing methods rely on high-frequency raw sensor data, raising concerns about privacy, computational overhead, and scalability across devices and populations. In this paper, we propose StepFM, a foundation model built solely on step counter data for broad-spectrum health prediction. Leveraging the ubiquity and low-dimensional nature of step data, StepFM provides a practical, privacy-preserving, and computation-efficient alternative to traditional sensor-based models. We design a scalable pre-training framework that captures temporal dynamics and behavioral patterns from large-scale step sequences, enabling transfer across more than 20 health risk prediction tasks spanning diverse devices, new regions, and novel disease types. Extensive experiments demonstrate that StepFM achieves strong performance compared to existing methods while maintaining robustness across heterogeneous settings. Furthermore, our analysis reveals interpretable and generalizable relationships between physical activity patterns and various health risks, offering new insights into activity-based health modeling. Our work establishes step-based sensing as a viable foundation for scalable and real-world health monitoring.
arXiv:2607.06956v1 Announce Type: new
Abstract: We introduce an activation hierarchy consisting of stage machines, stage domains, and stage languages generated by an activation operator. The central result is a Predictor-Impossibility Theorem (PIT), which shows that no effective predictor family can uniformly determine all stage languages of the hierarchy. The proof combines the semantic activation construction with the S-m-n Theorem and Kleene's Recursion Theorem to obtain a self-referential fixed point that yields a contradiction. We then define an aggregate language MIS and establish a slice theorem connecting aggregate inputs to individual stage languages. This provides a bridge from polynomial-time decidability of MIS to the existence of a predictor family. By PIT, the aggregate language is, therefore, not polynomial-time decidable. Under the aggregate growth condition defining valid aggregate objects, MIS is shown to belong to NP. Combining these two results yields MIS in NP-P. The paper is organized so that PIT stands independently as a recursion-theoretic result, while the complexity-theoretic consequences are derived from the aggregate-language framework.
arXiv:2607.07194v1 Announce Type: new
Abstract: In this Letter, we show how a suspension of light solid spheres settling through a polymer solution results in a chaotic and highly-intermittent state. By leveraging particle-resolved direct numerical simulations, we investigate the effect of increasing polymer relaxation time and Deborah number $De$ on viscoelastic sedimentation at a low density ratio $\rho_s/\rho_f=5$ and a low Galilei number $Ga=3.16$. Even at moderate $De$, the spheres form gravity-aligned clusters and settle faster, while the polymer stresses energize the large scales of motion. The onset of elastic turbulence and intermittency is signaled by a $-4$ spectral scaling in the high-wavenumber range and by nonlinear high-order exponents of the velocity structure functions. These results indicate that viscoelastic effects induce pseudoturbulence in the presence of viscous-dominated sedimentation.
arXiv:2607.06973v1 Announce Type: new
Abstract: We introduce a new context-enriched, multimodal time series forecasting benchmark, TimesX. TimesX contains a wide selection of high-quality real-world time series with diverse domains and textual contexts obtained from an automated data generation pipeline, which helps address three main issues of existing multimodal forecasting benchmarks: (1) poor generalization due to the small scale and synthetic nature of benchmark data, (2) very limited types of textual contexts in the benchmarks, and (3) an inability to mitigate data leakage in evaluation. We conduct a thorough empirical study of zero-shot multimodal forecasting approaches on TimesX. Our results suggest that many approaches that perform well on existing benchmarks may fail on TimesX. In contrast, simple ensemble methods that leverage rich textual context accompanying time-series can outperform strong baselines on TimesX.
arXiv:2607.07228v1 Announce Type: new
Abstract: We predict a class of bound states in the continuum (BICs) in non-subwavelength periodic metasurfaces -- Rayleigh BICs -- that emerge precisely at Rayleigh anomalies, where diffraction channels open. In contrast to conventional symmetry-protected and accidental BICs, which are typically engineered in the subwavelength regime with only a few radiation channels, Rayleigh BICs remain nonradiating far beyond this limit, even when multiple diffraction channels are open. Their formation originates from the interplay between collective lattice resonances, Rayleigh anomalies, and anapole states of the constituent metaatoms. We show that Rayleigh BICs are characterized by an unusual cubic scaling of the quality factor in reciprocal space, ($Q \propto 1/k^3$), an asymptotic behavior that is not characteristic of either symmetry-protected or accidental BICs. Our results establish a constructive route for engineering nonradiating states in metasurfaces beyond the subwavelength regime.
arXiv:2607.07507v1 Announce Type: new
Abstract: Hallucinations in vision language models (VLMs) are commonly treated as semantic errors, yet they often arise from partial or ambiguous visual evidence. Prior work mainly focuses on detecting or suppressing hallucinations at generation time, leaving the subsequent reasoning stage largely unexplored. In this work, we study Post Hallucination Reasoning (PHR), the stage in which hallucinated semantics enter the model's inference context and influence downstream predictions. To systematically investigate PHR, we introduce HIVE, Hallucination Inference and Verification Engine, an evaluation infrastructure that enables controlled comparisons between faithful and hallucinated captions. Across nine tasks and nine models, we observe structured modality dependent patterns: hallucinated captions often improve accuracy on vision language tasks, while text only tasks exhibit limited or unstable effects. Further analyses show that hallucinated cues broaden semantic coverage and reshape reasoning dynamics while preserving stable inference. These findings highlight that hallucinated semantics may influence downstream reasoning once they enter the model's inference context. Understanding this post hallucination stage is important for improving the reliability and interpretability of multimodal reasoning systems. Code is publicly available at https://github.com/hefengcs/HIVE.
arXiv:2506.18579v3 Announce Type: replace
Abstract: High-frequency (HF) ray tracing in complex ionospheric media generally involves a fundamental trade-off between path accuracy and computational efficiency, which directly affects practical applications such as over-the-horizon radar, ionospheric monitoring, and HF skywave communication systems. This paper presents RTM-GD, a ray-tracing framework that combines Hamiltonian ray integration with a continuously differentiable Galerkin--Difference (GD) interpolation strategy. Under discretized ionospheric grid conditions, the electron density is reconstructed as a $C^1$-continuous function within each grid cell, yielding an everywhere differentiable electron-density field for stable numerical integration and improved propagation-path accuracy. Numerical simulations and validations using measured HF oblique sounding data are conducted under diverse conditions, including different ionospheric states, low- and high-elevation angles, multiple operating frequencies, and both ordinary (O) and extraordinary (X) wave modes. Results show that RTM-GD consistently achieves sub-kilometer RMSE in both group-path and ground-distance metrics and sub-0.01-degree azimuth deviation relative to Richardson extrapolation, while reducing computational time by 98\%. Compared with Catmull--Rom interpolation, RTM-GD reduces the RMSEs of ray parameters by approximately one order of magnitude with less than 4\% additional computational cost. Measured-data validation based on ionogram synthesis further shows that the mean relative group-path error remains within 7\%, confirming reliable reproduction of practical HF oblique propagation characteristics. Overall, RTM-GD provides an accurate and computationally efficient framework for HF ray tracing in discretized ionospheric environments.
arXiv:2607.07386v1 Announce Type: new
Abstract: Linear attention models allow a fixed state size and a fixed amount of compute per token. However, due to their limited state size, linear attention models fall behind in long-context recall compared to softmax-attention-based transformer architectures. Increasing the state size of linear attention improves recall performance but at the cost of higher FLOPs. In this work, we introduce Sparse Delta Memory (SDM), an architecture that scales the hidden state of gated linear RNNs to orders of magnitude higher capacity using a sparse addressing scheme. SDM extends the Gated DeltaNet architecture by replacing the dense key-value outer product with sparse reads and writes to a large explicit memory. We show that, under an isoFLOP constraint and with an identical number of parameters, a higher state memory capacity significantly improves performance on in-context learning and long-context retrieval tasks. Moreover, by learning the initial state of the SDM memory and therefore using it as a parametric memory, we show that the model further improves on a wide range of common-knowledge and reasoning tasks.
arXiv:2607.07409v1 Announce Type: new
Abstract: Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel. Block-parallel drafters such as DFlash further improve drafting efficiency by predicting an entire block in one pass, but their position-wise predictions lack explicit intra-block causal conditioning. Recent methods such as Domino and DSpark attempt to introduce such causality into block-parallel drafting, but they require training the draft model from scratch, which limits their flexibility and increases training cost. We propose DeLS-Spec, a decoupled long-short context speculative decoding method. DeLS-Spec treats the fixed DFlash model as a long-context expert and introduces a lightweight local head as a short-context expert. The local head can be trained independently with a standard next-token prediction objective, without joint training with the target model or the DFlash backbone, leading to extremely low training cost. At inference time, DeLS-Spec combines long-context and short-context logits, and the local head is not tied to a specific DFlash checkpoint, making the method more modular and flexible. Experiments on Qwen3 models show that DeLS-Spec consistently improves speedup and average acceptance length over DFlash across math, code, and dialogue benchmarks.