arXiv:2511.18735v3 Announce Type: replace Abstract: In this work, we define Foresight Intelligence as the capability to anticipate and interpret future events-an ability essential for applications such as autonomous driving, yet largely overlooked by existing research. To bridge this gap, we introduce FSU-QA, a new Visual Question-Answering (VQA) dataset specifically designed to elicit and evaluate Foresight Intelligence. Using FSU-QA, we conduct the first comprehensive study of state-of-the-art Vision-Language Models (VLMs) under foresight-oriented tasks, revealing that current models still struggle to reason about future situations. Beyond serving as a benchmark, FSU-QA also enables the assessment of world models by measuring the semantic coherence of their generated predictions, quantified through performance gains when VLMs are augmented with such outputs. Our experiments further demonstrate that FSU-QA can effectively enhance foresight reasoning: even small VLMs fine-tuned on FSU-QA surpass much larger, advanced models by a substantial margin. Together, these findings position FSU-QA as a principled foundation for developing next-generation models capable of truly anticipating and understanding future events. Furthermore, beyond model performance, we examine whether WM-generated predictions remain semantically consistent by using VLM-based proxy judges, and validate this evaluation protocol through shuffled control experiments. Fine-tuning models on FSU-QA leads to substantial improvements in foresight understanding, demonstrating the dataset's effectiveness and offering a principled foundation for future research.
Science Journals
arXiv:2512.02452v2 Announce Type: replace Abstract: As is well known, classical PID control is ubiquitous in industrial processes, yet a rigorous and explicit design theory for nonlinear uncertain MIMO second-order systems remains underdeveloped. In this paper we consider a class of such systems with both uncertain dynamics and an unknown but strictly positive input gain, where the nonlinear uncertainty is characterized by bounds on the Jacobian with respect to the state variables. We explicitly construct a three-dimensional region for the PID gains that is sufficient to guarantee global stability and asymptotic tracking of constant references for all nonlinearities satisfying these Jacobian bounds. We then derive a corresponding necessary region, thereby revealing the inherent conservatism required to cope with worst-case uncertainties. Moreover, under additional structural assumptions on the nonlinearities, these sufficient and necessary regions coincide, yielding a precise necessary-and-sufficient characterization of all globally stabilizing PID gains. All these regions are given in closed form and depend only on the prescribed Jacobian bounds and the known lower bound of the input gain, in contrast to many qualitative tuning methods in the literature.
arXiv:2504.19655v3 Announce Type: replace-cross Abstract: We present a framework for mechanochemical self-organization in active solids where elasticity is reciprocally coupled to Hopf oscillators. Our model reveals a rich landscape of symmetry-protected phases, identified through amplitude equations and group-theoretic analysis. We uncover a universal transition to compression-driven oscillation death (COD), providing a physical basis for localized signaling dampening in biological tissues that resolves inconsistencies in previous models. Our work demonstrates that complex self-organization in active solids can be classified purely through symmetry arguments.
arXiv:2607.06711v1 Announce Type: cross Abstract: Twisted bilayer graphene reconstructs from weak breathing corrugation to large common bending near the magic angle, but the origin of this collective crossover has remained unclear. Here we show that the crossover is a soft-mode condensation of layer-symmetric $A_1$ moir\'e flexural phonons: these modes soften on the breathing branch, lose stiffness near the magic angle, and freeze into the bending morphology. We call this mechanism Moir\'e Phonon Condensation (MPC). At $\theta=1.08^\circ$, it is extremely surprising that displacements of all 11164 atoms in the moir\'e supercell, with a maximum atomic position shift of 2.30 Angstrom, is captured by only two $A_1$ phonon modes at more than $99.5\%$ spectral weight. A first-harmonic continuum theory identifies a dimensionless control parameter of the phenomenon, showing that as the twist approaches the magic angle, the growing moir\'e length scale amplifies a smooth stress-bending competition until the flexural stiffness changes sign. Mode-resolved tight-binding calculations further show that the condensed phonon coordinates are electronically active. This work identifies MPC as a twist-controlled structural order parameter for moir\'e reconstruction.
arXiv:2607.06729v1 Announce Type: cross Abstract: Density functional theory (DFT) often is the method of choice for simulating the electronic properties of extended solids and surfaces from first principles due to a favorable compromise between accuracy and computational cost. In the field of heterogeneous catalysis, DFT is indispensable for deriving mechanistic insights and understanding trends in surface reactivity. The accuracy of DFT for surface reaction energetics depends strongly on the exchange-correlation (XC) approximation. We show here that optimization of such XC functionals for surface binding energies can lead to a worse description of surface reaction barriers, unless important physical exact conditions are fulfilled.
arXiv:2607.06827v1 Announce Type: cross Abstract: Speech large language models (Speech LLMs) typically encode speech into sequences far longer than text, creating a major efficiency bottleneck during autoregressive decoding. A common remedy is to compress the speech sequence at the adapter level to remove temporal redundancy before it enters the LLM; however, such early downsampling risks discarding fine-grained information that cannot be recovered. We propose SpeechKV, which applies a learned pooling to the KV cache of speech tokens inside the LLM. This design allows the LLM to fuse speech and text internally while directly accelerating decoding. Trained on 71K hours of speech data, SpeechKV compresses the speech to approximately text-level granularity yet maintains performance on par with or even slightly better than the uncompressed baseline, with relative gains of 6.6% on out-of-domain entity recognition and 2.3% on OpenASR, while delivering at least 1.49 times decoding speedup that scales with audio length.
arXiv:2607.06893v1 Announce Type: new Abstract: The Stochastic-Oracle Turing Machine (SOTM) framework models AI-augmented computation as the interaction of a probabilistic Turing machine with an oracle whose responses are drawn from context-dependent distributions. This paper studies what an SOTM can achieve under two oracle-response schemes: in a cached-response oracle, each distinct query receives one response that is reused on later calls to the same query, while in a fresh-response oracle, each call returns an independent response. In both schemes, the SOTM first computes from its input and internal random source to generate its first query, then proceeds adaptively, computing from its query-response transcript (the record of queries issued and responses received) to generate each subsequent query or produce a final output. Cached responses impose two transcript-based ceilings on achievable performance: a correct-identification ceiling governed by the total variation distance between the transcript distributions induced by the hidden states of the oracle, and an output quality ceiling equal to the expected score of the best output the SOTM can compute from the transcript. Fresh responses can raise these ceilings by allowing repeated calls to accumulate independent evidence toward correct or high-quality outputs. In the binary single-informative-query case, the error probability decreases exponentially in the number of calls to the same query at the Chernoff rate. For output quality, query-count bounds characterize threshold stopping when the score function is incorporated as part of the SOTM, and majority-based amplification bounds characterize the binary candidate-output model when it is not. Together, the results identify how response reuse, transcript information, and access to the score function determine what an SOTM can compute and at what token cost.
arXiv:2607.06644v1 Announce Type: cross Abstract: Determinantal point processes have recently emerged as a kernel-based alternative to standard independent sampling for constructing efficient minibatches, coresets, and other compact representations of large-scale datasets. In particular, sampling mechanisms based on DPPs are believed to demonstrate better approximation properties compared to classical i.i.d. samplers, even at the scale of the exponent. One of the key strengths of DPP based samplers is that they can be deployed over very general spaces, in contrast to more classical sampling methods beyond i.i.d. which tend to work in very well-structured settings, principally Euclidean spaces. In this work, we establish explicit rate guarantees for determinantal sampling in spaces that extend far beyond known Euclidean setups, focusing on spectral kernels obtained from eigenspaces of naturally associated Laplacian and other Markov diffusion operators. This includes, in particular, Riemannian manifolds and weighted networks. In determinantal sampling from compact Riemannian manifolds, we establish sampling rates that automatically pick up the intrinsic dimensionality $d_{\text{int}}$ of the underlying manifold. In the setting of networks, we investigate DPP-based samplers on the celebrated k-nearest neighbour graphs, as well as weighted random geometric graphs, and demonstrate a similar improved dependence on the intrinsic dimensionality of the data. Overall, our approach achieves guarantees of $\big(\text{sample size}\big)^{-\frac{1}{2}-\frac{1}{2d_{\text{int}}}}$ that match known rates on Euclidean spaces of comparable dimension. In terms of techniques, we connect to the celebrated Weyl's Law for manifold spectra, and leverage tools from the theory of Markov diffusions and Dirichlet forms as well as certain ingredients from the theory of pseudodifferential operators, which could be of independent interest in this area.
arXiv:2607.07475v1 Announce Type: new Abstract: In robotics, the capability of an artificial agent to represent the range of its action possibilities, i.e. affordances, is crucial to understand how it can act on its environment. While functional affordances, which refer to the use of tools and objects, have been broadly studied in knowledge representation, the implications of a social context and the presence of other agents have remained unexplored in this field. Consequently, in the field of social robotics, a multi-agent context enables the agents to engage in new actions that are potentially complementary to their individual capabilities, leading to the perspective of agentexploitation. This work focuses on the concept of cooperative affordance within the realm of social affordances. Cooperative affordances refer to situations where agents interact with each other to extend their action possibilities range. From this definition, this paper proposes a tractable ontological representation of this concept with the aim of making it usable by an artificial agent. Expanding on those elementary patterns, we illustrate the effectiveness of these representations by combining them to depict a diverse range of scenarios.
arXiv:2607.06607v1 Announce Type: new Abstract: Accurate long-term forecasting in complex systems is frequently compromised by dataset-level distribution shifts, where diverse underlying behavioral modes and evolving system states drive the dynamic multivariate time-series. While existing methods predominantly focus on local temporal shifts, they fail to explicitly model the global structural challenge where datasets are composites of distinct operational regimes. In this paper, we propose NEST, a specialized framework designed to model and recompose these evolving structures through a two-phase dense MoE architecture. NEST first facilitates structural specialization by partitioning the dataset into distinct operational regimes through unsupervised clustering in a principled moment-entropy space. We introduce a regime-oriented router mechanism that generates initial expert weights based on temporal content, subsequently refined through geometric modulation to regime centroids. Crucially, rather than acting as monolithic predictors, individual experts function as specialized kernels that capture regime-specific dynamics by evolving unique variate-attention patterns. Extensive evaluations on diverse benchmarks, including heterogeneous network traffic and physical phenomena, demonstrate that NEST consistently achieves state-of-the-art performance. Our code and datasets are available at https://github.com/Aaralshin/NEST
arXiv:2607.07343v1 Announce Type: new Abstract: Online truckload bid acceptance is a closed-loop stochastic decision problem in which a carrier or broker must, in real time, accept or reject a tendered load subject to operational feasibility, fleet repositioning costs, and opportunity cost against future demand. Public, reproducible benchmarks for this problem are scarce: existing routing benchmarks are static, while dynamic-fleet studies typically rely on private operator data. We introduce FreightBidBench, a public-calibrated, dependency-free, closed-loop benchmark in which feasibility (pickup reach, appointment windows, simplified hours-of-service, stochastic yard delays) and economics (service-failure penalty, terminal fleet value, daily price-premium window) are explicit, versioned, and reproducible from public Freight Analysis Framework and U.S. Department of Agriculture truck rate data. We develop two full-horizon hindsight ceilings: a simple LP style relaxation and a tighter Lagrangian-per-truck information relaxation that retains per-truck hours-of-service and sequencing structure and is 20.7% tighter than the LP relaxation on a tight-capacity scenario and 39.3% tighter on a scarce-capacity scenario. We introduce a parametric surrogate-rollout cascade with boundary-band and scarcity-pressure escalation triggers. On ten-seed tight and scarce scenarios, the best simple policy retains 91.0% and 86.5% of rollout profit and the standard-library surrogate 94.2% and 89.3%; a cascade at a single escalation band recovers roughly 98% on both at 40-56% of rollout's mean decision latency, and on the tight scenario is statistically indistinguishable from the rollout teacher (paired-bootstrap 95% CI on the profit delta spans zero).
arXiv:2607.07004v1 Announce Type: cross Abstract: Magnetohydrodynamical (MHD) simulations of isothermal compressible turbulence report that the density distribution is well described by a lognormal with a variance proportional to the flow's Mach number. The distribution of magnetic field strength also has a lognormal component, but includes long, power-law-like tails. In this work, we use semi-analytical arguments to predict the distributions of density and magnetic field strength in compressible turbulent flows. Specifically, in the Lagrangian description of the continuity and the induction equations, we model the velocity gradients of the turbulent flow as a simple random process, essentially turning these equations into stochastic differential equations. Integrating them leads to a lognormal distribution for the density field and the strength of the magnetic field. The power-law tails in the magnetic field PDF appear when we introduce intermittent shocks due to sampling rare events. Gradually increasing the frequency of these events, essentially going closer to a continuous process, leads to lognormal-like distributions again. The asymmetry is connected to the relative abundance of slow and fast shocks. An overabundance of fast MHD shocks produces a high-value tail, while the contrary produces low-value tails. We propose that the appearance of power-law tails along lognormals in turbulent flows is the signature of the co-existence of continuous, diffusion-like propagation combined with localized, intermittent events.
arXiv:2607.07527v1 Announce Type: cross Abstract: Artificial intelligence (AI) is a double-edged sword: while it has achieved remarkable success across a wide range of domains, its deployment also calls for effective oversight and regulation, for which the detection of AI-related content and artifacts is perhaps the most direct and cost-effective approach. To this end, we propose a unified detection framework based on Mahalanobis distance scores (MDS), applicable to several important settings, including the detection of large language model (LLM) generated text, hallucination, watermark, and adversarial examples. A key component of the proposed method is to accurately characterize the positive class--such as human-generated text, factual statements, unwatermarked text, or non-adversarial samples--which requires an efficient and robust estimator of the covariance matrix of deep representations of positive samples before computing the MDS. Since the positive samples typically consist of multiple classes, and these classes may exhibit both homogeneity and heterogeneity, we develop joint estimation methods for both the casewise and cellwise minimum covariance determinant (MCD) estimators. We provide efficient optimization algorithms for both estimators and prove their convergence. We provide a reasonable definition of the breakdown point for the joint estimators and prove their corresponding high breakdown point properties. Empirical evaluations confirm the effectiveness of the proposed detection framework.
arXiv:2607.07634v1 Announce Type: cross Abstract: Time series analysis plays a vital role across a wide range of scientific and engineering domains but poses substantial computational challenges. A major difficulty arises from the time reparameterization invariance of time series data, which complicates the extraction of meaningful temporal features. In this work, we address the problem of time series classification by exploring the application of quantum computation techniques. We propose a hybrid quantum-classical architecture that integrates recent advances in quantum neural networks with the mathematical framework of path signatures, mitigating the impact of time reparametrization invariance. The architecture employs feature layers that compute a signature kernel between pairs of input paths, consisting of a reference path and a target path for classification, using either classical or quantum variational linear solvers (VQLS). These feature layers are followed by a Quantum Convolutional Neural Network (QCNN) to perform downstream learning tasks. We evaluate several realizations of the proposed architecture, differing in QCNN configurations, on a binary classification task involving time series representations of handwritten digits. Our experiments demonstrate the potential advantages of implementing path signature kernel layers within quantum circuits and provide an analysis of the computational limitations associated with the VQLS component.
arXiv:2605.15207v2 Announce Type: replace Abstract: Multi-agent LLM systems have shown promise for complex reasoning, yet recent evaluations reveal they often underperform single-model baselines. We identify a structural failure mode in sequential fine-tuning of shared-context teams: updating one agent shifts the team's context distribution, and when subsequent updates are evaluated on cached rollouts, this mismatch compounds. We formalize this as the compounding occupancy shift and prove that stale-occupancy evaluation incurs a penalty that scales quadratically with the number of agents. In contrast, intermediate-occupancy evaluation reduces this to linear scaling. We propose TeamTR, a trust-region framework that resamples trajectories after each component update and enforces per-agent divergence control, yielding rigorous per-update and per-stage improvement lower bounds. Experiments show that TeamTR outperforms single-agent and sequential baselines with 7.1% on average, mitigates coordination regressions, and supports plug-and-play component replacement. Code is available at https://github.com/Yydc/TeamTR.
arXiv:2606.08068v2 Announce Type: replace Abstract: Multi-agent large language model (LLM) systems often fail to reliably outperform a single strong model equipped with best-of-N sampling. We argue that a core source of this instability is ill-posed equilibrium selection: current systems specify what information agents share, but not which coordination convention should be selected. We formalize a broad class of such systems as discounted incomplete-information Markov games and show that two common pathologies, oscillation between competing conventions and drift across them, can both induce unstable learning and linear Bayesian regret. To obtain a well-posed target, we introduce the Heterogeneous Quantal Response Equilibrium (HQRE), an entropy-regularized equilibrium concept with agent- and state-dependent temperatures. Under a monotonicity condition, HQRE is unique, admits linearly convergent mirror updates, and yields bounded Bayesian regret; the same condition yields rollout-measurable stability diagnostics. We instantiate this objective in two algorithms: DICE-PC, which coordinates frozen models through prompt-control actions, and DICE-FT, which performs parameter-efficient mirror fine-tuning. Across eleven benchmarks in four domains, DICE improves accuracy-cost trade-offs over strong within-class baselines; on reasoning and planning tasks, DICE-PC improves by 4.3 percentage points on average and DICE-FT by 8.5 points.
arXiv:2603.11756v2 Announce Type: replace Abstract: Deep generative models for anomaly detection in multivariate time-series are typically trained by maximizing observed data likelihood. However, likelihood in observation space measures marginal density rather than conformity to structured temporal dynamics, and therefore can assign high probability to anomalous or out-of-distribution samples. We address this structural limitation by relocating the notion of anomaly to a prescribed latent space. We introduce explicit inductive biases in conditional normalizing flows, modeling time-series observations within a discrete-time state-space framework that constrains latent representations to evolve according to prescribed temporal dynamics. Under this formulation, expected behavior corresponds to compliance with a specified distribution over latent trajectories, while anomalies are defined as violations of these dynamics. Anomaly detection is consequently reformulated as a statistically grounded compliance test, such that observations are mapped to latent space and evaluated via goodness-of-fit tests against the prescribed latent evolution. This yields a principled decision rule that remains effective even in regions of high observation likelihood. Experiments on synthetic and real-world time-series demonstrate reliable detection of anomalies in frequency, amplitude, and observation noise, while providing interpretable diagnostics of model compliance.
arXiv:2603.12558v2 Announce Type: replace Abstract: Involution -- a phenomenon of excessive competition with diminishing returns -- has become a pressing socio-economic concern in contemporary China, prompting both academic inquiry and policy interventions. This paper proposes an evolutionary game model of involution that incorporates agent migration and spatial heterogeneity in resource distribution. The model captures realistic features such as effort-based resource allocation, local interactions on a lattice, and mobility driven by payoff comparisons. We explore how varying conditions of migration and resource allocation influence the dynamics of involution. The key findings from our simulations are as follows: when total resources are held constant, similar resource levels across different regions tend to suppress involution, whereas a large disparity between regions promotes it. Furthermore, increasing the total amount of resources exacerbates involution. In addition, the probability of migration does not significantly affect the final evolutionary outcome. We further identify threshold effects in the effort ratio and utility multiplier, revealing critical conditions under which involution emerges or subsides. To further elucidate these simulation results, we conduct a theoretical analysis using mean-field theory, which provides analytical expressions for the equilibria and stability conditions. The theoretical predictions are in excellent qualitative agreement with simulation outcomes. Finally, we discuss real-world counterparts of the model, including competition among food delivery riders and between stores offering similar services.
arXiv:2607.07429v1 Announce Type: new Abstract: In this work, we reviewed different approaches in mathematical modeling of biologically plausible neural systems. Models are characterized and classified based on their common features and special use cases. In addition to spiking models, different types of discrete and continuous analogs are considered to accurately simulate biological processes, including membrane potential dynamics. The models under investigation include neurons and various components encountered in neural systems and affected the dynamics. The selection of specific approaches was driven by their prevalence and innovative perspectives in order to enhance the relevance of the presented information.
arXiv:2607.07152v1 Announce Type: cross Abstract: Computationally efficient models for multimodal traffic flows with inter-modal interactions are foundational for coordinated traffic management. However, such models are lacking in the current literature. This study introduces a multimodal link transmission model (M-LTM) accommodating both continuous road traffic and discrete tramway traffic at the network level. M-LTM builds on a link model that captures the inter-modal interactions through the moving bottleneck theory. We further develop node models describing the flow transfer, tram operations, and mode interactions at tram stops and intersections. Specifically, the tram dwell process and dwell-induced congestion of road traffic are simulated at the stop node, and the intersection node model can reproduce queue spillback and tram priority. Case studies are conducted on two synthetic one-node networks, a synthetic arterial network, and the real network of Dresden to demonstrate the predictive power and the applicability of the proposed model in network traffic analysis and management. The GEH statistics of road traffic are below 4, and the average tram arrival time absolute errors are less than 10 seconds.
arXiv:2607.07435v1 Announce Type: new Abstract: Agents acting on our behalf in the real world (e.g. placing phone calls) must learn online from costly, often irreversible interactions rather than cheap simulator steps. Two things follow. First, deployability depends on the path, not only the outcome. An agent must respect outcome-neutral constraints such as not repeatedly calling an unresponsive user, respecting business hours, or completing required authentication constraints that outcome-based rewards cannot express, since violating them frequently improves apparent success. Second, because each interaction is expensive, the agent must learn efficiently from very few examples. Reinforcement learning from verifiable rewards (RLVR) is blind to both challenges: it optimizes solely on the outcome and wastes expensive rollouts on all-fail groups where group-relative advantage collapses to zero. Attempts to densify supervision by rewarding progress target the hard-to-verify direction. In contrast, real agentic environments can cheaply detect bad moves. Since group-relative advantage is equivalent to within-group variance, a dense signal helps only when it supplies variance the outcome lacks. A verifiable penalty on the path meets this condition reliably, while a progress potential helps only where partial progress is reachable. The resulting recipe "penalize the path, reward the outcome" achieves high task success with near-zero violations, where outcome-only training violates constraints on nearly every episode. We provide four design rules for effective penalties, including avoidance of the inaction trap that arises when a penalty is used in isolation.
arXiv:2607.07561v1 Announce Type: new Abstract: DNA loop formation is a key mechanism in gene regulation, and looping kinetics are sensitive to mechanical tension acting on the DNA. In both single-molecule experiments and biological settings, this tension is typically transmitted through DNA segments flanking the looping region, rather than acting directly at the looping sites. How this indirect force transmission affects the looping time has not been systematically investigated. Using molecular dynamics simulations of a wormlike chain, we show that such flanking segments significantly steepen the force dependence of the looping time, an effect that is insensitive to their length once it exceeds the persistence length, and vanishes when the junction to the looping region is made flexible. We develop an analytical framework that accounts for this effect through a force-dependent shift in the effective free energy landscape of the looping segment. In the limit of small forces, this shift reduces to a zero-force equilibrium average, after which the entire force dependence of the looping time follows analytically. Applying this framework using a coarse-grained DNA model that treats individual bases as rigid bodies, we obtain predictions in quantitative agreement with experimental looping data. Our results demonstrate that the geometry of force transmission has a significant and predictable effect on looping kinetics, with direct implications for the interpretation of tension-dependent looping in both single-molecule experiments and gene regulatory contexts.
arXiv:2606.19042v2 Announce Type: replace Abstract: In vibe coding, an emerging AI-driven paradigm, an LLM generates an entire program from a natural language prompt, but what happens to the variability that traditional software engineering carefully builds into code? To answer this question, we conducted an exploratory analysis on 10 vibe coded C/C++ projects, which suggests that there is near zero in-artifact variability, i.e., at compile- and runtime. All variability decisions are resolved at a single new binding time, generation time, the moment the LLM produces the source code. Rather than treating this as a defect to fix, we propose Variability by Regeneration (VbR), to our knowledge the first product line approach in which the LLM acts as the derivation engine, generating a dedicated, free of dead code binary for each variant from a declarative specification, while a variant dispatcher transparently routes user requests to the matching binary. We formalise VbR, contrast it with classical SPL derivation, and demonstrate its full pipeline on a wc product family. For SPL engineering, variability in AI-generated software belongs in the specification, not in the code.
arXiv:2211.14966v2 Announce Type: replace Abstract: Deep neural networks (DNNs) are highly vulnerable to adversarial attacks. Ideally, a robust model should perform well on both perturbed training data and unseen perturbed test data. While DNNs can fit perturbed training data, generalizing to perturbed test data remains a significant challenge. This motivates the study of generalization guarantees from a learning theory perspective. This paper focuses on adversarial Rademacher complexity (ARC), first introduced by Khim and Loh (2018) and Yin et al. (2019). Their work primarily addressed linear functions and highlighted the open question of how to bound ARC for neural networks. Since then, several attempts have been made, with the latest results applying ARC only to two-layer neural networks. The main challenge arises from the dynamic nature and unknown closed-form solution of adversarial examples. In this paper, we resolve this issue and provide the first bound on ARC for deep neural networks. Our bound is qualitatively comparable to Rademacher complexity bounds in similar settings. The key ingredient is a new concept we introduce, termed intermediate adversarial examples, along with a framework for calculating the covering number that is compatible with them. Finally, we present experiments to analyze poor robust generalization, demonstrating that the weight norm is a crucial factor influencing the robust generalization gap.
arXiv:2301.12680v2 Announce Type: replace Abstract: We present a new algorithm to train a robust malware detector. Modern malware detectors rely on machine learning algorithms. Now, the adversarial objective is to devise alterations to the malware code to decrease the chance of being detected whilst preserving the functionality and realism of the malware. Adversarial learning is effective in improving robustness but generating functional and realistic adversarial malware samples is non-trivial. Because: i) in contrast to tasks capable of using gradient-based feedback, adversarial learning in a domain without a differentiable mapping function from the problem space (malware code inputs) to the feature space is hard; and ii) it is difficult to ensure the adversarial malware is realistic and functional. This presents a challenge for developing scalable adversarial machine learning algorithms for large datasets at a production or commercial scale to realize robust malware detectors. We propose an alternative; perform adversarial learning in the feature space in contrast to the problem space. We prove the projection of perturbed, yet valid malware, in the problem space into feature space will always be a subset of adversarials generated in the feature space. Hence, by generating a robust network against feature-space adversarial examples, we inherently achieve robustness against problem-space adversarial examples. We formulate a Bayesian adversarial learning objective that captures the distribution of models for improved robustness. We prove that our learning method bounds the difference between the adversarial risk and empirical risk explaining the improved robustness. We show that adversarially trained BNNs achieve state-of-the-art robustness. Notably, adversarially trained BNNs are robust against stronger attacks with larger attack budgets by a margin of up to 15% on a recent production-scale malware dataset of more than 20 million samples.