arXiv:2607.15589v1 Announce Type: new
Abstract: Communication-limited robots in mission-critical scenarios such as disaster inspection and search-and-rescue must make reliable onboard decisions without access to remote operators or high-capacity reasoning services. Episodic memory reuse is an attractive low-cost fallback, but retrieval similarity does not guarantee execution validity, i.e., a retrieved action may match the current context yet be unsafe due to changed topology, insufficient battery margin, or unreliable prior outcomes. We call such high-similarity but execution-invalid episodes memory traps. This creates a safety-efficiency design space where similarity only reuse minimizes fallback cost but can be unsafe, while always invoking local reasoning improves safety at high computational and energy cost. This paper presents MemoGuard, a lightweight adaptive runtime that validates episodic memories against topology, resource, and outcome contracts before reuse, invoking fallback only when validation fails. In a graph-based corridor-inspection simulator, MemoGuard reduces battery safety violations by 76.6% over similarity-only top-1 reuse while reducing fallback calls by 21.4% over always reasoning. On an NVIDIA Jetson AGX Xavier with local llama3.2:3b fallback reasoning, this corresponds to 3.67 s and 36.97 J of avoided fallback-reasoning overhead per trial. We open-source MemoGuard at https://github.com/hetheiin/memoguard.
Science Journals
arXiv:2607.15663v1 Announce Type: new
Abstract: In this paper, we aim to automate the adjustment of air handling unit (AHU) setpoints within heating, ventilation, and air conditioning (HVAC) systems to maintain indoor temperatures at user-specified levels. A key challenge lies in obtaining sufficient high-quality sensor data from real buildings. To address this, we explore transfer learning and leverage simulation software to generate training data. We propose an adaptive model-based transfer learning approach for dynamic HVAC control, where the agent directly controls the source domain under conditions identical to the target domain. This eliminates the need for extensive target-specific knowledge to define data generation schedules and reduces the risk of collecting irrelevant samples, while also providing greater flexibility during learning. At the control level, we enhance performance through physics rule embedding, which ensures physical consistency, and long-term-aware setpoint selection strategy, which mitigates abrupt setpoint changes. Finally, to accelerate and stabilize deployment in new buildings, we enable knowledge transfer directly between similar real-world buildings, reducing the need to construct virtual source domains repeatedly.
arXiv:2510.01475v2 Announce Type: replace
Abstract: Model Predictive Control (MPC) has demonstrated significant performance improvements over today's control methods for residential Heating, Ventilation, and Air Conditioning (HVAC), but deploying MPC often requires substantial engineering effort. Reinforcement Learning (RL) may offer comparable performance with easier deployment, but its practical application for residential HVAC remains largely undemonstrated, leaving open questions related to occupant comfort and data requirements. To investigate these issues, we deployed one MPC variant and one model-based RL variant for one month each in an occupied house in a cold climate. The controllers adjusted an air-to-air heat pump's thermostat temperature setpoint based on measurements of the indoor temperature and the electric power used for heating. Relative to constant-setpoint operation, MPC saved 18.1\% (95\% confidence interval: 4.4 to 30.9\%) of weather-normalized heat pump energy and RL saved 20.9\% (2.6 to 38.3\%). MPC maintained acceptable occupant comfort. RL kept the house cooler, particularly during an initial adaptation phase, leading to three reports of occupant discomfort. The two algorithms had similar data requirements. We estimate that for a fresh deployment in another house, RL would take about one-third less engineering effort than MPC. While RL reduces deployment effort, it faces difficulties related to safe controller initialization and to mismatches between the modeled and true state and action spaces.
arXiv:2510.01894v3 Announce Type: replace
Abstract: Many natural dynamic processes -- such as in vivo cellular differentiation or disease progression -- can only be observed through the lens of static sample snapshots. While challenging, reconstructing their temporal evolution to decipher underlying dynamic properties is of major interest to scientific research. Existing approaches enable data transport along a temporal axis but are poorly scalable in high dimension and require restrictive assumptions to be met. To address these issues, we propose Multi-Marginal temporal Schr\"odinger Bridge Matching (MMtSBM) from unpaired data, extending the theoretical guarantees and empirical efficiency of Diffusion Schr\"odinger Bridge Matching (arXiv:2303.16852) by deriving the Iterative Markovian Fitting algorithm to multiple marginals in a novel factorized fashion. Experiments show that MMtSBM retains theoretical properties on toy examples, achieves state-of-the-art performance on real-world datasets such as transcriptomic trajectory inference in 100 dimensions, and, for the first time, recovers couplings and dynamics in very high-dimensional image settings. Our work establishes multi-marginal Schr\"odinger bridges as a practical and principled approach for recovering hidden dynamics from static data.
arXiv:2607.15732v1 Announce Type: new
Abstract: Visual grounding with multimodal large language models is commonly formulated as autoregressive coordinate generation, where a model outputs bounding-box coordinates as text given an image and a referring-expression prompt. While this interface is simple and compatible with instruction following, it introduces a mismatch between training and evaluation: training optimizes token-level likelihood over coordinate strings, whereas grounding quality is measured by geometric overlap. We propose IoUPD, an IoU-aware privileged distillation method for coordinate-generating multimodal large language models. IoUPD uses ground-truth boxes not only as coordinate targets, but also as privileged training-time guidance. During training, the student receives the original image and prompt, while a frozen teacher receives a box-marked image and an augmented prompt that indicates the marked region. The student is trained with a supervised fine-tuning anchor and a privileged distillation loss whose token weights reflect both geometric importance and teacher reliability. At inference time, IoUPD requires no box overlay, privileged hint, teacher branch, or additional prediction module. Experiments on standard referring-expression grounding benchmarks show consistent region-level improvements over strong coordinate-generating baselines, demonstrating that ground-truth boxes can provide useful privileged guidance beyond serving as coordinate labels.
arXiv:2607.15674v1 Announce Type: new
Abstract: Autonomous robots operating in dynamic environments require behaviour planning systems that combine reactivity, interpretability, and adaptability. While Large Language Models have been successfully integrated with Behaviour Trees for dynamic replanning, Finite State Machines, despite their widespread adoption and computational efficiency, remain unexplored for generative approaches. We propose a Generative Partially Specified Finite State Machine (GPSFSM) neurosymbolic architecture that utilises the symbolic and semantic structure of FSMs, including states and event-triggered transitions, to implement Behaviour Planning. This paper introduces the first GPSFSM framework for robotics, featuring Fabric, an FSM engine that parses, validates, and executes behaviour plans that contain Sequential, Recovery, Parallel-Any, and Parallel-All control structures. We extend the Capabilities2 package in ROS2 with an asynchronous event system for behaviour chaining and runtime parameter injection for configurable execution, addressing the ad-hoc function representations that limit current generative systems. PromptTools provides a unified ROS 2 interface to local and cloud LLMs, with prompt buffering, enabling dynamic asynchronous composition of task and context information. Together, these components enable standardised semantic capability descriptions for robot-agnostic development. Experimental evaluation on navigation tasks demonstrates that our GPSFSM approach achieves consistently higher plan-generation success rates than the state-of-the-art BTGenBot system, particularly excelling in zero-shot scenarios where BTs typically struggle, while maintaining comparable or lower planning latency to frontier LLMs. We also demonstrate that our system can generate complex behaviours. We release an open-source ROS2 stack that makes generative FSM planning practical and reproducible for robotic systems.
arXiv:2607.15596v1 Announce Type: new
Abstract: The rapid adoption of artificial intelligence (AI)-driven and agentic workflows is transforming traditional government and enterprise systems into language-based, tool-using and increasingly autonomous infrastructures. Conventional key management services authenticate who may invoke a cryptographic primitive, but remain agnostic to which workflow steps are authorized at runtime: an authenticated agent can still be hijacked by direct or indirect prompt injection into executing malicious actions that satisfy identity checks yet violate user intent. We present \emph{Neural Cryptographic Services} (NCS), an active security governance plane grounded in a neural-symbolic design and interposed between LLM agents and privileged tools. Under NCS, an untrusted neural planner compiles natural-language instructions into structured plan drafts but holds no execution authority. Execution is instead gated by a deterministic symbolic controller operating over an offline-signed, hash-chained instruction stream. NCS verifies the signature, incrementally validates the hash chain, releases exactly one instruction payload at a time, and enforces strict binding between the agent's proposed tool parameters and the verified payload. Mismatched or out-of-order tool calls are rejected fail-closed, while previously verified states are retained for post-hoc auditing. We evaluate NCS on AgentDojo and a custom argument-hijacking benchmark. NCS drives attack success rates to near zero while preserving acceptable utility on benign workflows. NCS thus reframes agent security from asking whether a model's intent complied to asking whether a proposed dispatch matches a cryptographically authorized step.
arXiv:2607.15733v1 Announce Type: new
Abstract: Real-time collision avoidance for robotic manipulators requires fast reactions to unexpected obstacle motion and lookahead to avoid becoming trapped by near-future constraints. Full model predictive control can provide this foresight, but its online cost may grow quickly with horizon length, model fidelity, and the number of active geometric constraints. Conversely, horizon-free reactive methods are computationally efficient but can be short-sighted in dynamic clutter. We present a task-space receding-horizon controller that uses a short contact-consistent rollout to generate a terminal kinematic reference satisfying internal non-penetration constraints, then computes only the first input of a smooth minimum-acceleration transition toward that reference. Starting from a closed-loop inverse-kinematics regulation law, the rollout is performed with an iterative dynamics solver operating on inflated convex robot and obstacle geometries, so that robot-obstacle contacts, dynamic obstacle motion, and self-collisions can shape the terminal reference without requiring full constrained trajectory optimization. We analyze the contact-inactive closed loop and show local exponential task-space regulation under standard regularity assumptions. For contacts activated inside the rollout, we characterize the corresponding discrete updates and bound the effect of moving obstacles on regular operating sets. Simulations on a 40-DOF multi-chain system show that intermediate horizons balance anticipation, responsiveness, and computational cost. Hardware experiments on a 6-DOF platform demonstrate consistent sim-to-real behavior without accurate inertial parameter estimation, and comparisons against dynamic optimization fabrics and model predictive control (MPC) baselines show improved success rates in dynamic clutter while preserving solve times compatible with real-time execution in the tested regimes.
arXiv:2510.03504v3 Announce Type: replace
Abstract: Connectivity is crucial in many multi-robot applications, yet balancing connectivity maintenance and fleet traversability in obstacle-rich environments remains challenging. Reactive controllers based on control barrier functions can preserve connectivity when it is initially satisfied, but often struggle with deadlocks in cluttered environments. We propose a real-time B\'ezier-based constrained motion planning algorithm, namely MPC--CLF--CBF, that produces trajectories and control inputs concurrently, subject to high-order control barrier function and control Lyapunov function constraints. Our motion planner supports connectivity-aware navigation in cluttered workspaces and recovers connectivity from initially disconnected configurations and after temporary obstacle-induced separation; it also provides analytic continuous-time derivatives, facilitating its application to agile differentially flat systems such as quadrotors. In simulations with $4$--$12$ robots, it maintains $95.8$--$100\%$ graph-connected time at $20\%$ obstacle density, compared with $48.9$--$61.3\%$ for MPC--CBF, with no observed collisions. We further validate the planner in a physical experiment with $8$ Crazyflie nano-quadrotors.
arXiv:2510.05509v3 Announce Type: replace
Abstract: Diffusion models are powerful deep generative models, but unlike classical models, they lack an explicit low-dimensional latent space that parameterizes the data manifold. This absence makes it difficult to perform manifold-aware operations, such as geometrically faithful interpolation or conditional guidance that respects the learned manifold. We propose a training-free Riemannian metric on the noise space, derived from the Jacobian of the score function. The key insight is that the spectral structure of this Jacobian separates tangent and normal directions of the data manifold; our metric leverages this separation to encourage paths to stay tangential to the manifold rather than drift toward high-density regions. To validate that our metric faithfully captures the manifold geometry, we examine it from two complementary angles. First, geodesics under our metric yield perceptually more natural interpolations than existing methods on synthetic, image, and video frame datasets. Second, the tangent-normal decomposition induced by our metric prevents classifier-free guidance from deviating off the manifold, improving generation quality while preserving text-image alignment.
arXiv:2510.05750v2 Announce Type: replace
Abstract: Graph neural networks (GNNs) have achieved remarkable success in node classification. Building on this progress, heterogeneous graph neural networks (HGNNs) integrate relation types and node and edge semantics to leverage heterogeneous information. Causal analysis for HGNNs is advancing rapidly, aiming to separate genuine causal effects from spurious correlations. However, whether HGNNs are intrinsically effective for node classification remains underexamined, and most studies implicitly assume rather than establish this effectiveness. In this work, we examine HGNNs for node classification from two perspectives: model architecture and heterogeneous information. We conduct a systematic reproduction across 21 datasets and 20 baselines, complemented by comprehensive hyperparameter retuning. To further disentangle the source of performance gains, we develop a causal mediation analysis framework that treats the introduction of heterogeneous relation information as the treatment, candidate structural properties as mediators, and node classification performance as the outcome. This framework first screens candidate mediators according to their treatment-induced changes and their associations with performance improvement, and then decomposes the total effect into mediated and direct effects. Our results lead to two conclusions. First, model architecture and complexity have no causal effect on node classification performance. Second, heterogeneous information exerts a positive causal effect primarily through increasing homophily and local-global distribution discrepancy, which makes node classes more distinguishable. The implementation is publicly available at https://github.com/YXNTU/CausalHGNN.
arXiv:2510.11539v5 Announce Type: replace
Abstract: Accurate state estimation is critical for legged and aerial robots operating in dynamic, uncertain environments. A key challenge lies in specifying process and measurement noise covariances, which are typically unknown or manually tuned. In this work, we introduce a bi-level optimization framework that jointly calibrates covariance matrices and kinematic parameters in an estimator-in-the-loop manner. The upper level treats noise covariances and model parameters as optimization variables, while the lower level executes a full-information estimator. Differentiating through the estimator allows direct optimization of trajectory-level objectives, resulting in accurate and consistent state estimates. We validate our approach on quadrupedal and humanoid robots, demonstrating significantly improved estimation accuracy and uncertainty calibration compared to hand-tuned baselines. Our method unifies state estimation, sensor, and kinematics calibration into a principled, data-driven framework applicable across diverse robotic platforms.
arXiv:2510.11600v3 Announce Type: replace
Abstract: We present a data-driven active flow control framework designed for deployment with few non-intrusive sensors. The method builds upon Artificial Intelligence driven reduced-order predictive models based on Long-Short-Term Memory (LSTM) networks and efficient gradient-based Model Predictive Control (MPC). The model uses only surface-mounted pressure probes to infer the wake state, and is trained entirely offline on a dataset built with open-loop actuations, thus avoiding the complexities of online learning. Sparsification of the sensors needed for control from an initially large set is achieved using SHapley Additive exPlanations (SHAP). A parsimonious set of sensors is then deployed in closed-loop control with MPC. The framework is tested in numerical simulations of a two-dimensional truck model at Reynolds number 500, with pulsed-jet actuators placed in the rear of the truck to control the wake. The resulting LSTM-MPC achieved a drag reduction of 12.8\%.
arXiv:2607.15482v1 Announce Type: new
Abstract: The increasing complexity of state-of-the-art machine learning models has made their behavior progressively harder to interpret, spurring rapid advancements in the field of eXplainable Artificial Intelligence (XAI). Among many methods proposed, perturbation-based approaches play a major role. By systematically altering (perturbing) input features, these approaches measure the impact on the model's predictions. For image data, traditional perturbation techniques, often involve replacing pixel values e.g., with a pre-defined color. However, such approaches, but also more refined deterministic techniques, generate unrealistic out-of-distribution samples and often leave visible artifacts, which can mislead the model and compromise explanation quality. In this work, we adjust LIME, a widely used perturbation-based method, to demonstrate how generative inpainting can improve perturbation-based explanations for images. We achieve photorealistic perturbed samples that align better with the original data distribution and enhance explanation quality.
arXiv:2607.15686v1 Announce Type: new
Abstract: We present S1-Omni, a unified multimodal reasoning model for scientific understanding, prediction, and generation. AI for Science (AI4S) has advanced significantly through domain-specific models, tool-augmented LLMs, and scientific language models. However, model capabilities remain highly fragmented, limiting the joint modeling of heterogeneous data, scientific laws, and expert knowledge. S1-Omni addresses this gap by consolidating these capabilities into a single, coherent scientific reasoning model. The architecture of S1-Omni is built upon three core components: unified representation of scientific data, natural-world knowledge alignment, and decoding for domain-specific tasks. First, S1-Omni maps natural-language instructions and scientific objects, including CIF, SMILES, protein sequences, spectra, and scientific images, into a shared representation space. Second, it incorporates scientific laws and expert knowledge into data construction and training, enabling the model to reason from scientific evidence. Third, it performs task-specific decoding to support a broad range of applications, including property prediction, spectrum-to-molecular generation, protein site and structure prediction, and scientific image generation and editing. S1-Omni is trained on S1-Omni-Corpus, which covers 200 scientific tasks and contains millions of reasoning samples, and is evaluated on over 60 scientific benchmarks. It outperforms GPT-5.5 and Gemini-3.1-Pro on most benchmarks and matches or surpasses domain-specific models on several benchmarks. Overall, S1-Omni provides a practical path toward unified scientific modeling.
Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Framework
arXiv:2607.15687v1 Announce Type: new
Abstract: Multimodal-attributed graphs (MAGs), whose nodes carry modalities such as images and text alongside topological structure, now pervade applications including social platforms, e-commerce, and biomedical networks, offering richer semantic signals than single-modality graphs. In practice, such graphs are fragmented across privacy-restricted silos owned by different platforms and institutions, so learning a broadly transferable model over them demands collaborative training that never exposes raw data. This places the task at the intersection of multimodal graph learning and federated learning, yet existing methods cover only one side of it. To address the challenges from these two perspectives, we propose FedGAMMA, casting federated multimodal graph foundation learning as a two-stage semantic-structural alignment problem of federated pre-training and prompt-based fine-tuning. During pre-training, a shared-private semantic enhancer disentangles cross-modal commonality from modality-specific information, aligning it through optimal transport, a topology-aware graph fusion module decouples semantic and structural views via semantic residual graphs and dual positional encodings, and a dual-channel affinity-aware aggregation mechanism estimates client similarity from feature and graph centroids without exposing raw data. During fine-tuning, FedGAMMA adapts the pretrained encoder through lightweight graph-aware prompts, a shared prompt pool with controlled exploration, and channel-wise prompt synchronization. Experiments on twelve multimodal graph datasets show FedGAMMA consistently surpassing a broad range of baselines across downstream tasks, with gains of up to 12.96%. FedGAMMA further outperforms competitive baselines accross multi-domain datasets on multiple tasks with up to 5.71% under few-shot learning scenario.
arXiv:2607.15734v1 Announce Type: new
Abstract: This paper presents a comprehensive analysis of the results from the Third International Competition on Document Forgery Detection on ID-Cards and Passports, which was held across two distinct tracks. Track 1 evaluates a synthetic-data-based ID-PAD system under controlled but diverse conditions, where the winning team, \textit{Incode}, achieves an $AV_{Rank}$ of 27.82%, confirming consistent performance across metrics and highlighting the importance of a balanced, generalizable design. In Track 2, the challenge intensifies with heterogeneous attack scenarios across different domains, where \textit{Incode} again achieved the top position with an $AV_{Rank}$ of 68.71% across thresholds, outperforming some baselines and established methods. These results demonstrate that PAD effectiveness requires not only high accuracy but also consistency across diverse attack types and imaging conditions. The success of this initiative across both tracks underscores the value of collaboration between companies and academic teams. This year, more than \textit{63 teams} were registered, and more than \textit{100 submission models} were evaluated. This competition has evolved into a leading benchmark state-of-the-art in PAD on ID documents, setting the standard for performance, reproducibility, and real-world applicability in secure identity verification.
arXiv:2607.15740v1 Announce Type: new
Abstract: As Text-to-Image (T2I) systems rapidly advance, evaluating the cultural authenticity of synthesized content has become increasingly important for fair and trustworthy generative AI. Existing T2I evaluation metrics and multimodal judges often rely on visual-semantic representations that underrepresent implicit cultural norms, leading to biased preference judgments and the omission of fine-grained cultural cues. In addition, visual question answering (VQA)-based evaluators typically depend on autoregressive text generation, which limits their scalability for real-time reward modeling. To address these limitations, we introduce an Implicit Cultural Alignment Reward Model built upon a lightweight 4.2-billion-parameter Multimodal Large Language Model (MLLM). Our framework integrates an Implicit Cultural Probe with a Skip-connection Cross-Attention (SkipCA) mechanism, enabling late-stage semantic features to directly attend to early-stage visual representations and better preserve culturally salient details. Evaluations on 3,323 challenging and carefully curated image pairs from the CulturalFrames benchmark show that our approach achieves 80.54% pairwise accuracy, with Pearson and Kendall correlation coefficients of 0.546 and 0.377, respectively, outperforming representative vision-language metrics and MLLM-based evaluators. Moreover, by bypassing autoregressive text generation, our model processes each evaluation in 0.21 seconds under our local inference setup, achieving a $10\times$ speedup over standard VQA-based evaluators. These results suggest that the proposed reward model can provide an efficient and culturally aware scalar signal for preference optimization pipelines such as Reinforcement Learning from Human Feedback and Direct Preference Optimization.
arXiv:2607.15898v1 Announce Type: new
Abstract: Current world models operate at a single level of abstraction, with most prioritizing perceptual fidelity while lacking the spatial reasoning and semantic understanding required for real-world downstream tasks. We present a hierarchical driving world model that factorizes future prediction across two levels operating at distinct temporal and abstraction scales: a high-level predictor that forecasts coarse scene structure over extended temporal horizons, and a low-level generator that produces detailed predictions conditioned on the high-level output. This decomposition yields high perceptual fidelity while also capturing strong spatial and semantic representations. We further show that pretraining with a diffusion forcing objective yields substantially richer internal representations than the standard teacher forcing objective, while teacher forcing -- predicting only the next frame from clean context -- produces more stable autoregressive rollouts. We therefore introduce a generic two-stage training paradigm that pretrains the model with diffusion forcing and fine-tunes with teacher forcing, combining the representational benefits of the former with the rollout stability of the latter. Our approach achieves state-of-the-art results across the standard suite of driving world model evaluations on established benchmarks, including long-horizon generation fidelity, steering responsiveness evaluated on counterfactual scenarios, and internal representation quality. Project page with code, demo, checkpoints and qualitative results: https://lmb-freiburg.github.io/orbis2.github.io/
arXiv:2607.15752v1 Announce Type: new
Abstract: Personalized Image Aesthetic Assessment (PIAA) aims to predict aesthetic ratings of images that vary across individuals. The aesthetic preferences manifest to different extents across distinct visual stimuli and exhibit cohort-specific patterns. Motivated by the above fact, this paper presents a Multimodal Large Language Model (MLLM)-based approach, which models individual aesthetic preferences by Preference-Rich sample mining and Aesthetically-resonant Cohort merging (PRAC). Specifically, PRAC first identifies preference-rich samples by analyzing both Collective Controversy and Personalized Deviation of images, maximizing the utility of limited user data. Based upon the preference-rich samples, cross-user preference similarities are measured by comparing preference embeddings. Then, a cohort-based model merging strategy, is proposed by aggregating preference patterns from aesthetically-resonant users, which further enhances the personalization for the target individual. Extensive experiments and comparisons on four benchmark PIAA databases demonstrate the superiority of the proposed PRAC model over the state-of-the-arts. The code and model will be public at https://github.com/yzc-ippl/PRAC.
arXiv:2607.16060v1 Announce Type: new
Abstract: Artistic charts make data memorable and visually engaging, but generating them faithfully demands simultaneously preserving numerical geometry, rendering exact in-image text, binding labels to correct marks, and maintaining coherent artistic style. Current text-to-image and image editing models frequently fail on these coupled constraints, producing distorted geometries, hallucinated text, misbound labels, or over-stylized marks that undermine readability and mathematical integrity. This paper introduces ArtChart, a framework for artistic chart generation with integrated text rendering, encompassing a task definition, benchmark, and evaluation protocol. This is the first work to simultaneously address mathematically faithful chart synthesis, accurate in-image text rendering, and artistic stylization of chart elements. ArtChart features a chart-specific plug-and-play module conditioned on text-free grayscale chart layouts, ensuring mathematical and logical fidelity. A RL learning strategy with OCR accuracy, layout quality, and aesthetic rewards refines generation, while a multi-expert distillation framework resolves inter-reward conflicts through specialized expert optimization. We construct ArtChart-Bench, a bilingual 2K-prompt benchmark spanning four chart types and diverse label formats. We further design ArtChart-Eval, a six-axis evaluation suite covering mathematical logic, text accuracy, text layout, aesthetics, instruction following, and readability, supporting comparison across T2I, image-editing, controllable-generation, and closed-source API models. Extensive experiments demonstrate that ArtChart consistently outperforms open-source baselines, producing charts that are both visually appealing and math faithful.
arXiv:2607.16107v1 Announce Type: cross
Abstract: We present Audio-Visual Flamingo (AV-Flamingo), a fully open state-of-the-art audio-visual large language model (AV-LLM) for joint understanding and reasoning over audio, images, and long-form videos. Unlike prior AV-LLMs that primarily focus on short clips, AV-Flamingo is designed for understanding and reasoning over long and complex real-world (audio-visual) videos. To support this, we make three key contributions: (i) Audio-Visual-Skills, a large-scale collection of real-world videos with ~7M caption and question-answer training instances designed to emphasize temporal, compositional, and cross-modal audio-visual reasoning; (ii) a novel three-stage curriculum that progressively trains the model from short-range perception to long-horizon multi-event reasoning; and (iii) Temporal Audio-Visual Interleaved Chain-of-Thought, a reasoning framework that explicitly grounds intermediate reasoning steps to timestamps in long audio-visual streams, improving temporal alignment and interpretability. Extensive experiments across 15+ audio-visual, omni-modal, audio, and vision benchmarks show that AV-Flamingo outperforms similarly sized open models by clear margins and remains highly competitive with, and in some cases surpasses, much larger open-weight and closed models, particularly on long and complex real-world audio-visual understanding and reasoning tasks. Beyond benchmark performance, AV-Flamingo exhibits strong real-world utility and transfers well to unseen tasks, highlighting its robustness and generalization ability.
arXiv:2607.15772v1 Announce Type: new
Abstract: Maintaining recurring character identities across scene transitions and long temporal gaps is a central challenge in narrative long video generation. Methods targeting global consistency often retrieve memory using cues that are not aligned with character identity preservation, while recent character-centric variants still rely on coarse frame-level kv memory that entangles identity with incidental visual factors and lacks a continuous update mechanism under limited memory capacity. To address these limitations, we propose \textbf{SlotMem}, a character-addressable internal memory framework for multi-character narrative long video generation. Specifically, SlotMem uses a Character-Semantic Probe to localize character-relevant visual tokens from cross-attention responses, and a Memory Encoder to compress DiT tokens into compact role-wise slot memory. As generation proceeds, a Memory Writer conservatively updates each character's memory with new observations, while Character-Wise Cross-Attention retrieves the role memory and injects it only into localized tokens of the same character. Experiments on multiple narrative long video generation benchmarks show that SlotMem improves long-range character consistency over existing baselines, while maintaining comparable video quality. Our code is available at https://github.com/YilaiLiu-HKU/SlotMem.
arXiv:2505.19909v2 Announce Type: replace
Abstract: Neural Network-based Quantum Monte Carlo (NNQMC), an emerging method for solving many-body quantum systems with high accuracy, has been mainly applied to small systems due to demanding computation requirements. In this work, we introduce a framework based on local pseudopotentials to break through such limitation, improving the computational efficiency and scalability of NNQMC. The incorporation of local pseudopotentials reduces the number of electrons treated in neural network and also achieves better relative energy accuracy than all electron NNQMC calculations for complex systems. This counterintuitive outcome is made possible by the distinctive characteristics inherent to NNQMC. Notably, by avoiding costly integration terms, this approach is also substantially more efficient than its widely used semilocal counterparts. Our approach enables the reliable treatment of large and challenging systems, such as the $\text{Fe}_4 \text{S}_4 (\text{SCH}_3)_4$ iron-sulfur cluster. Overall, our findings demonstrate that the synergy between NNQMC and local pseudopotentials substantially expands the scope of accurate ab initio calculations.
arXiv:2607.15746v1 Announce Type: new
Abstract: Robotic systems increasingly demand tactile sensing that approaches the adaptability and resolution of human skin to enable dexterous manipulation and safe interaction. OptiTac is a biomimetic tactile sensor that emulates the mechanoreceptor-to-nerve architecture of human touch by pairing each mechanical pin on a soft skin with an optical fiber acting as an artificial nerve. This design demonstrates an architectural principle for routing tactile information away from the sensing surface while preserving high spatial resolution, establishing a practical route toward distributed tactile sensing in future robotic systems. By treating tactile signals as images, simple analytical methods, rather than opaque deep-learning models, are used to infer contact location, size, and shape, providing interpretable and scalable tactile intelligence. This work demonstrates how evolutionary principles from biology can guide the development of artificial nerve systems for robots, offering a pathway toward human-like tactile perception in next-generation robotic platforms. More broadly, OptiTac establishes an artificial nerve-inspired sensing framework for interpretable robotic touch and a scalable route toward future distributed tactile systems.