Forskningsradar

Science Journals

Peer-reviewade publikationer — 55483 artiklar

Few-Step Diffusion Language Models via Trajectory Self-Distillation
arXiv:2602.12262v4 Announce Type: replace Abstract: Diffusion large language models (DLLMs) have emerged as powerful generative models with the promise of fast text generation through parallel decoding. However, realizing this potential in practice remains challenging: reducing the number of decoding steps, typically causes a substantial degradation in output quality due to token factorization error. To alleviate this, we propose a self-distillation framework that trains a few-step student to match the generative trajectory of a full-step teacher. We theoretically and empirically show that trajectory-level supervision mitigates this factorization error, thereby enabling effective few-step decoding. We further incorporate Direct Discriminative Optimization (DDO), a reverse-KL objective that encourages mode-seeking toward the teacher's modes, yielding stronger performance on challenging reasoning tasks. Across reasoning and code-generation benchmarks, our method substantially narrows the gap between few-step and full-step decoding. The source code is available at https://github.com/Tyrion58/T3D.
Opus: Photonic Rail-Optimized Fabric in ML Datacenters
arXiv:2602.12521v3 Announce Type: replace Abstract: Rail-optimized network fabrics have become the de facto datacenter scale-out fabric for large-scale ML training. However, the use of high-radix electrical switches to provide all-to-all connectivity in rails imposes substantial power and cost. We propose a rethinking of the rail abstraction by retaining its communication semantics, but realizing it using optical circuit switches. The key challenge is that optical switches support one-to-one connectivity at a time, limiting the fan-out of traffic in ML workloads using hybrid parallelisms. We overcome this through \emph{parallelism-driven rail reconfiguration}, which exploits the non-overlapping communication phases of different parallelism dimensions. This time-multiplexes a single set of physical ports across circuit configurations tailored to each phase within a training iteration. We design and implement Opus, a control plane that orchestrates this in-job reconfiguration of photonic rails at parallelism phase boundaries, and evaluate it on a physical OCS testbed, the Perlmutter supercomputer, and in simulation at up to 2,048 GPUs. Our results show that photonic rails can achieve over $23\times$ network power reduction and $4\times$ cost savings while incurring only modest training overhead at production-relevant OCS reconfiguration latencies.
Explicit Credit Assignment through Local Rewards and Dependence Graphs in Multi-Agent Reinforcement Learning
arXiv:2601.21523v2 Announce Type: replace Abstract: To promote cooperation in Multi-Agent Reinforcement Learning, the reward signals of all agents can be aggregated together, forming global rewards that are commonly known as the fully cooperative setting. However, global rewards are usually noisy because they contain the contributions of all agents, which have to be resolved in the credit assignment process. On the other hand, using local reward benefits from faster learning due to the separation of agents' contributions, but can be suboptimal as agents myopically optimize their own reward while disregarding the global optimality. In this work, we propose a method that combines the merits of both approaches. By using a graph of interaction between agents, our method discerns the individual agent contribution in a more fine-grained manner than a global reward, while alleviating the cooperation problem with agents' local reward. We also introduce a practical approach for approximating such a graph. Our experiments demonstrate the flexibility of the approach, enabling improvements over the traditional local and global reward settings.
Efficient privacy loss accounting for subsampling and random allocation
arXiv:2602.17284v2 Announce Type: replace Abstract: We consider the privacy amplification properties of a sampling scheme in which a user's data isused in $k$ steps chosen randomly and uniformly from a sequence (or set) of $t$ steps. This sampling scheme has been recently applied in the context of differentially private optimization (Chua et al., 2024a; Choquette-Choo et al., 2025) and communication-efficient high-dimensional private aggregation (Asi et al., 2026), where it was shown to have utility advantages over the standard Poisson sampling. Theoretical analyses of this sampling scheme (Feldman & Shenfeld, 2025; Dong et al., 2025) lead to bounds that are close to those of Poisson sampling, yet still have two significant shortcomings. First, in many practical settings, the resulting privacy parameters are not tight due to the approximation steps in the analysis. Second, the computed parameters are either the hockey stick or Renyi divergence, both of which introduce overheads when used in privacy loss accounting. In this work, we demonstrate that the privacy loss distribution (PLD) of random allocation applied to any differentially private algorithm can be computed efficiently. When applied to the Gaussian mechanism, our results demonstrate that the privacy-utility trade-off for random allocation is at least as good as that of Poisson subsampling. In particular, random allocation is better suited for training via DP-SGD. To support these computations, our work develops new tools for general privacy loss accounting based on a notion of PLD realization. This notion allows us to extend accurate privacy loss accounting to subsampling which previously required manual noise-mechanism-specific analysis.
ParEVO: Synthesizing Code for Irregular Data: High-Performance Parallelism through Agentic Evolution
arXiv:2603.02510v2 Announce Type: replace Abstract: The transition from sequential to parallel computing is essential for modern high-performance applications but is hindered by the steep learning curve of concurrent programming. This challenge is magnified for irregular data structures (such as sparse graphs, unbalanced trees, and non-uniform meshes) where static scheduling fails and data dependencies are unpredictable. Current Large Language Models (LLMs) often fail catastrophically on these tasks, generating code plagued by subtle race conditions, deadlocks, and sub-optimal scaling. We bridge this gap with ParEVO, a framework designed to synthesize high-performance parallel algorithms for irregular data. Our contributions include: (1) The Parlay-Instruct Corpus, a curated dataset of 13,820 tasks synthesized via a "Critic-Refine" pipeline that explicitly filters for empirically performant algorithms that effectively utilize Work-Span parallel primitives; (2) specialized DeepSeek, Qwen, and Gemini models fine-tuned to align probabilistic generation with the rigorous semantics of the ParlayLib library; and (3) an Evolutionary Coding Agent (ECA) that improves the "last mile" of correctness by iteratively repairing code using feedback from compilers, dynamic race detectors, and performance profilers. On the ParEval benchmark, ParEVO achieves an average 106x speedup (with a maximum of 1103x) across the suite, and a robust 13.6x speedup specifically on complex irregular graph problems, outperforming state-of-the-art commercial models. Furthermore, our evolutionary approach matches state-of-the-art expert human baselines, achieving up to a 4.1x speedup on specific highly-irregular kernels. Source code and datasets are available at https://github.com/WildAlg/ParEVO.
SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity
arXiv:2607.04189v1 Announce Type: new Abstract: Federated Learning (FL) is fundamentally challenged by statistical heterogeneity, where non-identically distributed (non-IID) data induces client drift that severely hampers global convergence. While existing approaches attempt to mitigate this drift through spatial-domain gradient correction or regularization, they overlook the intrinsic spectral structure of optimization signals. In this work, we revisit client drift from a novel frequency-domain perspective and uncover a critical Spectral Bias of Drift: inter-client gradient divergence is predominantly concentrated in low-frequency components which encode client-specific distributional shifts, while high-frequency components representing fine-grained features remain relatively consistent. Motivated by this, we propose SpecGradFilter, a unified Spectral Gradient Filtering Framework that tames heterogeneity by suppressing discordant low-frequency signals. Crucially, we demonstrate that SpecGradFilter is a generalizable principle, effective not only via precise FFT-based truncation but also through spatial approximations like Gaussian detrending. Extensive experiments on benchmarks such as CIFAR-10/100 and Tiny-ImageNet demonstrate that SpecGradFilter significantly performs better performance in highly Non-IID settings with negligible communication overhead, establishing a new paradigm for robust federated optimization.
Graph Unitary Message Passing
arXiv:2403.11199v2 Announce Type: replace Abstract: Unitarity is a useful principle for stabilizing deep neural networks, but in graph neural networks (GNNs) instability is induced not only by learnable parameters but also by the graph propagation operator. Motivated by this distinction, we propose Graph Unitary Message Passing (GUMP), a message-passing framework that uses a unitary propagation operator on a transformed graph to avoid graph-induced exponential decay under repeated propagation. GUMP combines (i) a graph transformation that maps an input graph to an Eulerian line-graph construction admitting unitary adjacency matrices, and (ii) a practical unitary projection procedure based on Newton-Schulz iteration. Theoretical analysis clarifies that, under standard analysis assumptions, unitary propagation keeps the graph-propagation term depth-stable, while vanilla normalized propagation exhibits exponential decay in its non-trivial spectral components. Across synthetic long-range tasks, TUDataset benchmarks, and LRGB datasets, GUMP improves over vanilla message passing and achieves competitive or superior performance against strong baselines. Code is available at https://github.com/ucker/gump_code.
Computational Complexity of the Recoverable Robust Shortest Path Problem with Discrete Recourse
arXiv:2403.20000v3 Announce Type: replace Abstract: In this paper the recoverable robust shortest path problem is investigated. Discrete budgeted interval uncertainty representation is used to model uncertain second-stage arc costs. The known complexity results for this problem are strengthened. It is shown that it is Sigma_3^p-hard for the arc exclusion and the arc symmetric difference neighborhoods. Furthermore, it is also proven that the inner adversarial problem for these neighborhoods is Pi_2^p-hard.
Local Order Average-Atom Interatomic Potentials
arXiv:2510.06459v5 Announce Type: replace-cross Abstract: This article describes an extension to the effective Average-Atom (AA) method for random alloys to account for local ordering (short-range order) effects by utilizing information from partial radial distribution functions. The new Local Order Average-Atom (LOAA) method is rigorously derived based on statistical mechanics arguments and validated for non-stoichiometric binary 2D hexagonal crystals and 3D FeNiCr and NiAl alloys whose ground state is obtained through Monte Carlo sampling. Material properties for these alloys computed from atomistic simulations using standard interatomic potentials (IPs) exhibit a strong dependence on local ordering that is captured by simulations with effective LOAA IPs, but not the original AA method. The advantage of LOAA is that it requires smaller system sizes to achieve statistically converged results and therefore enables the simulation of complex materials, such as high-entropy alloys, at a fraction of the computational cost of standard IPs.
Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalation
arXiv:2603.17673v2 Announce Type: replace Abstract: LLM agents are becoming increasingly important in the security domain, but leading systems are often closed-source, cloud-based, hard to reproduce or use with sensitive code. This creates a need for small, local models that can perform security tasks under strict resource constraints, though effective methods for developing them remain unexplored. In this paper, we address this gap by proposing a two-stage post-training recipe that turns a small local language model into a security agent. To this end, we focus on Linux privilege escalation as a representative setting to systematically study the training of local models, as the task is both automatically verifiable and requires multi-step interactive reasoning. Using an experimental setup that mitigates data leakage, we post-train a small 4B model in two stages: supervised fine-tuning on traces from procedural privilege-escalation environments, followed by reinforcement learning with verifiable rewards. On a held-out benchmark of 12 Linux privilege-escalation scenarios, supervised fine-tuning doubles the baseline success rate under a tight budget of 20 interaction rounds, and subsequent reinforcement learning training improves our model, PrivEsc-LLM 4B, to 93.3% success, behind only Claude Opus 4.7 at this budget. At the same time, the expected inference cost per successful escalation decreases by more than 80x. Our findings not only show that small local models can be adapted to complex security tasks, but also document the challenges involved, offering guidance for transferring this recipe to other settings.
Correlation-Weighted Multi-Reward Optimization for Compositional Generation
arXiv:2603.18528v2 Announce Type: replace Abstract: Text-to-image models produce images that align well with natural language prompts, but compositional generation has long been a central challenge. Models often struggle to satisfy multiple concepts within a single prompt, frequently omitting some concepts and resulting in partial success. Such failures highlight the difficulty of jointly optimizing multiple concepts during reward optimization, where competing concepts can interfere with one another. To address this limitation, we propose Correlation-Weighted Multi-Reward Optimization (CMO), a framework that leverages the correlation structure among concept rewards to adaptively weight each attribute concept in optimization. By accounting for interactions among concepts, CMO balances competing reward signals and emphasizes concepts that are partially satisfied yet inconsistently generated across samples, improving compositional generation. Specifically, we decompose multi-concept prompts into pre-defined concept groups (e.g., objects, attributes, and relations) and obtain reward signals from dedicated reward models for each concept. We then adaptively reweight these rewards, assigning higher weights to conflicting or hard-to-satisfy concepts using correlation-based difficulty estimation. By focusing optimization on the most challenging concepts within each group, CMO encourages the model to consistently satisfy all requested attributes simultaneously. We apply our approach to train state-of-the-art diffusion models, SD3.5 and FLUX.1-dev, and demonstrate consistent improvements on challenging multi-concept benchmarks, including ConceptMix, GenEval 2, and T2I-CompBench. The code is available at https://github.com/TheDarkKnight-21th/CMO.
Markov Chain Monte Carlo in Tensor Network Representation
arXiv:2412.02974v2 Announce Type: replace-cross Abstract: Markov chain Monte Carlo (MCMC) is a powerful tool for sampling from complex probability distributions. Despite its versatility, MCMC often suffers from strong autocorrelation and the negative sign problem, leading to slowing down the convergence of statistical error. We propose a novel MCMC formulation based on tensor network representations to reduce the population variance and mitigate these issues systematically. By introducing stochastic projectors into the tensor network framework and employing Markov chain sampling, our method eliminates the systematic error associated with low-rank approximation in tensor contraction while maintaining the high accuracy of the tensor network method. We demonstrate the effectiveness of the proposed method on the two-dimensional Ising model, achieving an exponential reduction in statistical error with increasing bond dimension cutoff. Furthermore, we address the sign problem in systems with negative weights, showing significant improvements in average signs as bond dimension cutoff increases. The proposed framework provides a robust solution for accurate statistical estimation in complex systems, paving the way for broader applications in computational physics and beyond.
Polarization of photoluminescence by optically driven orbital reconstruction in magnetically ordered CrCl3
arXiv:2502.16285v2 Announce Type: replace-cross Abstract: Control of magnetic order through optical manipulation of orbital states can be realized in a wide range of materials, including insulators. In the latter case, electrons are excited into different electronic states, whose character may be a complex result of spin-orbit-lattice coupling. Here, we use a combination of polarization-resolved photoluminescence spectroscopy and density functional theory calculations to study excited Cr3+ ions in bulk, insulating antiferromagnetic CrCl3. Our results demonstrate a temperature-dependent preferential polarization direction below ~17 K, which reorients in the presence of an applied in-plane magnetic field. Density functional theory calculations reveal a local distortion of the ligand field and the generation of out-of-plane spin and orbital moments, following Cr3+ ion excitation. These changes link the photoluminescence polarization to the magnetic state of CrCl3. Our findings provide insight into the excited-state electronic structure of Cr3+ ions and suggest that CrCl3 is a promising candidate for opto-orbitronic applications.
A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective
arXiv:2503.23650v2 Announce Type: replace Abstract: Reinforcement learning (RL), with its ability to explore and optimize policies in complex, dynamic decision-making tasks, has emerged as a promising approach to addressing motion planning (MoP) challenges in autonomous driving (AD). Despite rapid advancements in RL and AD, a systematic description and interpretation of the RL design process tailored to diverse driving tasks remains underdeveloped. This survey provides a comprehensive review of RL-based MoP for AD, focusing on lessons from task-specific perspectives. We first outline the fundamentals of RL methodologies, and then survey their applications in MoP, analyzing scenario-specific features and task requirements to shed light on their influence on RL design choices. Building on this analysis, we summarize key design experiences, extract insights from various driving task applications, and provide guidance for future implementations. Additionally, we examine the frontier challenges in RL-based MoP, review recent efforts to addresse these challenges, and propose strategies for overcoming unresolved issues.
Foundation Models for Software Engineering of Cyber-Physical Systems: the Road Ahead
arXiv:2504.04630v3 Announce Type: replace Abstract: Foundation Models (FMs), particularly Large Language Models (LLMs), are increasingly used to support various software engineering activities (e.g., coding and testing). Their adoption in software engineering of Cyber-Physical Systems (CPSs) is also growing. However, research in this area remains limited. Most existing studies have primarily focused on LLMs, only one type of FM, leaving ample opportunities to explore other FMs, such as vision-language models. We argue that, in addition to LLMs, other FMs utilizing different data modalities (e.g., images, audio) and multimodal models (which integrate multiple modalities) hold great potential for supporting CPS software engineering, given that these systems process diverse data types. To address this, in this first systematic effort, we present a forward-looking research roadmap for integrating FMs into commonly known phases of CPS software engineering, thereby making it accessible to most software engineers. We derive the roadmap from the literature, emerging trends in FMs, and gaps identified from the literature. The roadmap highlights key challenges and actionable research opportunities for the software engineering community to guide future research. Moreover, we discuss the common challenges associated with applying FMs across six dimensions (e.g., technical, economic and resource, and human aspects). This roadmap aims to provide a visionary guide for researchers and practitioners, outlining directions for future work and performing future empirical studies.
HilEnT: Hilbert, Entropy Transformed Image Based Malware Detection
arXiv:2607.04772v1 Announce Type: new Abstract: With the increasing threat of malware across various software related domains, malware detection and classification is critical to determine the response actions. Different strategies have been adopted to address the challenge of malware detection. With the advent of deep learning techniques, malware detection using image processing has garnered research attention. In this work, we proposed a novel malware binary to image transformation technique HilEnT based on a combination of Hilbert curve-based transformation of malware binary and the entropy feature comparison of malware file with benign and malware classes. Three grayscale images produced during this process are combined to form a three-channel colored image which is then used for malware detection using machine learning techniques. We performed supervised binary and multiclass classification to evaluate the effectiveness of our proposed HilEnT. We also evaluated a few-shot learning technique to assess the robustness of our proposed HilEnT in a practical setting where the number of available class samples is limited. Furthermore, we investigated the benefits of combination of Histogram of Oriented Gradients and Principal Component Analysis for time performance improvements through feature reduction techniques. We evaluated our proposed methodology on four datasets: Dike, Michael Lester Dataset, Microsoft BIG 2015 and a self-collected dataset, and achieved the state-of-the-art results.
Resolving the inverse problem in pulse response analysis of TAP reactors
arXiv:2607.04776v1 Announce Type: new Abstract: Pulse experiments in the temporal analysis of products (TAP) reactor are one of the most important methods for studying transient kinetics of gas-solid catalytic reactions. The Y-procedure (Yablonsky et al., Chem. Eng. Sci. 62, 6754, 2007) is a model-free analysis framework for inferring the relationship between the reaction-rate $R$ and the reactant concentration $C$ from measurements of the outlet flux of gas. While elegant in conception, its application is hindered by the amplification of measurement noise that results from having to backtrack diffusive transport from the outlet to the reaction zone. Here, we explicitly recognize the inverse problem inherent in the Y-procedure and treat it using well-developed tools from the field of inverse problems. While previous implementations of the Y-procedure used Fourier-based filtering, we do not pre-process the measurements with an ad hoc noise-filter. Instead, we use a basis of localized square pulses to formulate a discrete inverse problem, whose regularized solution is obtained via the truncated singular value decomposition (TSVD) method. This method requires one to select a cutoff mode number; while we show how the choice of this regularization parameter can be guided by a Picard plot, we also develop an objective selection strategy for state defining experiments, for which $R(C)$ is a single-valued function. We apply our proposed inverse-problem approach to synthetic data corresponding to linear and nonlinear reactions and compare the results with the Fourier-filtration method. The former produces better reconstructions of the $R$ vs $C$ relationship, especially for nonlinear reactions. Our work facilitates the automation of pulse response analyses and enables the application of other discrete inverse-problem techniques, such as Tikhonov regularization or machine-learning methods.
MACS: Measurement-Aware Consistency Sampling for Inverse Problems
arXiv:2510.02208v3 Announce Type: replace-cross Abstract: Diffusion models have emerged as powerful generative priors for solving inverse imaging problems. However, their practical deployment is hindered by the substantial computational cost of slow, multi-step sampling. Although Consistency Models (CMs) address this limitation by enabling high-quality generation in only one or a few steps, their direct application to inverse problems has remained largely unexplored. This paper introduces a modified consistency sampling framework specifically designed for inverse problems. The proposed approach regulates the sampler's stochasticity through a measurement-consistency mechanism that leverages the degradation operator, thereby enforcing fidelity to the observed data while preserving the computational efficiency of consistency-based generation. Comprehensive experiments on the Fashion-MNIST and LSUN Bedroom datasets demonstrate consistent improvements across both perceptual and pixel-level metrics, including the Fr\'echet Inception Distance (FID), Kernel Inception Distance (KID), peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM), compared with baseline consistency and diffusion-based sampling methods. The proposed method achieves competitive or superior reconstruction quality with only a small number of sampling steps.
Parallel Spooky Pebbling Makes Regev Factoring More Practical
arXiv:2510.08432v3 Announce Type: replace-cross Abstract: Pebble games, an abstraction from classical reversible computing, have found use in the design of quantum circuits for inherently sequential tasks. Gidney showed that allowing Hadamard basis measurements during pebble games can dramatically improve costs -- an extension termed "spooky pebble games" because the measurements leave temporary phase errors called ghosts. Separately, previous work by Blocki et al. studied the benefits of parallelism in pebble games. In this work we define and study parallel spooky pebble games, showing that parallelism and spookiness can yield impressive gains when used together. First, we show by construction that a line graph of length $\ell$ can be pebbled in depth $2\ell$ (exactly optimal) using space $\leq 2.47\log \ell$. Then, to explore pebbling schemes using even less space, we use a highly optimized $A^*$ search implemented in Julia to find the lowest-depth parallel spooky pebbling possible for a range of concrete line graph lengths $\ell$ given a constant number of pebbles $s$. We then show that these techniques can significantly reduce the cost of the arithmetic in Regev's factoring algorithm. For example, we find that 4096-bit integers $N$ can be factored in multiplication depth 193, which outperforms the 680 required of previous variants of Regev and the 444 reported by Eker{\aa} and G\"artner for Shor's algorithm. While the space required for Shor's algorithm is considerably less than any variant of Regev's algorithm including ours, and thus Shor likely remains the best candidate for the first quantum factorization of large integers, our results show that implementations of Regev's algorithm are far from fully optimized, and Regev's algorithm may have practical importance in the future. We also believe our pebbling techniques are applicable in quantum cryptanalysis beyond integer factorization, and in quantum circuit compilation more broadly.
Linear Response and Optimal Fingerprinting for Nonautonomous Systems
arXiv:2602.08022v3 Announce Type: replace-cross Abstract: We provide a link between response theory, pullback measures, and optimal fingerprinting method that paves the way for a) predicting the impact of acting forcings on time-dependent systems and b) attributing observed anomalies to acting forcings when the reference state is not time-independent. We derive formulas for linear response theory for time-dependent Markov chains and diffusion processes. We discuss existence, uniqueness, and differentiability of the equivariant measure under general (not necessarily slow or periodic) perturbations of the transition kernels. Our results allow for extending the theory of optimal fingerprinting for detection and attribution of climate change (or change in any complex system) when the background state is time-dependent amd when the optimal solution is sought for multiple time slices at the same time. We provide numerical support for the findings by applying our theory to a modified version of the Ghil-Sellers energy balance model. We verify the precision of response theory - even in a coarse-grained setting - in predicting the impact of increasing CO$_2$ concentration on the temperature field. Additionally, we show that the optimal fingerprinting method developed here is capable to attribute the climate change signal to multiple acting forcings across a vast time horizon.
An Adaptive Proximal Framework for Weakly Convex Optimization with Unknown Parameter: Deterministic and Heavy-Tailed Stochastic Guarantees
arXiv:2606.17285v2 Announce Type: replace-cross Abstract: Many nonsmooth, nonconvex objectives in learning and signal recovery are $\rho$-weakly convex. We minimize such a function in deterministic and stochastic settings when the weak-convexity parameter $\rho$ is unknown. The objective is not required to be globally Lipschitz continuous or smooth. We propose the Adaptive Prox-Guided Scheme (APS), a single-trial adaptive proximal framework that adapts the proximal parameter online and bidirectionally through a descent test, allowing it to exploit favorable local structure. The framework covers adaptive proximal-point, prox-linear, and proximal-gradient methods from a model-based viewpoint. In the deterministic proximal setting, APS obtains an $O(\varepsilon^{-2})$ iteration complexity for producing an $\varepsilon$-subgradient stationary point. In the stochastic setting, APS achieves a high-probability $O(\varepsilon^{-2})$ iteration bound for driving the Moreau-envelope gradient below $\varepsilon$. This result holds under deliberately weak oracle assumptions: the function-difference estimates may be biased and heavy-tailed, and the stochastic candidate oracle need only be sufficiently accurate with constant probability when the proximal parameter is small, and can be arbitrary otherwise.
FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning
arXiv:2406.07925v2 Announce Type: replace Abstract: Large language models (LLMs) have emerged as important components across various fields, yet their training requires substantial computation resources and abundant labeled data. It poses a challenge to robustly training LLMs for individual users (clients). To tackle this challenge, the intuitive idea is to introduce federated learning (FL), which can collaboratively train models on distributed private data. However, existing methods suffer from the challenges of data heterogeneity, system heterogeneity, and model size, resulting in suboptimal performance and high costs. In this work, we proposed a variant of personalized federated learning (PFL) framework, namely FDLoRA, which allows the client to be a single device or a cluster and adopts low-rank adaptation (LoRA) tuning. FDLoRA sets dual LoRA modules on each client to capture personalized and global knowledge, respectively, and only the global LoRA module uploads parameters to the central server to aggregate cross-client knowledge. Finally, an adaptive fusion approach is employed to combine the parameters of the dual LoRAs. This enables FDLoRA to make effective use of private data distributed across different clients, thereby improving performance on the client without incurring high communication and computing costs. We conducted extensive experiments in two practice scenarios. The results demonstrate that FDLoRA outperforms six baselines in terms of performance, stability, robustness, computation cost, and communication cost.
AgentDynEx: Nudging the Mechanics and Dynamics of Multi-Agent Simulations
arXiv:2504.09662v4 Announce Type: replace Abstract: Multi-agent large language model simulations have the potential to model complex human behaviors and interactions. If the mechanics are set up properly, unanticipated and valuable social dynamics can surface. However, it is challenging to consistently enforce simulation mechanics while still allowing for rich and emergent dynamics. We present AgentDynEx, an AI system that helps set up, track, and repair simulations. Specifically, AgentDynEx introduces milestones that act as checkpoints and failure conditions that act as guardrails to ensure dynamics are relevant and mechanics are respected as the simulation progresses. It also introduces a method called nudging, where the system dynamically reflects on simulation progress and gently intervenes if it begins to deviate from intended outcomes. A technical evaluation found that nudging enables simulations to progress further without reducing the presence notable dynamics compared to simulations without nudging. A case study with AgentDynEx documented instances where real users were able to simulate lived experiences. We discuss the importance of nudging as a technique for guiding agents towards desirable behaviors while preserving their freedom of choice.
CORA: Generalizable coronary artery disease assessment and risk stratification from coronary CT angiography using pathology-centric representation learning
arXiv:2603.24847v2 Announce Type: replace Abstract: Coronary artery disease, a leading cause of cardiovascular mortality worldwide, can be assessed non-invasively by coronary computed tomography angiography (CCTA). Although deep learning has advanced automated CCTA analysis, clinical translation remains constrained by the scarcity of expert-annotated data and by the spatial sparsity of coronary pathology, which occupies only a small fraction of each scan. Widely used label-free pretraining strategies, such as masked image modeling and contrastive learning, optimize for global anatomical reconstruction and tend to under-represent these tiny localized pathological features. Here we present CORA, an annotation-efficient model for comprehensive coronary artery disease assessment. Rather than reconstructing background anatomy, CORA learns from volumetric CCTA through a synthesis-driven self-supervised strategy: an anatomy-guided engine inserts diverse synthetic calcified and non-calcified lesions into unlabeled scans, reframing pretraining as an abnormality-detection task that biases representation learning toward clinically relevant disease features. We pretrained CORA on 10,138 unlabeled CCTA volumes and evaluated it across datasets from nine independent hospitals. Across plaque characterization, stenosis detection, and coronary artery segmentation, CORA consistently outperformed strong self-supervised pretraining baselines, with the largest gains on external multi-center data, indicating robust generalization under distributional shift. Coupling the imaging encoder with structured clinical variables further enabled near-term major adverse cardiac event (MACE) risk stratification. Our results show that pathology-centric, synthesis-driven pretraining is an effective and scalable strategy for annotation-efficient coronary artery disease assessment from CCTA.
Toward Trustworthy Large Language Model Agents in Healthcare
arXiv:2607.05055v1 Announce Type: new Abstract: Healthcare appointment scheduling remains a persistent operational bottleneck, driven by manual coordination, fragmented legacy systems, and high administrative overhead. These inefficiencies constrain provider availability and degrade patient access to care. This paper presents CareConnect, a safety-first conversational agent for healthcare logistics automation that leverages large language model (LLM) function calling, retrieval-augmented generation (RAG), and layered deterministic safety guardrails. The system orchestrates eight domain-specific tools to support appointment booking, modification, cancellation, and facility information retrieval, while enforcing strict scope constraints that prohibit medical advice or diagnosis. Safety-critical situations are handled through deterministic short-circuit mechanisms for emergency detection and medical intent refusal. We evaluate CareConnect on a comprehensive benchmark of 680 task-oriented scenarios spanning end-to-end workflows, multi-turn interactions, and edge cases. Experimental results demonstrate a 91.8% task completion rate with a median per-request latency of 2.2 seconds, 96.0% safety compliance on the dedicated safety-critical evaluation subset, and an average operational cost of $0.0324 per appointment, yielding a significant cost reduction compared to manual human scheduling. These findings show that carefully scoped and rigorously safeguarded LLM-based agents can reliably automate complex healthcare operational workflows while maintaining safety guarantees and achieving substantial cost efficiency. The source code and system implementation are publicly available at https://github.com/Hadi-Hsn/CareConnect.