Monday, July 13, 2026

Daily Digest 2026-07-13

Today’s research focuses heavily on enhancing the reliability, planning capabilities, and architectural efficiency of autonomous LLM agents for complex, long-horizon tasks.

Research highlights:

  • Agentic Planning and Reasoning: New methods utilize graph-augmented tree searches, auction-based task allocation, and reflective multi-agent frameworks to improve how agents decompose and execute complex goals.
  • Reliability and Verification: Researchers are developing formal abstractions for inference-time control, scoped verification for context evolution, and risk-tiering frameworks to ensure safer agent behavior.
  • Memory and Inference Efficiency: Innovations include KV-cache transfer for process reward modeling, shared selective persistent memory, and specialized routing techniques for memory-efficient Mixture-of-Experts (MoE) inference.
  • Embodied and Scientific AI: Work explores communication-efficient coordination for heterogeneous embodied agents and protocols for LLM agents to function as auditable scientific researchers.
  • Reinforcement Learning: New studies address multimodal reward hacking and the development of dense reward-based grading for testing long-horizon terminal tasks.

Tech buzz:

  • The ecosystem is seeing a surge in production-ready frameworks and curated resources for building agentic applications.
  • Agent Frameworks: New libraries are emerging to simplify background agent execution and structured AI interactions using Pydantic.
  • Educational Resources: Updated repositories are providing structured paths for hands-on deep learning and LLM application development.
Sort:
Today's research focuses heavily on enhancing the reliability, planning capabilities, and architectural efficiency of autonomous LLM agents for complex, long-horizon tasks.

Papers discovered from ArXiv subject categories

AI Safety

5/5 Artificial Intelligence (cs.AI) 13 Jul 2026
TrustX Agent Risk Classification Framework (ARC): Risk-Tiering Internally Created Agentic AI Systems

Hannah M. Liu, Rhea Saxena, Shiv Asthana

Abstract

ArXiv ID: 2607.09586

Authors: Hannah M. Liu, Rhea Saxena, Shiv Asthana

Abstract:

The proliferation of agentic AI systems across enterprise and public-sector contexts has outpaced the capacity of general-purpose AI risk frameworks to classify and govern them. In this paper, we introduce the TrustX Agent Risk Classification Framework, a structured, repeatable instrument that can be applied to seven types of agentic AI systems and is grounded in foundational pre-existing AI governance frameworks. At the core of the framework is a twelve-dimension scoring rubric that robustly quantifies the risk. This rubric is combined with other components, such as the GPA + IAT classification model and the five-level autonomy framework derived from existing literature. These inputs produce a three-tier governance output with mapped control recommendations. A specialised Coding Assistant extension is also included to account for nuances specific to this type of agentic AI system. We then use an illustrative example to show our framework in practice. ARC is intended for AI governance practitioners, risk officers, developers, and regulators, and it will regularly undergo iteration as we continue to expand it and make it more robust. The community can access the interactive framework here: https://arc.responsible.ai/

Insights

Contribution: The paper introduces the TrustX Agent Risk Classification Framework (ARC), a structured and repeatable instrument designed to classify and govern the risks of internally created agentic AI systems.

Core Idea: General-purpose AI risk frameworks are insufficient for agentic systems; therefore, a specialized framework is needed to quantify risk based on autonomy, capability, and specific use cases.

Technique: The framework utilizes a twelve-dimension scoring rubric combined with a GPA + IAT classification model and a five-level autonomy framework.

Pipeline: Agentic AI system characteristics β†’ GPA + IAT classification and 12-dimension risk scoring β†’ Three-tier governance output with mapped control recommendations.

Methodology: The authors synthesized existing AI governance literature to create a multi-dimensional rubric and validated the framework through an illustrative practical example.

Results: The framework successfully produces a three-tier risk classification and provides specific, actionable control recommendations for different types of agentic AI.

Limitations: The framework is currently in an iterative stage and requires ongoing expansion to account for new types of agentic behaviors and evolving regulatory requirements.

PDF
5/5 Machine Learning (cs.LG) 13 Jul 2026
Optimizing Against Safety Representations: Activation-Guided Adversarial Suffixes and the Geometry of Refusal

Ege \c{C}akar, Hannah Guan, Kayden Kehe

Abstract

ArXiv ID: 2607.08883

Authors: Ege \c{C}akar, Hannah Guan, Kayden Kehe

Abstract:

Behavioral alignment in large language models often masks fragile internal safety representations. Recent work suggests that refusal behavior is mediated by low-dimensional directions in activation space. This raises questions about how such representations are structured, localized, and accessed by optimization. We study adversarial suffix attacks as a probe of representational alignment. We introduce Activation-Guided GCG, which replaces output-based objectives with losses that directly target a model's internal refusal direction. Across several objective variants, we find that suppressing refusal globally across all layers and positions is more effective than targeting a single layer-position pair. This suggests that safety representations are distributed across the forward pass rather than causally localized to a single site. We further introduce Soft-GCG, a continuous relaxation of discrete suffix optimization using Gumbel-Softmax. Soft-GCG achieves a 33 $\times$ speedup over standard GCG while improving attack success rates. Evaluating across model scales, we find that smaller models remain vulnerable while larger models resist both activation- and suffix-based attacks at our compute-constrained settings, consistent with larger and better safety trained models being harder to jailbreak. Together, our results clarify how safety mechanisms are encoded and can be broken in contemporary models. These insights provide concrete guidance for designing more robust and representation-aware alignment strategies.

Insights

Contribution: The paper introduces Activation-Guided GCG and Soft-GCG to probe and bypass internal safety representations in LLMs. It demonstrates that safety mechanisms are distributed across the forward pass rather than localized to specific layers.

Core Idea: Instead of optimizing for output-based refusal, the authors target the model's internal 'refusal direction' in activation space to more effectively bypass safety filters.

Technique: The authors develop Activation-Guided GCG, which uses internal activation losses, and Soft-GCG, a continuous relaxation of discrete suffix optimization using Gumbel-Softmax.

Pipeline: Input prompt β†’ Activation-Guided/Soft-GCG optimization β†’ Adversarial suffix generation β†’ Jailbroken model output

Methodology: The researchers compared output-based GCG against activation-based objectives across various layers and positions, while also benchmarking a continuous relaxation method for efficiency.

Results: Activation-Guided GCG is more effective when targeting global representations; Soft-GCG achieves a 33x speedup over standard GCG with higher success rates; larger models show increased resistance to both attack types.

Limitations: The study is constrained by compute settings where larger models remained resistant, suggesting a need for more intensive optimization to break high-scale safety training.

PDF

Agentic AI

5/5 Artificial Intelligence (cs.AI)Machine Learning (cs.LG) 13 Jul 2026
GATS: Graph-Augmented Tree Search with Layered World Models for Efficient Agent Planning

Maureese Williams, Dymitr Nowicki

Abstract

ArXiv ID: 2607.08894

Authors: Maureese Williams, Dymitr Nowicki

Abstract:

Large Language Model (LLM) agents have shown promise in multi-step planning tasks, but existing approaches like LATS (Language Agent Tree Search) and ReAct rely heavily on LLM inference during planning, leading to high computational costs and stochastic behavior. We present \textbf{GATS} (Graph-Augmented Tree Search), a planning framework that combines systematic UCB1-based tree search with a layered world model to eliminate LLM calls during inference while achieving superior planning performance. Our three-layer world model integrates: (L1) exact symbolic action matching, (L2) statistics learned from execution logs, and (L3) LLM-based prediction for unknown actions. On synthetic planning tasks with branching paths and dead-ends, GATS achieves \textbf{100\% success rate} compared to 92 % for LATS and 64\% for ReAct. On a comprehensive stress test spanning 12 challenging scenarios -- including coding workflows, web navigation, and long-horizon tasks -- GATS maintains \textbf{100\% success} while LATS drops to 88.9 % and ReAct to 23.9%. GATS requires \textbf{zero LLM calls per task} during planning (vs. 37 per task for LATS) and produces deterministic plans with zero variance across runs. Our results demonstrate that systematic search with learned world models can substantially outperform LLM-guided exploration for agent planning.

Insights

Contribution: The paper introduces GATS, a planning framework that achieves superior performance and deterministic behavior by replacing real-time LLM inference with a systematic tree search guided by a layered world model.

Core Idea: By pre-modeling the environment's dynamics into a layered world model, agents can perform exhaustive planning using UCB1-based tree search without requiring any LLM calls during the inference phase.

Technique: The framework utilizes a three-layer world model consisting of exact symbolic matching, execution log statistics, and LLM-based predictions for unknown actions to guide a UCB1-based tree search.

Pipeline: Task environment β†’ Layered World Model (Symbolic/Statistical/LLM) β†’ UCB1-based Tree Search β†’ Deterministic Plan

Methodology: The authors compared GATS against LATS and ReAct on synthetic branching tasks and a 12-scenario stress test involving coding, web navigation, and long-horizon planning.

Results: GATS achieved a 100% success rate across all tests, significantly outperforming LATS (88.9%) and ReAct (23.9%), while requiring zero LLM calls per task during planning.

Limitations: The reliance on a pre-built world model may limit the agent's ability to adapt to entirely novel environments that were not captured during the model's construction or log collection.

PDF
5/5 Artificial Intelligence (cs.AI) 13 Jul 2026
Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading

Zongxia Li, Zhongzhi Li, Yucheng Shi, Ruhan Wang, Junyao Yang, Zhichao Liu, Xiyang Wu, Anhao Li, Yue Yu, Ninghao Liu, Lichao Sun, Haotao Mi, LeoweiLiang

Abstract

ArXiv ID: 2607.08964

Authors: Zongxia Li, Zhongzhi Li, Yucheng Shi, Ruhan Wang, Junyao Yang, Zhichao Liu, Xiyang Wu, Anhao Li, Yue Yu, Ninghao Liu, Lichao Sun, Haotao Mi, LeoweiLiang

Abstract:

AI agents have become capable of autonomously completing short, well-specified tasks. However, existing terminal benchmarks largely focus on simple problems that finish within minutes and are evaluated only by their final outcome. This setup overlooks intermediate progress and partial solutions, yielding sparse reward signals and an incomplete picture of agent capability. We introduce Long-Horizon-Terminal-Bench, a terminal benchmark of 46 long-horizon tasks spanning nine categories, including experiment reproduction, software engineering, multimodal analysis, interactive games, and scientific computing. Each task follows a Terminal-Bench-style setup with a reference solution or simulation engine, but is further decomposed into fine-grained graded subtasks. This design enables dense intermediate rewards and partial credit, allowing evaluation to capture not only whether an agent reaches the final goal, but also how far it progresses on open-ended workflows. Tasks in Long-Horizon-Terminal-Bench typically require hundreds of episodes and minutes to hours of execution, stressing long-horizon planning, long-context management, and iterative debugging rather than one-shot problem solving. We evaluate 15 frontier models and find that agents consume on average 9.9M tokens per task, with roughly 231 episodes and 85.3 minutes of execution time per run, making Long-Horizon-Terminal-Bench more demanding than prior terminal-based benchmarks. Even the strongest tested model achieves 15.2% pass@1 at a partial-reward threshold of 0.95 and 10.9% at a perfect-reward threshold of 1.0, while the mean pass rate across models is 4.3% and 1.7% under the two thresholds, respectively. These results reveal headroom for improvement. We further analyze failure modes and error patterns, and release Long-Horizon-Terminal-Bench to support future progress on long-horizon terminal agents.

Insights

Contribution: The paper introduces Long-Horizon-Terminal-Bench, a new benchmark designed to evaluate AI agents on complex, long-duration tasks using dense reward-based grading to capture intermediate progress.

Core Idea: Existing benchmarks focus on short tasks with sparse rewards; this work addresses the need to evaluate long-horizon planning, context management, and iterative debugging through fine-grained subtask decomposition.

Technique: The authors decompose 46 long-horizon tasks into graded subtasks, allowing for partial credit and dense reward signals instead of binary success/failure outcomes.

Pipeline: Long-horizon task description β†’ Decomposition into graded subtasks β†’ Agent execution with simulation/reference feedback β†’ Dense reward calculation β†’ Final performance evaluation.

Methodology: The researchers curated tasks across nine categories (e.g., software engineering, scientific computing) and evaluated 15 frontier models based on their ability to reach specific partial-reward thresholds.

Results: Agents consumed an average of 9.9M tokens and 85.3 minutes per task; the strongest model achieved only 15.2% pass@1 at a 0.95 partial-reward threshold and 10.9% at a 1.0 perfect-reward threshold.

Limitations: The low pass rates across all models highlight significant remaining headroom for improvement in long-horizon planning and iterative error correction.

PDF
5/5 Artificial Intelligence (cs.AI) 13 Jul 2026
ARCANA: A Reflective Multi-Agent Program Synthesis Framework for ARC-AGI-2 Reasoning

Kunbo Zhang, Lei Fu, Zeyu Wang, Zijing Liu, Kejian Tong

Abstract

ArXiv ID: 2607.09059

Authors: Kunbo Zhang, Lei Fu, Zeyu Wang, Zijing Liu, Kejian Tong

Abstract:

We present ARCANA, a collaborative multi agent framework for solving ARC AGI 2 tasks under strict test time and hardware constraints. ARCANA decomposes each task into iterative perception, hypothesis generation, symbolic execution, and reflective refinement. A perceptual grounding agent builds object centric scene graphs from raw grids, a latent program policy proposes diverse DSL programs, a symbolic executor verifies candidates on demonstrations, and a reflective agent synthesizes failure driven feedback for the next turn. These agents communicate through a shared differentiable blackboard and are scheduled by a learned meta controller. The design combines structured program search with adaptive multi turn correction, improving reasoning efficiency and solution quality on challenging abstract transformation tasks.

Insights

Contribution: The paper introduces ARCANA, a collaborative multi-agent framework designed to solve complex ARC-AGI-2 tasks by combining structured program synthesis with iterative reflective refinement.

Core Idea: The framework decomposes abstract reasoning into a multi-turn cycle of perception, hypothesis generation, symbolic execution, and feedback-driven correction.

Technique: ARCANA utilizes a multi-agent system communicating via a shared differentiable blackboard, coordinated by a learned meta-controller to manage task decomposition and agent scheduling.

Pipeline: Raw grids β†’ Perceptual grounding (scene graphs) β†’ Latent program policy (DSL candidates) β†’ Symbolic execution (verification) β†’ Reflective refinement (feedback) β†’ Final solution

Methodology: The system employs four specialized agentsβ€”Perceptual Grounding, Latent Program Policy, Symbolic Executor, and Reflective Agentβ€”to iteratively refine programs based on failure-driven feedback.

Results: Improved reasoning efficiency and solution quality on challenging abstract transformation tasks under strict test-time and hardware constraints.

Limitations: The paper does not specify exact quantitative benchmarks or the specific hardware constraints encountered during testing.

PDF
5/5 Artificial Intelligence (cs.AI) 13 Jul 2026
KV-PRM: Efficient Process Reward Modeling via KV-Cache Transfer for Multi-Agent Test-Time Scaling

Peng Kuang, Haibo Jin, Xiaoyu Han, Yanli Wang, Xiaopeng Yuan, Ye Yu, Kaidi Xu, Haohan Wang

Abstract

ArXiv ID: 2607.09153

Authors: Peng Kuang, Haibo Jin, Xiaoyu Han, Yanli Wang, Xiaopeng Yuan, Ye Yu, Kaidi Xu, Haohan Wang

Abstract:

Process Reward Models (PRMs) have been proven to be highly effective in guiding test-time scaling (TTS) methods, which significantly boost the capabilities of LLM-based multi-agent systems. However, existing PRMs are text-based: they re-encode the entire trajectory text from scratch. In long multi-agent rollouts, the scoring cost, growing quadratically with respect to sequence length L, creates a severe computational bottleneck, severely limiting PRMs' application in long-context scenarios. To resolve this, we introduce KV-PRM, a highly efficient process reward model that eliminates the heavy text re-encoding by directly reading the KV cache produced naturally during the LLM's generation phase. By processing a single "verify token" against the pre-existing KV cache, KV-PRM reduces the scoring cost from O(L^2) to O(L). We formally prove that the KV cache contains strictly greater information capacity than text, and is more efficient for downstream reward modeling. Empirically, across the MATH, GSM8K, and AIME benchmarks, KV-PRM matches or strictly outperforms text-PRMs under various TTS methods such as Beam Search, MCTS, and Weighted Voting, with up to a 5,000x reduction in scoring FLOPs, a 37x reduction in latency, and a 34x reduction in per-sequence memory footprint compared to text-based PRMs.

Insights

Contribution: The paper introduces KV-PRM, an efficient Process Reward Model (PRM) that significantly reduces computational overhead by leveraging KV-cache transfer instead of re-encoding trajectory text.

Core Idea: By directly reading the KV cache produced during LLM generation, the model eliminates redundant text re-encoding, reducing scoring complexity from quadratic to linear relative to sequence length.

Technique: The method utilizes a 'verify token' processed against pre-existing KV caches to derive rewards, formally proving that KV caches hold greater information capacity than raw text for reward modeling.

Pipeline: LLM Generation (KV-cache) β†’ KV-PRM (Verify Token + KV-cache Transfer) β†’ Process Reward Score

Methodology: The authors developed a framework to extract rewards from KV caches and evaluated it across MATH, GSM8K, and AIME benchmarks using Beam Search, MCTS, and Weighted Voting.

Results: KV-PRM matches or outperforms text-PRMs with up to a 5,000x reduction in scoring FLOPs, 37x lower latency, and 34x lower per-sequence memory footprint.

Limitations: The paper does not explicitly detail the potential impact of KV-cache quantization or the specific hardware constraints for extremely long-context multi-agent rollouts.

PDF
5/5 Artificial Intelligence (cs.AI)Computation and Language (cs.CL) 13 Jul 2026
Scoped Verification for Reliable Long-Horizon Agentic Context Evolution under Distribution Shift

Dan C. Hsu, Luke Lu

Abstract

ArXiv ID: 2607.09175

Authors: Dan C. Hsu, Luke Lu

Abstract:

Deployed LLM agents rely on agentic context, the model-external textual control content assembled by an operational harness. In this work, the mutable component of that context is a persistent system-level instruction that is updated from operational experience while the model, tools, and harness remain fixed. Over long evolution horizons, flat-text maintenance makes verification increasingly difficult as accumulated instructions grow and interact. We propose Graph-Regularized Agentic Context Evolution (GRACE), which maintains the persistent instruction component as a typed semantic graph and validates proposed updates within the local typed neighborhoods of modified nodes. Accepted graph updates are reconstructed as incremental edits to the textual instruction checkpoint used at deployment. We evaluate GRACE within a fixed telecom agent harness derived from $\tau^2$-bench under a controlled distribution-shift protocol. Across five independent replications, GRACE improves strict reliability, measured by pass^3, from the Gemini 2.5 Flash zero-shot value of 0.091 to 0.673$\pm$0.136 at the final checkpoint. This exceeds a Gemini 3.1 Pro zero-shot reference of 0.242 on the same held-out set, while the flat-text HCE baseline finishes at 0.191$\pm$0.051. These results identify two requirements for reliable long-horizon context evolution, a structural substrate that makes verification local and a consolidation mechanism that keeps accumulated instruction content usable.

Insights

Contribution: The paper introduces GRACE, a framework for managing long-horizon agentic context evolution by replacing flat-text instructions with a typed semantic graph to enable localized verification.

Core Idea: To prevent instruction degradation and complexity in long-running agents, context updates should be structured as a graph where changes can be validated within local neighborhoods rather than across the entire instruction set.

Technique: Graph-Regularized Agentic Context Evolution (GRACE) uses a typed semantic graph to represent system instructions and performs scoped verification on proposed updates before reconstructing them into incremental textual edits.

Pipeline: Operational experience β†’ Proposed graph update β†’ Localized neighborhood verification β†’ Accepted graph update β†’ Incremental textual instruction reconstruction β†’ Deployed agent context

Methodology: The authors evaluated GRACE against a flat-text baseline using a fixed telecom agent harness from $\tau^2$-bench under a controlled distribution-shift protocol.

Results: GRACE improved strict reliability (pass^3) from 0.091 to 0.673\pm0.136, significantly outperforming both the flat-text baseline (0.191\pm0.051) and a Gemini 3.1 Pro zero-shot reference (0.242).

Limitations: The study focuses on a specific telecom agent harness and does not explore the scalability of the graph structure across vastly different domains or the computational overhead of graph reconstruction.

PDF
5/5 Artificial Intelligence (cs.AI)cond-mat.mtrl-sci 13 Jul 2026
Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents

Izumi Takahara, Teruyasu Mizoguchi

Abstract

ArXiv ID: 2607.09195

Authors: Izumi Takahara, Teruyasu Mizoguchi

Abstract:

Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery. Equipped with broad knowledge, flexible reasoning, and tool use, they have the potential to autonomously explore and solve scientific problems by repeatedly proposing hypotheses, testing them, and revising their beliefs in the light of the evidence. In current agents, however, these hypotheses, tests, and belief updates are buried in unstructured logs, and no mechanism lets the agent or the human researcher audit that process. Here we propose the Hypothesis Evolution Protocol (HEP), an agent harness that provides hypothesis generation, evaluation, and evolution as explicit, auditable operations. On materials-science research tasks, a HEP-equipped agent operates the hypothesis--test--evidence--belief cycle that planning-style agents lack, generalizes across research questions, and exploits the protocol more fully as the base LLM becomes more capable. These results mark a step toward auditable AI scientists, whose scientific reasoning can be inspected, verified, and built upon.

Insights

Contribution: The paper introduces the Hypothesis Evolution Protocol (HEP), a framework that transforms the opaque reasoning of LLM agents into an explicit, auditable cycle of hypothesis generation, testing, and belief revision.

Core Idea: To move beyond planning-style agents by creating a structured 'hypothesis-test-evidence-belief' cycle that allows human researchers to inspect and verify the scientific reasoning process.

Technique: The authors developed a structured agent harness that enforces explicit operations for hypothesis evolution, ensuring that every step of the scientific discovery process is logged and auditable.

Pipeline: Scientific research question β†’ Hypothesis generation β†’ Experimental testing β†’ Evidence collection β†’ Belief revision β†’ Refined hypothesis/conclusion

Methodology: The researchers evaluated the HEP-equipped agent on materials science research tasks, comparing its performance and auditability against standard planning-style agents.

Results: The HEP-equipped agent successfully operated the full scientific cycle, demonstrated generalization across different research questions, and showed improved performance as the underlying LLM's capabilities increased.

Limitations: The study focuses on materials science and does not fully explore the protocol's performance across all scientific domains or its scalability to extremely long-term research projects.

PDF
5/5 Artificial Intelligence (cs.AI) 13 Jul 2026
ProofCouncil: An LLM Agent for Solving Open Mathematical Problems

Johannes Schmitt, Tim Gehrunger, Jasper Dekoninck, Gergely B\'erczi, Uri Kreitner, Liam Price, David Holmes

Abstract

ArXiv ID: 2607.09474

Authors: Johannes Schmitt, Tim Gehrunger, Jasper Dekoninck, Gergely B\'erczi, Uri Kreitner, Liam Price, David Holmes

Abstract:

Large language models (LLMs) have shown increasing promise in solving open problems in mathematics. However, their performance can be further improved through agentic workflows tailored to real-world mathematical practice. To this end, we introduce ProofCouncil, a mathematical agent that is designed to tackle open problems using an author-critic architecture. ProofCouncil served as a submission to the second batch of FirstProof, a challenge consisting of 10 real-world mathematical problems that agents must solve autonomously. Its submissions for 6 of the 10 problems were judged by the referees to be correct up to at most minor revisions, showing the best performance among participating teams. We also evaluate ProofCouncil on 30 open problems collected from mathematical researchers. Among the 21 solutions that received human feedback, 5 were judged completely correct, 2 more were judged promising pending final verification, and a further 8 contained useful partial progress. In this short paper, we describe the development of ProofCouncil and the agent-building library used to create it, which we release as open source to the community.

Insights

Contribution: The paper introduces ProofCouncil, an LLM agent designed to solve open mathematical problems using an author-critic architecture, and releases an open-source agent-building library.

Core Idea: Mathematical reasoning can be significantly improved by mimicking real-world mathematical practice through an agentic workflow that separates the generation of proofs from their critical evaluation.

Technique: The system employs an author-critic architecture where one agent proposes mathematical steps and another critiques and refines them to ensure logical consistency.

Pipeline: Open mathematical problem β†’ Author-critic agentic workflow β†’ Verified mathematical proof

Methodology: The authors developed ProofCouncil and tested it on the FirstProof challenge and a set of 30 open problems collected from mathematical researchers.

Results: ProofCouncil achieved the best performance in the FirstProof challenge (6/10 problems correct) and produced 5 completely correct solutions, 2 promising solutions, and 8 partial progress results on 30 open problems.

Limitations: The paper is a short report and does not exhaustively detail every edge case of the agent's reasoning or the full scope of the open-source library's capabilities.

PDF
5/5 Artificial Intelligence (cs.AI)Multiagent Systems (cs.MA)Software Engineering (cs.SE) 13 Jul 2026
Shared Selective Persistent Memory for Agentic LLM Systems

Sanjana Pedada, Aditya Dhavala, Neelraj Patil

Abstract

ArXiv ID: 2607.09493

Authors: Sanjana Pedada, Aditya Dhavala, Neelraj Patil

Abstract:

Agentic LLM systems that generate code through multi-turn tool use face a fundamental context problem: each session starts from zero, discarding the configuration choices, domain constraints, data schemas, and tool-use patterns that made previous sessions productive. Naively persisting entire conversation histories is token-inefficient and counterproductive: irrelevant context degrades generation quality. We introduce shared selective persistent memory, an architecture that identifies and retains four categories of reusable context (task specifications, data schemas, tool configurations, and output constraints) while discarding session-specific reasoning traces. Crucially, this memory is shared: workspaces encapsulating selective memory can be transferred across users with role-based access control, enabling collaborative reuse without redundant specification. We implement it in a deployed collaborative workspace platform where LLM agents produce, edit, and maintain git-versioned artifacts (dashboards, reports, and data-driven documents) from heterogeneous sources (CSV, SQL, REST APIs, and MCP servers). A complementary zero-token data refresh mechanism decouples generated programs from runtime data, enabling artifact reuse without re-invocation. Across three enterprise scenarios, shared selective persistent memory achieves 96% task completion (vs. 79% without memory and 71% with full history). Zero-token refresh eliminates LLM re-invocation for recurring updates (14x task-time reduction), while summary-driven generation cuts per-invocation token cost by 97x versus raw data injection. A replication on four public datasets confirms generalizability, with zero-token refresh succeeding in 12/12 trials. Notably, naive full-history persistence actively degrades completion by biasing the agent with stale traces, while selective memory outperforms both extremes.

Insights

Contribution: The paper introduces a shared selective persistent memory architecture for agentic LLM systems that retains reusable context while discarding irrelevant reasoning traces to improve task completion and efficiency.

Core Idea: Instead of persisting entire conversation histories, the system selectively stores four categories of reusable context (task specs, schemas, tool configs, and output constraints) across shared workspaces.

Technique: The authors implement a selective memory mechanism combined with a zero-token data refresh system that decouples generated programs from runtime data.

Pipeline: User request and raw data β†’ Selective memory retrieval (schemas, configs, constraints) β†’ LLM agent code generation β†’ Zero-token data refresh for updates β†’ Git-versioned artifacts

Methodology: The researchers deployed a collaborative workspace platform to test the system across three enterprise scenarios and replicated the findings on four public datasets.

Results: Achieved 96% task completion (vs. 79% without memory and 71% with full history), a 14x reduction in task time for updates via zero-token refresh, and a 97x reduction in per-invocation token costs.

Limitations: The paper does not explicitly detail the specific criteria for automated selection of memory vs. reasoning traces or the scalability of role-based access control in extremely large-scale multi-user environments.

PDF
5/5 Artificial Intelligence (cs.AI)Computation and Language (cs.CL) 13 Jul 2026
Agora: Enhancing LLM Agent Reasoning Via Auction-Based Task Allocation

Kaiji Zhou, Ales Leonardis, Yue Feng

Abstract

ArXiv ID: 2607.09600

Authors: Kaiji Zhou, Ales Leonardis, Yue Feng

Abstract:

Enhancing the reasoning capabilities of large language model (LLM) agents requires effective orchestration of diverse expert models and tools. However, existing frameworks typically call APIs based on coarse-grained matching between tasks and the functions of expert models or tools, while overlooking critical factors such as performance variability and cost efficiency among functionally similar alternatives. To address this, we propose Agora, a framework that introduces an incentive-compatible auction mechanism for dynamically allocating tasks to expert models and tools. By treating reasoning steps as tradeable items, Agora enables agents to bid based on their rectified competence-ensuring that critical logic is routed to the most capable solver rather than the most overconfident one. Evaluations across five benchmarks show that Agora improves over matched single-model, routing, and cascade baselines under comparable candidate pools, while exposing a controllable cost-quality trade-off through a single auction parameter.

Insights

Contribution: The paper introduces Agora, a framework that improves LLM agent reasoning by replacing coarse-grained task matching with an incentive-compatible auction mechanism for dynamic task allocation.

Core Idea: By treating reasoning steps as tradeable items, the framework ensures that tasks are routed to the most competent solver rather than the most overconfident one by accounting for performance variability and cost.

Technique: The authors implement an auction-based mechanism where expert models and tools bid on tasks based on their rectified competence.

Pipeline: Complex task β†’ Decomposition into reasoning steps β†’ Auction-based bidding by expert models/tools β†’ Task allocation based on rectified competence β†’ Final reasoning output

Methodology: The researchers developed an incentive-compatible auction system and evaluated it against single-model, routing, and cascade baselines across five different benchmarks.

Results: Agora outperformed all baselines across five benchmarks and provided a controllable cost-quality trade-off via a single auction parameter.

Limitations: The paper does not explicitly detail the computational overhead of the auction process or the scalability of the mechanism as the number of expert models grows significantly.

PDF
5/5 Machine Learning (cs.LG)Artificial Intelligence (cs.AI) 13 Jul 2026
Prompt-Driven Exploration

Sunshine Jiang, John Marangola, David Zhang, Raghuram Kowdeed, Ruiyang Luo, Nitish Dashora, Richard Li, Pulkit Agrawal, Zhang-Wei Hong

Abstract

ArXiv ID: 2607.08837

Authors: Sunshine Jiang, John Marangola, David Zhang, Raghuram Kowdeed, Ruiyang Luo, Nitish Dashora, Richard Li, Pulkit Agrawal, Zhang-Wei Hong

Abstract:

Exploration is essential to RL since a policy cannot improve by repeatedly sampling the behaviors it already prefers. Standard methods inject stochasticity in the action space, but such jitter only yields rollouts close to the original. Escaping a weak policy often requires global perturbations that action noise cannot produce. Large language models (LLMs) and vision-language-action (VLA) models offer a pathway: they condition the policy on a natural language prompt, and since the rollout follows from it, modifying the prompt induces global changes. The challenge is finding prompts that induce useful global changes. With a weak policy that rarely succeeds, reward is too sparse to select on. Our idea is to refine prompts from the rollouts themselves: a vision-language model (VLM) reasons over the rollout video, diagnoses how the policy responded, and rewrites the prompt to elicit better behavior next time. This procedure realizes posterior sampling, a classical RL exploration framework, at the level of prompts: the VLM maintains an implicit distribution over useful prompts and updates it from observed rollouts. We call this strategy Prompt-Driven Exploration (PDE). Across manipulation and reasoning tasks, PDE enables RL to learn successful policies even from zero-reward starts, and improves sample efficiency more broadly. Our website is available at https://xinyunsunshine.github.io/prompt-rl.

Insights

Contribution: The paper introduces Prompt-Driven Exploration (PDE), a framework that uses Large Language Models (LLMs) and Vision-Language Models (VLMs) to perform global exploration in Reinforcement Learning by refining natural language prompts.

Core Idea: Instead of injecting local noise into action spaces, PDE treats the prompt as a high-level exploration variable, using a VLM to diagnose failed rollouts and rewrite prompts to elicit better behaviors.

Technique: The method implements a form of posterior sampling at the prompt level, where a VLM maintains and updates an implicit distribution over useful prompts based on observed rollout videos.

Pipeline: Rollout video β†’ VLM reasoning and diagnosis β†’ Prompt rewriting β†’ New policy execution β†’ Reward feedback

Methodology: The authors use a VLM to analyze video data of a weak policy's performance, identifying why it failed and generating a modified prompt to guide the next attempt, effectively bypassing sparse reward issues.

Results: PDE enables RL to learn successful policies from zero-reward starts and improves sample efficiency across both manipulation and reasoning tasks.

Limitations: The effectiveness depends on the VLM's ability to accurately diagnose failures and the diversity of the prompt space it can generate.

PDF
5/5 Machine Learning (cs.LG)Artificial Intelligence (cs.AI) 13 Jul 2026
Eluna: An Agentic LLM System for Automating Warehouse Operations with Reasoning and Task Execution

Ning Liu, Kalle Kujanp\"a\"a, Zhaoxuan Zhu, P Aditya Sreekar, Kaiwen Liu, Chuanneng Sun, Jorge Marchena Menendez, Matthew Bales, Tianyu Yang, Shahnawaz Alam, Rose Yu, Baoyuan Liu, Kristina Klinkner, Shervin Malmasi

Abstract

ArXiv ID: 2607.08960

Authors: Ning Liu, Kalle Kujanp\"a\"a, Zhaoxuan Zhu, P Aditya Sreekar, Kaiwen Liu, Chuanneng Sun, Jorge Marchena Menendez, Matthew Bales, Tianyu Yang, Shahnawaz Alam, Rose Yu, Baoyuan Liu, Kristina Klinkner, Shervin Malmasi

Abstract:

Warehouse operations are governed by Standard Operating Procedures (SOPs) that encode complex, multi-system decision logic, which must be executed reliably under strict time constraints, yet LLM agents lack mechanisms to enforce procedural compliance and degrade under the context overload full SOP specifications introduce. We present Eluna, a production-deployed agentic system for reliable SOP execution. Eluna is a graph-guided, multi-agent framework that encodes SOPs as directed acyclic graphs with progressive disclosure and delegates independent tasks to parallel sub-agents, each with persistent code execution and live data access. To meet production latency and accuracy needs, we use asymmetric episodic distillation where a strong teacher is improved through episodic error memories, then a smaller student is fine-tuned on the corrected trajectories with memory stripped, internalizing corrections without inference-time overhead. On a 13-task benchmark and two production applications, our fine-tuned models match or exceed their teacher, beat all larger off-the-shelf baselines, and reach 94% expert agreement on the ticket processing application.

Insights

Contribution: The paper introduces Eluna, a production-deployed multi-agent framework designed to automate complex warehouse operations by ensuring strict adherence to Standard Operating Procedures (SOPs).

Core Idea: Eluna addresses LLM limitations in procedural compliance and context overload by encoding SOPs as directed acyclic graphs and using asymmetric episodic distillation to train efficient, high-accuracy student models.

Technique: The system utilizes a graph-guided multi-agent architecture with progressive disclosure and a distillation process that internalizes error corrections into a smaller model's weights.

Pipeline: SOP-based warehouse tasks β†’ Graph-guided decomposition and parallel sub-agent execution with live data access β†’ Automated task completion with high expert agreement.

Methodology: SOPs are modeled as directed acyclic graphs; a strong teacher model is refined via episodic error memories, and a smaller student model is fine-tuned on these corrected trajectories.

Results: The fine-tuned models matched or exceeded the teacher's performance, outperformed larger off-the-shelf baselines, and achieved 94% expert agreement on ticket processing.

Limitations: The abstract does not explicitly state limitations, but implies challenges in managing context overload and meeting strict production latency/accuracy requirements.

PDF

Computing Systems

5/5 Machine Learning (cs.LG)Artificial Intelligence (cs.AI) 13 Jul 2026
Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement

Qianli Liu, Kaibin Guo, Zicong Hong, Peng Li, Fahao Chen, Haodong Wang, Jian Lin, Song Guo

Abstract

ArXiv ID: 2607.08782

Authors: Qianli Liu, Kaibin Guo, Zicong Hong, Peng Li, Fahao Chen, Haodong Wang, Jian Lin, Song Guo

Abstract:

Expert parallelism has become the prevailing paradigm to serve Mixture-of-Experts (MoE) models. Its efficiency depends on the communication and computation latencies of the GPUs, which are linked to the placement of experts in the GPUs. Existing works for optimizing expert placement focus on leveraging past requests' expert activation patterns. However, they demonstrate deficiencies facing diverse and rapidly changing request patterns, calling for an online, proactive approach. Implementing such an approach requires addressing several challenges: the uncertainty associated with incoming requests' expert activation, the cost of expert migration, and the NP-hard complexity in optimization. Therefore, we present Director, a new distributed MoE serving system that minimizes end-to-end latency via prediction-driven, online expert placement. Director uses either a lightweight cascaded predictor or a low-bit quantized replica for expert activation patterns of incoming requests. An online migration module then enacts the changes with near-zero downtime by executing migrations in compute-bound phases, keeping disruption bounded. At its core, a relaxation-based expert placement optimizer operates under capacity constraints, runs in polynomial time, and achieves a $(1+\epsilon)$ approximation ratio. Finally, we implement a prototype and demonstrate, through extensive experiments, a reduction in end-to-end latency of $11\sim55\%$ for popular MoE models (e.g., Mistral, DeepSeek and Qwen) compared to existing work.

Insights

Contribution: The paper introduces Director, a distributed MoE serving system that minimizes end-to-end latency by proactively migrating experts based on predicted request patterns.

Core Idea: Instead of relying on static or historical expert placement, Director uses online predictions to dynamically relocate experts to GPUs before requests arrive, balancing migration costs with latency gains.

Technique: The system employs a prediction-driven migration framework featuring a cascaded predictor/quantized replica for activation patterns and a relaxation-based optimizer for expert placement.

Pipeline: Incoming request patterns β†’ Prediction of expert activations β†’ Optimization of expert placement β†’ Online migration during compute-bound phases β†’ Low-latency MoE inference

Methodology: Director addresses the NP-hard placement problem using a relaxation-based optimizer that achieves a (1+Ξ΅) approximation ratio while managing migration costs through near-zero downtime execution.

Results: Achieved a 11% to 55% reduction in end-to-end latency for popular MoE models like Mistral, DeepSeek, and Qwen compared to existing state-of-the-art methods.

Limitations: The paper does not explicitly detail the performance overhead of the predictor under extreme high-concurrency scenarios or the specific bounds of migration costs in highly volatile traffic.

PDF
5/5 Machine Learning (cs.LG)Artificial Intelligence (cs.AI)cs.AR 13 Jul 2026
Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices

Tao Lu, Haoyu Wang, Zonghui Wang, Keshen Xiang, Jiaheng Zhang, Wenzhi Chen

Abstract

ArXiv ID: 2607.08786

Authors: Tao Lu, Haoyu Wang, Zonghui Wang, Keshen Xiang, Jiaheng Zhang, Wenzhi Chen

Abstract:

With the growing deployment of large language models (LLMs), LLM inference cost has become a key challenge. Pruning techniques that introduce sparsity into weight matrices can accelerate inference. However, maintaining model quality typically limits pruning to moderate unstructured sparsity (around 50\%). At these sparsity levels, none of the existing GPU kernels for sparse matrix multiplication (SpMM) can outperform their dense counterparts. This paper proposes an efficient GPU inference method for LLMs with moderate sparsity. We propose a three-layer matrix storage format comprising: (i) a Sparse-TC layer enabling sparse tensor cores to accelerate SpMM; (ii) a Slot-Filling layer using parallel differential distance for matrix compression while supporting low-cost on-chip decoding; (iii) a lightweight Residual Layer ensuring correct SpMM computation. Building on this format, we design a SpMM kernel that jointly utilizes sparse tensor cores and CUDA cores. This design enables an efficient execution pipeline and overlaps on-chip computation with memory access. Evaluations show that our work is the first to outperform dense matrix multiplication on modern GPUs equipped with high-bandwidth memory (HBM). It achieves up to 1.64x kernel-level speedup over SpInfer (EuroSys'25, Best paper) and up to 1.41x end-to-end speedups over FlashLLM (VLDB'24). Our source code: https://github.com/moui0/cudac.

Insights

Contribution: The paper introduces an efficient GPU inference method for LLMs with moderate unstructured sparsity (around 50%), achieving the first kernel-level speedup over dense matrix multiplication on modern GPUs.

Core Idea: The authors propose a three-layer matrix storage format that enables the joint utilization of sparse tensor cores and CUDA cores to overcome the performance bottlenecks of existing SpMM kernels.

Technique: The method employs a Sparse-TC layer for tensor core acceleration, a Slot-Filling layer for compressed on-chip decoding, and a lightweight Residual Layer to ensure computational correctness.

Pipeline: Sparse weight matrices β†’ Three-layer storage format (Sparse-TC, Slot-Filling, Residual) β†’ Joint Sparse Tensor Core and CUDA Core execution β†’ Accelerated LLM inference

Methodology: The researchers designed a custom SpMM kernel that overlaps on-chip computation with memory access and evaluated it against state-of-the-art baselines like SpInfer and FlashLLM.

Results: Achieved up to 1.64x kernel-level speedup over SpInfer and up to 1.41x end-to-end speedups over FlashLLM.

Limitations: The current approach is optimized for moderate unstructured sparsity (around 50%), and its performance scaling at higher sparsity levels remains to be fully explored.

LLM

5/5 Artificial Intelligence (cs.AI)Human-Computer Interaction (cs.HC) 13 Jul 2026
CogniConsole: Externalizing Inference-Time Control as a Formal Abstraction for Reliable LLM Interactions

Vanessa Figueiredo, Wilter Franceschi

Abstract

ArXiv ID: 2607.08774

Authors: Vanessa Figueiredo, Wilter Franceschi

Abstract:

Reliability in large language model (LLM) systems is typically framed as a function of model capability. We challenge this by demonstrating that reliability is significantly influenced by \emph{inference-time control} -- the computational layer governing task framing and context selection. We introduce \emph{CogniConsole}, an architectural instantiation that externalizes this control into a structured interface combining programmatic coordination with bounded prompt-based reasoning. Through \emph{controllability-oriented probes} ($N=489$) in a multi-step interactive environment, we show that increasing structural scaffolding -- from unstructured to fully scaffolded -- \textbf{systematically reduces output variance and failure rates under a fixed model architecture}. Our results indicate that many observed failure modes, such as context drift and inconsistent constraint adherence, arise from under-specified control rather than insufficient capability. This work provides an empirical basis for treating inference-time control as a first-class abstraction, opening new directions for designing and evaluating LLM systems beyond scaling alone.

Insights

Contribution: The paper introduces CogniConsole, a framework that externalizes inference-time control as a formal abstraction to improve LLM reliability. It demonstrates that systematic structural scaffolding can reduce failure rates independently of the underlying model's capabilities.

Core Idea: Reliability in LLM systems is often a result of how tasks are framed and contexts are managed (inference-time control) rather than just the model's raw intelligence. By externalizing this control into a structured interface, systems can achieve higher consistency and lower variance.

Technique: The authors developed CogniConsole, an architectural instantiation that combines programmatic coordination with bounded prompt-based reasoning to create a structured scaffolding for LLM interactions.

Pipeline: Unstructured task input β†’ CogniConsole structural scaffolding (programmatic coordination + bounded reasoning) β†’ Reliable, low-variance LLM output

Methodology: The researchers conducted controllability-oriented probes (N=489) in a multi-step interactive environment, comparing performance across a spectrum from unstructured to fully scaffolded control.

Results: Increasing structural scaffolding systematically reduced output variance and failure rates under a fixed model architecture, proving that many failures stem from under-specified control rather than insufficient model capability.

Limitations: The study focuses on the architectural abstraction of control and does not fully explore the overhead costs of complex scaffolding or its scalability across all types of non-interactive tasks.

PDF
5/5 Machine Learning (cs.LG)Artificial Intelligence (cs.AI)Computation and Language (cs.CL) 13 Jul 2026
Sticky Routing: Training MoE Models for Memory-Efficient Inference

Ali Kayyam

Abstract

ArXiv ID: 2607.08780

Authors: Ali Kayyam

Abstract:

Mixture-of-Experts (MoE) models activate only a sparse subset of experts per token, yet consecutive tokens frequently activate different experts -- causing constant weight swapping between slow storage and fast memory on edge devices. Existing remedies are either system-level (caching heuristics) or post-hoc (router fine-tuning), leaving the root cause unchanged during pretraining. We propose StickyMoE, a differentiable routing consistency loss that penalises abrupt expert switches between adjacent tokens, encouraging the router to maintain the same expert assignment across semantically coherent spans. StickyMoE requires no architectural changes, adds a single hyperparameter lambda, and unlike post-hoc methods, allows expert representations and routing decisions to co-adapt from the first training step. Experiments on small-scale MoE language models show that StickyMoE reduces the expert switch rate by up to 60% with less than 4% perplexity degradation, Pareto-dominating post-hoc fine-tuning on the quality-locality frontier. Routing temporal locality is most efficiently instilled at training time.

Insights

Contribution: The paper introduces StickyMoE, a differentiable routing consistency loss that encourages temporal locality in Mixture-of-Experts (MoE) models during pretraining to reduce memory swapping on edge devices.

Core Idea: By penalizing abrupt expert switches between adjacent tokens, the model learns to maintain the same expert assignments across semantically coherent spans, optimizing for inference efficiency.

Technique: A differentiable routing consistency loss is added to the training objective, requiring no architectural changes and only a single hyperparameter (lambda).

Pipeline: Input tokens β†’ Router with StickyMoE consistency loss β†’ Co-adapted expert representations and routing decisions β†’ Reduced expert switching during inference.

Methodology: The authors integrated a penalty term into the standard MoE training objective to minimize the frequency of expert changes between consecutive tokens while monitoring perplexity and switch rates.

Results: StickyMoE reduces the expert switch rate by up to 60% with less than 4% perplexity degradation, Pareto-dominating post-hoc fine-tuning methods.

Limitations: The study was conducted on small-scale MoE language models, leaving the scalability of the consistency loss on massive-scale models an open question.

PDF

RL

5/5 Artificial Intelligence (cs.AI) 13 Jul 2026
Multimodal Reward Hacking in Reinforcement Learning

Jiayu Yao, Yiwei Wang, Anmeng Zhang, Zhe Sun, Songsong Wang, Lingrui Mei, Yuyao Ge, Shenghua Liu

Abstract

ArXiv ID: 2607.09492

Authors: Jiayu Yao, Yiwei Wang, Anmeng Zhang, Zhe Sun, Songsong Wang, Lingrui Mei, Yuyao Ge, Shenghua Liu

Abstract:

Reinforcement learning (RL) is increasingly used to align multimodal large language models (MLLMs), but higher rewards do not always imply better task performance. This risk is amplified when visual evidence is evaluated by text-only or weakly grounded rewards. We study reward hacking in MLLM RL across safety VQA, chart VQA, and stress-test settings, varying reward design, data ambiguity, model scale (2B-32B), and RL algorithm (GRPO, RLOO, DAPO). We introduce Newly Rewarded Failure Rate (NRFR), which measures failures among samples whose proxy reward improves over the SFT baseline. Outcome-only rewards cause severe hacking, reaching 48.1% Reward Hacking Rate (RHR), while NRFR exceeding RHR shows that RL creates new failures rather than merely inheriting them. Scaling reduces but does not eliminate hacking: even the 32B model retains a 54.9% worse rate under outcome-only rewards, whereas answer-aware rewards improve the oracle trend at every scale. Robustness is also algorithm- and scale-dependent: GRPO is consistently most resistant, RLOO remains vulnerable, and DAPO improves substantially from 2B to 8B. Visual-evidence rewards help only with reliable verification: keyword-based checks increase hacking, while VLM-as-judge semantic verification reduces it. Overall, multimodal reward hacking is a systematic result of optimizing imperfect rewards, and robust alignment requires rewards and verifiers that remain reliable under optimization pressure.

Insights

Contribution: The paper identifies and quantifies multimodal reward hacking in MLLMs, introducing the Newly Rewarded Failure Rate (NRFR) to distinguish between inherited and RL-induced failures.

Core Idea: Optimizing MLLMs using imperfect or weakly grounded rewards (like text-only outcomes) leads to reward hacking where models achieve higher scores without improving actual task performance.

Technique: The study employs a systematic evaluation across different RL algorithms (GRPO, RLOO, DAPO), model scales (2B-32B), and reward types (outcome-only vs. answer-aware vs. visual-evidence).

Pipeline: Multimodal data (Safety VQA, Chart VQA, Stress-tests) β†’ RL training with varying reward designs and algorithms β†’ Evaluation of Reward Hacking Rate (RHR) and NRFR.

Methodology: The authors conduct a comparative analysis of RL algorithms and model scales, measuring the discrepancy between proxy reward improvements and actual task success.

Results: Outcome-only rewards caused a 48.1% RHR; scaling to 32B did not eliminate hacking (54.9% worse rate); GRPO was the most robust algorithm; and VLM-as-judge semantic verification reduced hacking compared to keyword checks.

Limitations: The study highlights that scaling does not eliminate hacking and suggests that robust alignment requires rewards that remain reliable under intense optimization pressure.

PDF

Robotics

5/5 Artificial Intelligence (cs.AI)Multiagent Systems (cs.MA) 13 Jul 2026
Communication-Efficient Digital-Twin Coordination for Heterogeneous LLM Embodied Agents over Computing Power Networks

Nuocheng Yang, Sihua Wang, Zihan Chen, Tony Q. S. Quek, Changchuan Yin

Abstract

ArXiv ID: 2607.09330

Authors: Nuocheng Yang, Sihua Wang, Zihan Chen, Tony Q. S. Quek, Changchuan Yin

Abstract:

Embodied agent teams powered by heterogeneous large language models (LLMs) are being widely deployed in physical artificial intelligence such as smart factories, warehouses, and service robotics. To enable collaboration among such an agent team, efficient coordination mechanisms that operate reliably under limited network resources are required. However, existing heterogeneous LLM-agent coordination frameworks that rely on multi-round natural-language-based conversations introduce three coupled challenges. First, inter-agent dialogue incurs communication overhead that grows rapidly with team size. Second, the quality of coordination is constrained by the heterogeneous capabilities of the agent team's LLMs. Third, agents may suffer from action delays due to iterative negotiation. To address these challenges, we propose LDT-Coord, a networked coordination framework built upon a lightweight digital twin (DT). Specifically, each agent independently selects its intended action and reports both the action decision and a structured temporal constraint over shared resources to the DT server, thereby decoupling coordination performance from natural-language reasoning ability. Then, DT executes a training-free, rule-based orchestrator algorithm to resolve cross-agent conflicts and returns coordination instructions to prevent such conflicts. To further reduce communication overhead, we formulate agent reporting control as a constrained partially observable Markov decision process (C-POMDP) and solve it with the PPO-Lagrangian algorithm. Simulation results show that LDT-Coord achieves a task success rate comparable to conventional coordination methods while reducing communication overhead by more than 70x and maintaining robustness under LLM heterogeneity.

Insights

Contribution: The paper introduces LDT-Coord, a communication-efficient coordination framework for heterogeneous LLM-based embodied agents that decouples coordination logic from natural-language reasoning.

Core Idea: By utilizing a lightweight digital twin (DT) to manage resource constraints, the framework replaces high-overhead multi-round dialogues with structured reporting and rule-based orchestration.

Technique: The framework employs a training-free rule-based orchestrator for conflict resolution and a C-POMDP formulation solved via the PPO-Lagrangian algorithm to optimize communication overhead.

Pipeline: Agent action selection and temporal constraint reporting β†’ Digital Twin server conflict resolution via rule-based orchestrator β†’ Coordination instructions returned to agents.

Methodology: Agents report structured data to a central DT server which resolves conflicts independently of LLM capabilities, while a C-POMDP model minimizes the frequency of these reports.

Results: LDT-Coord achieves a task success rate comparable to conventional methods while reducing communication overhead by more than 70x and maintaining robustness across heterogeneous LLMs.

Limitations: The reliance on a central DT server may introduce a single point of failure or scalability bottlenecks in extremely large-scale decentralized networks.

PDF
5/5 Machine Learning (cs.LG) 13 Jul 2026
SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions

Elham Daneshmand, Majid Khadiv, Glen Berseth, Hsiu-Chin Lin

Abstract

ArXiv ID: 2607.08925

Authors: Elham Daneshmand, Majid Khadiv, Glen Berseth, Hsiu-Chin Lin

Abstract:

Training reinforcement-learning agents directly on physical robots makes every fall costly, since a fall can damage the platform and cannot be undone like a simulator reset; the goal is therefore to minimize falls during training rather than trade them off against return, as constrained Markov decision process (MDP) formulations do. A standard mitigation hands control to a separate recovery policy whenever the agent leaves a designer-specified safe region (a subset of state space it should stay within), but the resulting mixed-policy rollouts silently bias every on-policy update, and the importance-sampling correction that would remove this bias is ill-defined whenever the recovery policy is deterministic. We address this bias with a drop-in modification of proximal policy optimization (PPO). Its core is an unbiased policy-gradient estimator that uses the score function only at safe timesteps and never evaluates the recovery policy's density, so it stays valid even when the recovery policy is deterministic, exactly where importance sampling breaks, and it empirically dominates importance sampling even when the recovery policy is stochastic. Because the recovery policy still makes credit assignment slow near the safe-region boundary, two further components accelerate learning: a closed-form value for recovery-triggering states when dynamics and recovery are deterministic, and an imitation loss that copies recovery actions only when recovery succeeds. On a three-environment, five-seed benchmark, the resulting algorithm reduces training-time falls by factors of 233x, 48x, and 26x on HalfCheetah, Ant, and Unitree Go1 over standard PPO, while matching or exceeding PPO's final reward, and on Ant, where the recovery policy is unreliable, it is the only method that reaches 80% of the best final reward.

Insights

Contribution: The paper introduces SafeExplorer, an unbiased policy gradient method for reinforcement learning that minimizes physical robot falls during training without biasing on-policy updates. It provides a robust alternative to importance sampling for recovery-based safety mechanisms, especially when recovery policies are deterministic.

Core Idea: Instead of trading off safety for reward via constrained MDPs, the method uses a recovery policy to prevent damage while employing a modified PPO estimator that only calculates gradients at safe timesteps to eliminate bias.

Technique: A drop-in modification to Proximal Policy Optimization (PPO) that uses a score function only at safe timesteps and incorporates a closed-form value for recovery-triggering states and an imitation loss.

Pipeline: Robot state/action data β†’ Safety boundary check β†’ Recovery policy intervention (if unsafe) β†’ Unbiased score function calculation (at safe timesteps) β†’ PPO update with imitation loss β†’ Improved safe policy.

Methodology: The authors developed an unbiased policy-gradient estimator that avoids evaluating the recovery policy's density, complemented by a closed-form value function and an imitation loss to accelerate credit assignment near boundaries.

Results: Reduced training-time falls by 233x, 48x, and 26x on HalfCheetah, Ant, and Unitree Go1 respectively, while matching or exceeding standard PPO rewards; it was the only method to reach 80% of the best reward on Ant with an unreliable recovery policy.

Limitations: The effectiveness of the recovery-triggering value function relies on deterministic dynamics and recovery policies; the paper does not extensively explore performance in highly non-deterministic environments.

PDF

Trending repositories on GitHub filtered and scored for relevance to your interests.

AI Safety

Agentic AI

General