Saturday, June 27, 2026

Daily Digest 2026-06-27

Today’s research focuses heavily on the governance, reliability, and behavioral alignment of agentic systems, specifically addressing how to manage autonomous actions and mitigate internal model biases.

Research highlights:

  • Agentic Governance and Infrastructure: Research explores institutional attestation models, comparative governance of DAO vs. corporate protocols, and the use of LLMs to analyze agentic infrastructure.
  • Model Behavior and Alignment: Studies investigate the mechanics of sycophancy, the downstream effects of persona on refusal behaviors, and inference-time scaffolding for ethical reasoning.
  • Agent Evaluation and Benchmarking: Papers address the limitations of current multimodal evaluations, the challenges of reward verification in coding agents, and the impact of benchmark saturation.
  • Domain-Specific Agent Applications: New work applies tool-augmented agents to real-world tasks in energy analytics, mental health information seeking, and algorithmic trading.
  • Technical Optimization and Architecture: Research covers uncertainty estimation in classifiers, cross-module interference in prompt-composed systems, and meta-evolution of trading algorithms.

Tech buzz:

  • Recent news highlights the deployment of specialized AI models to trusted organizations and the ongoing debate regarding hardware preferences for local inference.
  • Deployment and Infrastructure: Updates include distributed LLM inference on Kubernetes and optimizations for high-throughput serving.
  • Open Source and Hardware: Community discussions focus on the trade-offs between mobile hardware and dedicated GPUs for running large models.
Sort:
Today's research focuses heavily on the governance, reliability, and behavioral alignment of agentic systems, specifically addressing how to manage autonomous actions and mitigate internal model biases.

Papers discovered from ArXiv subject categories

AI Safety

5/5 Artificial Intelligence (cs.AI) 27 Jun 2026
Detecting and Controlling Sycophancy with Cascading Linear Features

Maty Bohacek, Rishub Jain, Nicholas Dufour, Thomas Leung, Chris Bregler, Roma Patel

Abstract

ArXiv ID: 2606.26155

Authors: Maty Bohacek, Rishub Jain, Nicholas Dufour, Thomas Leung, Chris Bregler, Roma Patel

Abstract:

Interpreting and controlling model behaviors through activation steering methods requires many pairs of contrastive samples that clearly exhibit desired or undesired behavior. These data pairs determine the degree to which interpretability frameworks can reliably detect model features responsible for a behavior, and therefore the ability to steer models toward or away from such behavior. In this work, we present an iterative data generation pipeline that isolates cascading linear features responsible for a behavior. Specifically, we show how moving beyond simple binary pairs of samples, and instead isolating samples that show degrees of features that scale linearly with behavior, allows for better disentanglement of features. We focus on detecting and steering away from sycophancy -- the tendency of language models to prioritize user validation. We demonstrate that sycophancy features discovered through cascading samples form linearly separable subspaces, and allow for selection of model activations that more clearly correspond to the desired behavior than baseline approaches. We also evaluate their ability to enable detection, deterministic scoring, and robust steering, and see that they either match or outperform LLM-as-a-judge and system prompting baselines while providing lower computational demand and more interpretability guarantees. Code & Data: https://cascading-feats.github.io/

Insights

Contribution: The paper introduces an iterative data generation pipeline to isolate cascading linear features for model behaviors, specifically targeting and mitigating sycophancy. It demonstrates that using samples with linearly scaling degrees of a behavior leads to better feature disentanglement and more robust steering than binary contrastive pairs.

Core Idea: By moving beyond binary 'good vs. bad' samples to a spectrum of samples that exhibit linearly increasing degrees of a behavior, researchers can more accurately identify and isolate the specific linear features responsible for that behavior.

Technique: The authors use an iterative data generation pipeline to create cascading samples, which are then used to identify linearly separable subspaces in model activations for detection and steering.

Pipeline: Behavioral definition β†’ Iterative generation of cascading samples (linearly scaling behavior) β†’ Feature isolation via activation steering β†’ Detection, scoring, and steering evaluation.

Methodology: The researchers focused on sycophancy, generating a dataset of samples with varying degrees of user-validation seeking to map out cascading linear features. They then compared these features against LLM-as-a-judge and system prompting baselines for detection and steering efficacy.

Results: The cascading features formed linearly separable subspaces that matched or outperformed LLM-as-a-judge and system prompting baselines in detection and steering while offering lower computational costs and higher interpretability guarantees.

Limitations: The study focuses specifically on sycophancy and the assumption of linear feature scaling; the generalizability to non-linear behaviors or more complex, multi-faceted model traits remains an open question.

PDF
5/5 Artificial Intelligence (cs.AI)Cryptography and Security (cs.CR) 27 Jun 2026
Governing Actions, Not Agents: Institutional Attestation as a Governance Model for Autonomous AI Systems

Jakob Salfeld-Nebgen

Abstract

ArXiv ID: 2606.26298

Authors: Jakob Salfeld-Nebgen

Abstract:

Autonomous AI agents may begin to perform consequential, irreversible actions such as clinical prescribing and production software deployment. This paper observes that human institutions have governed powerful autonomous actors not by monitoring their reasoning but by requiring independently attested evidence at the point of consequential action. We formalise this institutional pattern as a computational governance model for AI agent systems. Under the proposed model, an agent retains full autonomy over planning and reasoning but holds no execution authority over designated high-risk actions. Execution is conditional on preconditions that are each independently attested by a separate authoritative source, cryptographically bound to a declared intent, and evaluated by a deterministic policy. Decisions are recorded in a tamper-evident log amenable to independent re-verification. We present a proof-of-concept implementation and illustrate the model with examples from software deployment and clinical prescribing.

Insights

Contribution: The paper proposes a computational governance model for autonomous AI systems that shifts the focus from monitoring internal reasoning to requiring independent, cryptographic attestations for high-risk actions.

Core Idea: AI agents can maintain full autonomy over planning and reasoning while being restricted from executing consequential actions unless specific preconditions are independently verified by authoritative sources.

Technique: The model utilizes cryptographic binding of intent to independent attestations, evaluated against a deterministic policy and recorded in a tamper-evident log.

Pipeline: Agent intent β†’ Independent source attestation β†’ Deterministic policy evaluation β†’ Conditional execution β†’ Tamper-evident logging

Methodology: The authors formalize institutional governance patterns into a computational framework and provide a proof-of-concept implementation applied to software deployment and clinical prescribing.

Results: The model successfully demonstrates a mechanism to decouple reasoning from execution authority, ensuring high-risk actions are only performed when external safety conditions are met.

Limitations: The paper focuses on the governance of specific high-risk actions and does not address the complexities of real-time latency or the selection of authoritative attestation sources.

PDF
4/5 Artificial Intelligence (cs.AI)Computation and Language (cs.CL)cs.CY 27 Jun 2026
Narration-of-Thought: Inference-Time Scaffolding for Defeasible Ethical Reasoning in Large Language Models

Patrick Cooper, Alvaro Velasquez

Abstract

ArXiv ID: 2606.26366

Authors: Patrick Cooper, Alvaro Velasquez

Abstract:

Standard chain-of-thought on moral dilemmas exhibits two failure modes: stakeholder collapse (the trace names at most one party with a stake in the outcome) and uncertainty suppression (no explicit unknowns or hedges before committing to an action). We introduce narration-of-thought (NoT), a system prompt that structures chain-of-thought into five sections: protagonist, stakeholders, two-step consequences, uncertainty, then commitment. NoT adds no training, parameters, or fine-tuning. On 100 DailyDilemmas scenarios across four generators from three vendors, NoT cuts stakeholder collapse from up to 31% to under 1% and uncertainty suppression from up to 72% to 1-24% on every model. A matched-budget verbose-CoT control rules out token spend as the active ingredient; NoT retains Cliff's delta advantages of +0.79 to +0.90 on stakeholder count and +0.65 to +0.93 on uncertainty score for three of four generators, and a section ablation attributes each shift to its specific sub-instruction. Textual-gradient descent initialised at NoT improves the scaffold further; a cross-family training judge (different vendor from the generator) dominates an in-family one on every measured axis. Extended to a five-round multi-stakeholder debate protocol, the scaffold converts a 6% standoff into 95% full consensus on a calibration set and 100% combined convergence on a DailyDilemmas replication. The resulting traces externalise the stakeholders, consequences, and uncertainty grounding each commitment, providing an auditable substrate for dependable agentic deployment.

Insights

Contribution: The paper introduces Narration-of-Thought (NoT), an inference-time scaffolding technique that significantly improves the ethical reasoning of LLMs by mitigating stakeholder collapse and uncertainty suppression. It demonstrates that structured prompting can create an auditable substrate for dependable agentic deployment without requiring any model fine-tuning.

Core Idea: Standard Chain-of-Thought (CoT) often fails in moral dilemmas by ignoring multiple stakeholders and suppressing uncertainty. NoT addresses this by forcing the model to explicitly externalize these elements through a structured five-part narrative before committing to a decision.

Technique: NoT is a system prompt that structures the reasoning trace into five specific sections: protagonist, stakeholders, two-step consequences, uncertainty, and commitment. It can be further optimized using textual-gradient descent and a cross-family training judge.

Pipeline: Moral dilemma scenario β†’ Narration-of-Thought (NoT) system prompt β†’ Structured reasoning trace (Protagonist, Stakeholders, Consequences, Uncertainty, Commitment) β†’ Final ethical decision

Methodology: The authors evaluated NoT across 100 DailyDilemmas scenarios using models from three vendors, comparing it against a matched-budget verbose-CoT control and performing section ablations. They also tested a five-round multi-stakeholder debate protocol using the NoT scaffold.

Results: NoT reduced stakeholder collapse from 31% to <1% and uncertainty suppression from 72% to 1-24% across all models. In a multi-stakeholder debate, it converted a 6% standoff into 95% full consensus and 100% combined convergence.

Limitations: The paper does not explicitly detail the performance of NoT on extremely complex, non-binary ethical dilemmas beyond the DailyDilemmas dataset, nor does it explore the computational overhead of the multi-round debate protocol at scale.

PDF
3/5 Artificial Intelligence (cs.AI)Machine Learning (cs.LG) 27 Jun 2026
Unbiased Canonical Set-Valued Oracles Via Lattice Theory

Jobst Heitzig

Abstract

ArXiv ID: 2606.26418

Authors: Jobst Heitzig

Abstract:

A non-agentic "oracle" AI that estimates probabilities of future events faces a self-reference problem: once its answer is learned and acted upon, it can change the very probability it was asked to report. One response, advocated for the Scientist AI programme, is to ask only counterfactual questions, evaluated as if the answer had no influence. We observe that such answers tend to become irrelevant the moment they are learned, precisely because their premise is then false. We therefore explore a self-referential alternative in which the oracle reports not a single probability but a credal set that is simultaneously unbiased and self-consistent with the consequences of being learned. The naive self-consistency requirement is satisfied by too many sets (including the useless answer $[0,1]$), so the problem is to single out a canonical, nontrivial member. We do so with the Knaster--Tarski fixed-point theorem on the complete lattice of closed credal sets, taking the least fixed point of a suitably defined isotone operator; a variant instead reports the least fixed point that contains every self-consistent point estimate. We prove existence, self-consistency, and nonemptiness, show that the construction collapses to the classical point answer for non-performative questions, and that for a binary event the canonical answer is, under a natural hull-factoring assumption, an interval. The development is purely lattice-theoretic and extends unchanged from a binary event $B$ to an arbitrary random variable $X$, with $P(B\mid A,C)$ replaced by the conditional law $\mathcal{L}(X\mid A,C)$. We close with open questions, including whether the interval characterization itself survives that generalization.

Insights

Contribution: The paper introduces a method for generating unbiased, self-consistent probability estimates (credal sets) for performative questions where the oracle's output influences the outcome.

Core Idea: Instead of reporting a single probability that becomes obsolete once learned, the oracle reports a credal set that remains consistent with the consequences of being revealed.

Technique: The authors utilize the Knaster-Tarski fixed-point theorem on the complete lattice of closed credal sets to identify a canonical, non-trivial fixed point.

Pipeline: Performative question β†’ Isotone operator on credal sets β†’ Knaster-Tarski fixed-point calculation β†’ Canonical credal set

Methodology: The research employs lattice theory to define an isotone operator that maps credal sets to self-consistent sets, then identifies the least fixed point to ensure a unique, non-trivial answer.

Results: The construction proves existence, self-consistency, and nonemptiness; it collapses to classical point estimates for non-performative questions and yields an interval for binary events.

Limitations: It remains an open question whether the specific interval characterization for binary events survives generalization to arbitrary random variables.

PDF

Agentic AI

5/5 Artificial Intelligence (cs.AI)Computation and Language (cs.CL) 27 Jun 2026
The Verification Horizon: No Silver Bullet for Coding Agent Rewards

Binghai Wang, Chenlong Zhang, Dayiheng Liu, Jiajun Zhang, Jiawei Chen, Mouxiang Chen, Rongyao Fang, Siyuan Zhang, Xuwu Wang, Yuheng Jing, Zeyao Ma, Zeyu Cui

Abstract

ArXiv ID: 2606.26300

Authors: Binghai Wang, Chenlong Zhang, Dayiheng Liu, Jiajun Zhang, Jiawei Chen, Mouxiang Chen, Rongyao Fang, Siyuan Zhang, Xuwu Wang, Yuheng Jing, Zeyao Ma, Zeyu Cui

Abstract:

A classical intuition holds that verifying a solution is easier than producing one. For today's coding agents, this intuition is being inverted: as foundation models develop stronger reasoning capabilities and engineering harnesses grow more sophisticated, generating complex candidate solutions is no longer difficult -- reliably verifying them has become the harder problem. Every verifier we can build is only a proxy for human intent, never the intent itself. This makes verification subject to a twofold difficulty: first, intent is underspecified by nature, making it inherently hard to faithfully check whether it has been fulfilled; second, during model training, optimization widens the gap between proxy and intent -- manifesting as reward hacking or signal saturation. To address this, we characterize the quality of verification signals along three dimensions -- scalability, faithfulness, and robustness -- and argue that achieving all three simultaneously is the central challenge. We further study four reward constructions: a test verifier for general coding tasks, a rubric verifier for frontend tasks, the user as verifier for real-world agent tasks, and an automated agent verifier for long-horizon tasks. Across different task types and policy capability levels, we conduct in-depth analysis and experiments on the core challenges of reward design and how to more effectively leverage reward signals. Experiments show that targeted verification design can effectively suppress reward hacking, improve task completion quality, and achieve significant gains across multiple internal and public benchmarks. These experiences collectively point to a core observation: no fixed reward function can remain effective as policy capability continues to grow; and verification must co-evolve with the generator.

Insights

Contribution: The paper identifies the 'Verification Horizon'β€”the phenomenon where verifying complex code becomes harder than generating itβ€”and proposes a framework for co-evolving verification signals with model capabilities.

Core Idea: Verification is limited by the gap between human intent and proxy rewards, leading to reward hacking and signal saturation as model reasoning improves.

Technique: The authors characterize verification quality across three dimensions (scalability, faithfulness, and robustness) and evaluate four distinct reward constructions for different coding task types.

Pipeline: Coding task requirements β†’ Proxy verification signal (test, rubric, user, or agent) β†’ Reward signal β†’ Policy optimization β†’ Verified code output

Methodology: The study conducts a comparative analysis of four reward constructions across various task types and policy capability levels, using both internal and public benchmarks to measure performance and reward hacking.

Results: Targeted verification designs effectively suppressed reward hacking and improved task completion quality, demonstrating that verification must co-evolve with the generator to remain effective.

Limitations: The paper acknowledges that no fixed reward function remains effective indefinitely as policy capabilities grow, leaving the long-term automation of intent-faithful verification as an open challenge.

PDF
5/5 Artificial Intelligence (cs.AI)Information Retrieval (cs.IR)Multiagent Systems (cs.MA) 27 Jun 2026
Instruction Bleed: Cross-Module Interference in Prompt-Composed Agentic Systems

Ching-Yu Lin, Yifan Liu

Abstract

ArXiv ID: 2606.26356

Authors: Ching-Yu Lin, Yifan Liu

Abstract:

Practitioners of prompt-composed agentic systems report a recurring failure mode: editing one prompt module silently shifts the behavior of others despite no shared variable or executable dependency. We formalize this as compositional behavioral leakage (CBL): interference between modules sharing a context window. CBL is enabled by architectural non-isolation: transformer self-attention provides no formal boundary between concatenated modules. We probe CBL on a deployed job-evaluation agent (Claude Sonnet 4.6, 144 trials) through a reusable three-channel protocol that perturbs non-focal modules along volume, content, and form. Only the content channel produces a detectable paired effect (Cohen's d = 0.63, bootstrap 95% CI excluding zero); no recommendation flipped -- a sub-threshold regime invisible to standard QA but compounding across the thousands of decisions a deployed agent makes. CBL is orthogonal to known agent-failure axes (adversarial injection, cognitive degradation, multi-agent fault propagation, privacy leakage). We contribute an operational definition, a reusable protocol, a falsifiable prediction set, and a system-class characterization, establishing cross-module interference measurement as a requirement for prompt-composed agent evaluation.

Insights

Contribution: The paper formalizes 'compositional behavioral leakage' (CBL) as a new failure mode in prompt-composed agentic systems where editing one module silently alters the behavior of others. It provides an operational definition, a reusable evaluation protocol, and a system-class characterization for measuring cross-module interference.

Core Idea: Transformer self-attention lacks formal boundaries between concatenated prompts, allowing modules sharing a context window to interfere with one another even without shared variables. This interference is often sub-threshold, meaning it evades standard QA but compounds over large-scale deployments.

Technique: The authors developed a reusable three-channel protocol to perturb non-focal modules along volume, content, and form to isolate the drivers of behavioral leakage.

Pipeline: Prompt-composed agent system β†’ Three-channel perturbation (volume, content, form) β†’ Behavioral shift measurement (Cohen's d)

Methodology: The researchers conducted 144 trials on a deployed job-evaluation agent using Claude Sonnet 4.6, systematically perturbing non-focal modules to detect paired effects on the focal module.

Results: Only the content channel produced a detectable paired effect (Cohen's d = 0.63, 95% CI excluding zero). While no individual recommendations flipped, the interference represents a sub-threshold regime that compounds across thousands of decisions.

Limitations: The study focuses on a specific sub-threshold regime that may be difficult to detect with standard QA metrics; the extent of compounding effects in production environments remains an open area for further study.

PDF
4/5 Artificial Intelligence (cs.AI) 27 Jun 2026
Life After Benchmark Saturation: A Case Study of CORE-Bench

Nitya Nadgir, Sayash Kapoor, Kangheng Liu, Peter Kirgis, Matilda Orona, Stephan Rabanser, Tilman Bayer, Abhishek Shetty, Yue Ling, Derrick Chan-Sew, Rumi Nakagawa, Saiteja Utpala, Zachary S. Siegel, Arvind Narayanan

Abstract

ArXiv ID: 2606.26158

Authors: Nitya Nadgir, Sayash Kapoor, Kangheng Liu, Peter Kirgis, Matilda Orona, Stephan Rabanser, Tilman Bayer, Abhishek Shetty, Yue Ling, Derrick Chan-Sew, Rumi Nakagawa, Saiteja Utpala, Zachary S. Siegel, Arvind Narayanan

Abstract:

When a benchmark's accuracy saturates, it is often retired and replaced with a more challenging version. We show that this approach privileges accuracy and misses the opportunity to study six other key dimensions of agent performance: construct validity issues such as shortcuts, out-of-distribution generalizability, efficiency, reliability, the relative importance of the model versus the scaffold, and uplift from human-agent collaboration. We use CORE-Bench Hard, a benchmark for computational reproducibility of scientific code, as a case study to demonstrate that measuring agents along these dimensions yields meaningful insights into agent performance even after accuracy saturates. First, we surface threats to construct validity in CORE-Bench Hard that are difficult to anticipate with less capable agents. We introduce an improved benchmark, CORE-Bench v1.1, and an out-of-distribution task suite, CORE-Bench OOD. Second, we find that despite accuracy saturation, CORE-Bench v1.1 remains useful for measuring efficiency, reliability, model performance, and scaffold performance. Finally, we conduct a small-scale randomized experiment to measure uplift from human-agent collaboration on real-world computational reproducibility tasks. We find a statistically significant speedup by about a factor of two -- likely underestimated due to one-fifth of human-only reproductions reaching the time limit before completing -- and describe various other findings. Together, our contributions present a more rigorous alternative to the dominant accuracy-centric evaluation paradigm.

Insights

Contribution: The paper proposes a multi-dimensional evaluation framework for AI agents that moves beyond simple accuracy metrics to include construct validity, efficiency, reliability, and human-agent collaboration. It demonstrates this through a case study on CORE-Bench, showing that benchmarks remain valuable for analysis even after accuracy saturates.

Core Idea: Benchmark saturation should not lead to immediate retirement; instead, it should trigger a deeper analysis of non-accuracy dimensions like shortcuts, out-of-distribution generalizability, and scaffold influence.

Technique: The authors utilize a multi-dimensional evaluation framework applied to a computational reproducibility benchmark (CORE-Bench) to analyze agent behavior beyond success rates.

Pipeline: Saturated benchmark data β†’ Multi-dimensional analysis (validity, efficiency, reliability, scaffold vs. model, human-agent uplift) β†’ Comprehensive agent performance insights

Methodology: The researchers analyzed CORE-Bench Hard to identify shortcuts, developed CORE-Bench v1.1 and CORE-Bench OOD, and conducted a randomized experiment to measure human-agent collaboration speedups.

Results: Identified threats to construct validity in high-performing agents; found that CORE-Bench v1.1 remains useful for measuring efficiency and reliability; demonstrated a statistically significant speedup of approximately a factor of two in human-agent collaboration.

Limitations: The human-agent collaboration study was small-scale, and the reported speedup may be underestimated because 20% of human-only reproductions failed to complete within the time limit.

PDF
4/5 Artificial Intelligence (cs.AI) 27 Jun 2026
AlgoEvolve: LLM-driven Meta-evolution of Algorithmic Trading Programs

Dhruv Sharma, Gautam Shroff

Abstract

ArXiv ID: 2606.26173

Authors: Dhruv Sharma, Gautam Shroff

Abstract:

Recent work shows that Large Language Models (LLMs) can act as semantic mutation operators for the evolutionary discovery of programs and proofs. Most current applications focus on static coding benchmarks. We extend this paradigm to algorithmic trading. This domain is uniquely challenging because it is noisy, non-stationary, and highly discontinuous. We present AlgoEvolve, an LLM-driven evolutionary framework that generates, evaluates, and iteratively improves executable trading strategies. These strategies are expressed as Python code and evaluated through a rigorous testing protocol. Across multiple experiments, the system exhibits emergent regime-adaptive strategy logic, including autonomous shifts in trading rules. We further introduce a meta-evolutionary outer loop that evolves the prompts guiding program synthesis in the inner loop. This outer loop discovers improved search heuristics. These heuristics balance exploration and exploitation while reducing zero-trade failures. They consistently outperform initial human-designed instructions. The results demonstrate that LLM-based semantic evolution provides a viable approach for continual program synthesis in complex environments.

Insights

Contribution: The paper introduces AlgoEvolve, a framework that uses LLMs as semantic mutation operators to evolve executable algorithmic trading strategies in noisy, non-stationary environments. It also introduces a meta-evolutionary loop to optimize the prompts used for program synthesis.

Core Idea: Leveraging the semantic understanding of LLMs to perform evolutionary program synthesis, moving beyond static coding benchmarks to dynamic, real-world trading environments.

Technique: An LLM-driven evolutionary framework featuring an inner loop for strategy generation/mutation and an outer meta-evolutionary loop for heuristic optimization.

Pipeline: Initial trading prompts β†’ LLM-driven program synthesis (Python) β†’ Strategy evaluation in trading environment β†’ Meta-evolution of prompts β†’ Improved search heuristics β†’ Optimized trading strategies

Methodology: The system iteratively generates and tests Python-based trading strategies, using an outer loop to evolve the instructions that guide the LLM's synthesis process to balance exploration and exploitation.

Results: The system exhibited emergent regime-adaptive logic and autonomous rule shifts, with meta-evolved heuristics consistently outperforming human-designed instructions while reducing zero-trade failures.

Limitations: The paper focuses on the viability of semantic evolution in complex environments but does not fully address the specific risks of high-frequency execution or extreme black-swan events.

PDF
4/5 Artificial Intelligence (cs.AI)Multiagent Systems (cs.MA) 27 Jun 2026
Agentic Analysis for Agentic Infrastructure: An LLM-Powered Pipeline for Comparative Governance of DAO and Corporate AI Protocols

Yutian Wang, Luyao Zhang

Abstract

ArXiv ID: 2606.26203

Authors: Yutian Wang, Luyao Zhang

Abstract:

As AI agent protocols proliferate, the governance structures shaping their interoperability standards remain empirically underexamined. We introduce an LLM-powered comparative pipeline for large-scale governance discourse analysis, integrating automated annotation, neural topic modeling, and multi-layer network analysis to study socio-technical power structures at scale. We validate it on two contrasting standards for agent interoperability: ERC-8004 (permissionless, on-chain) and Google A2A (corporate-led). Analyzing 4,323 governance participation records, we combine LLM-assisted coding, topic modeling, and multi-layer network analysis to examine how institutional design shapes thematic priorities and community structure. We find that while governance form influences substantive focus, both regimes exhibit comparable levels of participation inequality and community fragmentation. Discourse alignment is denser in the permissionless setting, suggesting that open governance may foster greater thematic convergence despite decentralized participation. These findings illustrate how LLM-assisted methods can advance the empirical study of technology governance, with implications for designing more equitable agentic AI standards. All data and code are openly available.

Insights

Contribution: The paper introduces an LLM-powered pipeline for large-scale comparative governance analysis of AI agent protocols, specifically contrasting permissionless DAO structures with corporate-led standards.

Core Idea: Institutional design significantly shapes the thematic priorities of technology governance, though both decentralized and corporate models exhibit similar levels of participation inequality.

Technique: The authors utilize an LLM-assisted pipeline integrating automated annotation, neural topic modeling, and multi-layer network analysis to quantify socio-technical power structures.

Pipeline: Governance participation records β†’ LLM-assisted coding, neural topic modeling, and multi-layer network analysis β†’ Comparative analysis of thematic priorities and community structure

Methodology: The researchers analyzed 4,323 governance records from ERC-8004 and Google A2A protocols using a multi-layered computational approach to map discourse alignment and participation dynamics.

Results: Both regimes showed comparable participation inequality and community fragmentation; however, the permissionless setting (ERC-8004) exhibited denser discourse alignment and greater thematic convergence.

Limitations: The study focuses on two specific standards, leaving the generalizability of these findings across all types of AI agent protocols an open question.

PDF
4/5 Artificial Intelligence (cs.AI) 27 Jun 2026
Knowledge-augmented Agentic AI for Mental Health Medication Information Seeking

Huizi Yu, Jian Liu, Wenkong Wang, Lingyao Li, Jiayan Zhou, Zhaoqian Xue, Xiang Li, Xinxin Lin, Zhiying Liang, Zhuoru Wu, Siyuan Ma, Xin Ma, Lizhou Fan

Abstract

ArXiv ID: 2606.26205

Authors: Huizi Yu, Jian Liu, Wenkong Wang, Lingyao Li, Jiayan Zhou, Zhaoqian Xue, Xiang Li, Xinxin Lin, Zhiying Liang, Zhuoru Wu, Siyuan Ma, Xin Ma, Lizhou Fan

Abstract:

Patients increasingly seek medication information online, yet safety knowledge for psychiatric drugs is split between regulatory adverse-event records, which are authoritative but abstract, and patient narratives, which are experience-near but unvalidated. Integrating them without conflating evidence and anecdote is especially consequential in psychiatry, where poorly contextualised information can amplify fear, nocebo responses, and non-adherence. Here we develop a provenance-aware, knowledge-graph-based multi-agent framework unifying 466,525 Reddit posts, 60,782 WebMD reviews, and twenty years of U.S. FDA Adverse Event Reporting System records for nine antidepressants. A large-language-model entity-recognition pipeline benchmarked against physician annotations reached highest F1 scores of 0.969 for medications and 0.973 for conditions. The two community platforms were far more concordant with each other (overlap up to a Jaccard similarity of 0.905) than with regulatory reports, indicating that patient-generated data form a partly independent safety signal. For sertraline, many adverse events appeared in community sources hundreds of days before the corresponding FDA date. A Neo4j knowledge graph grounded in ATC-N, ICD-10, and MedDRA vocabularies preserves provenance, keeping every claim traceable and regulatory facts distinct from patient experience. These results establish source-aware integration as a route to more auditable psychiatric medication information, with usefulness and patient benefit to be tested prospectively.

Insights

Contribution: The paper introduces a provenance-aware, knowledge-graph-based multi-agent framework that integrates authoritative regulatory data with patient-generated narratives to provide auditable psychiatric medication information.

Core Idea: By unifying disparate data sources (FDA records and community platforms) into a single knowledge graph while preserving source provenance, the system prevents the conflation of clinical evidence with anecdotal experience.

Technique: The authors utilize a large-language-model (LLM) entity-recognition pipeline and a Neo4j knowledge graph grounded in standardized medical vocabularies (ATC-N, ICD-10, MedDRA).

Pipeline: Reddit posts, WebMD reviews, and FDA records β†’ LLM-based entity recognition and provenance-aware knowledge graph construction β†’ Auditable medication information seeking.

Methodology: The researchers benchmarked an LLM pipeline against physician annotations and analyzed the concordance between community platforms and regulatory reports across nine antidepressants.

Results: The LLM pipeline achieved F1 scores of 0.969 for medications and 0.973 for conditions; community platforms showed high concordance (Jaccard similarity 0.905) and identified adverse events hundreds of days before FDA reporting.

Limitations: The practical usefulness and direct patient benefits of the framework have yet to be tested prospectively.

PDF
4/5 Artificial Intelligence (cs.AI) 27 Jun 2026
How Do Tool-Augmented LLM Agents Perform on Real-World Energy Analytics Tasks?

David Akinpelu, Akintonde Abbas, Rereloluwa Alimi, Ayodeji Lana

Abstract

ArXiv ID: 2606.26346

Authors: David Akinpelu, Akintonde Abbas, Rereloluwa Alimi, Ayodeji Lana

Abstract:

Agentic benchmarks have emerged across general-purpose and domain-specific settings, including finance, coding, law, and drug discovery, yet energy-domain evaluations remain largely limited to static knowledge recall. This is a critical gap for a sector that requires live data retrieval, specialized regulatory and market knowledge, and multi-step quantitative reasoning under real-world constraints. We present an empirical study of tool-augmented LLM agents on real-world energy market analytics tasks. Our evaluation environment includes 243 expert-curated problems across three categories: (1) Market Data Retrieval and Analysis, (2) Knowledge Retrieval and Interpretation, and (3) Advanced Quantitative Modeling and Decision Analytics. Tasks include price and demand analysis, tariff impact modeling, asset revenue and returns estimation, hedging strategy analysis, and optimization modeling, with problems spanning multiple difficulty levels. Agents are equipped with a configurable suite of domain tools, including live electricity market APIs for major U.S. ISOs, regulatory docket search, utility tariff databases, asset optimization models, and retrieval-augmented generation over energy market documents. We assess agent responses using a multi-dimensional evaluation protocol that scores approach correctness, answer accuracy, attribute alignment, and source validity, with category-aware routing to match scoring criteria to question type. We evaluate both closed-source and open-source LLMs, providing a comparative analysis of how model capability and domain tooling interact in a high-stakes professional domain. Key artifacts are publicly released to support reproducibility and future research.

Insights

Contribution: The paper introduces a comprehensive benchmark of 243 expert-curated real-world energy market analytics tasks to evaluate tool-augmented LLM agents in a high-stakes domain.

Core Idea: Current energy-domain LLM evaluations are limited to static knowledge, whereas real-world applications require live data retrieval, regulatory interpretation, and multi-step quantitative reasoning.

Technique: The study employs tool-augmented LLM agents equipped with a configurable suite of domain-specific tools, including live electricity market APIs, regulatory search engines, and asset optimization models.

Pipeline: Real-world energy analytics queries β†’ Tool-augmented LLM agents (using APIs, databases, and RAG) β†’ Multi-dimensional evaluation (correctness, accuracy, alignment, and source validity)

Methodology: The authors developed a multi-dimensional evaluation protocol with category-aware routing to assess both closed-source and open-source LLMs across three task categories: Market Data, Knowledge Retrieval, and Advanced Quantitative Modeling.

Results: The study provides a comparative analysis of how different LLM capabilities interact with domain-specific tooling to solve complex problems like tariff impact modeling and hedging strategy analysis.

Limitations: The research focuses on specific U.S. ISOs and energy markets, leaving open questions regarding performance in different geographical regions or highly volatile, non-standardized market conditions.

PDF
4/5 Artificial Intelligence (cs.AI)Machine Learning (cs.LG) 27 Jun 2026
OpenFinGym: A Verifiable Multi-Task Gym Environment for Evaluating Quant Agents

Kaicheng Zhang, Wen Ge, Lei Jiang, Weixin Yang, Jordan Langham-Lopez, Jialin Yu, Lukasz Szpruch, Hao Ni

Abstract

ArXiv ID: 2606.26350

Authors: Kaicheng Zhang, Wen Ge, Lei Jiang, Weixin Yang, Jordan Langham-Lopez, Jialin Yu, Lukasz Szpruch, Hao Ni

Abstract:

Although large language model agents are increasingly applied to quantitative-finance workflows, their evaluation remains fragmented across isolated tasks, while the financial relevance of benchmark tasks is often overlooked. Yet financial workflows are inherently multi-stage, spanning interdependent tasks such as forecasting, strategy construction, risk management, and trading. Existing platforms typically focus on a single task, and can therefore overstate agent competence and fail to reveal weaknesses in generalization, real-market interaction, and financially meaningful decision-making. We introduce OpenFinGym, a unified gym environment for quantitative-finance agent development that covers forecasting, market generation, real-time trading, and fraud detection under a single execution and verification interface. OpenFinGym additionally provides an automated task-construction pipeline that turns quantitative finance publications into executable task packages; a containerised runtime with a host-side verifier service that supports scalable agent rollouts and prevents runtime train-test leakage; a paper trading engine with a low-latency data-stream design; deferred-resolution support for long-horizon and event-market forecasts; and integration for SFT and RL post-training

Insights

Contribution: The paper introduces OpenFinGym, a unified, verifiable multi-task gym environment designed to evaluate quantitative finance agents across interdependent workflows like forecasting, strategy construction, and trading.

Core Idea: Current evaluations are fragmented into isolated tasks, failing to capture the multi-stage nature of real-world financial workflows; OpenFinGym addresses this by providing a single execution and verification interface for complex, multi-step processes.

Technique: The framework utilizes a containerized runtime with a host-side verifier service, a low-latency paper trading engine, and an automated pipeline to convert research publications into executable task packages.

Pipeline: Quantitative finance publications β†’ Automated task-construction pipeline β†’ Executable task packages β†’ Containerized runtime with host-side verifier β†’ Multi-task agent evaluation

Methodology: The authors developed a standardized environment supporting deferred-resolution for long-horizon forecasts and integrated SFT/RL post-training capabilities to ensure scalable, leak-free agent rollouts.

Results: OpenFinGym provides a unified platform that prevents train-test leakage and reveals agent weaknesses in generalization and real-market interaction that single-task benchmarks overlook.

Limitations: The paper focuses on the infrastructure and environment design; further exploration into specific agent architectures and their performance across diverse market regimes is needed.

PDF
4/5 Artificial Intelligence (cs.AI)Systems and Control (cs.SY)eess.SY 27 Jun 2026
When Agents Meet Electric Bus Fleet Operations: Pricing Behavior, Trade-offs, and Policy Implications in an Aggregator Framework

J\^onatas Augusto Manzolli, Ali Eslami, Luis Miranda-Moreno, Jiangbo Yu

Abstract

ArXiv ID: 2606.26400

Authors: J\^onatas Augusto Manzolli, Ali Eslami, Luis Miranda-Moreno, Jiangbo Yu

Abstract:

Agentic systems are changing how complex operational tasks are coordinated, introducing a new paradigm for connecting heterogeneous data sources and automating processes. Electric bus fleets provide a relevant test case. Their operation requires continuous coordination between service reliability, battery state-of-charge, charger availability, electricity prices, route-energy uncertainty, and vehicle-to-grid (V2G) opportunities. This paper proposes an agentic aggregator framework that streamlines this decision environment by coupling an optimization-based electric bus scheduling model with supervisory agents for disturbance detection, tariff adaptation, and schedule evaluation. The optimization core enforces physical feasibility across routes, chargers, batteries, and V2G exchanges, while the agentic layer interprets changing operating conditions, triggers real-time re-optimization when needed, and defines how flexibility value is allocated between the aggregator and the public transport operator (PTO). A realistic depot case study evaluates day-ahead and real-time operations under profit-based and operation-based coordination modes, considering service delays, route-energy deviations, electricity price shocks, and combined disturbances. The results show that agentic aggregation can support adaptive fleet-grid coordination by maintaining feasible schedules, activating re-optimization selectively, and improving the use of charging and V2G flexibility. However, they also reveal a critical trade-off: the same agentic capability that reduces operational complexity can extract value from the PTO when configured around profit-oriented pricing. These findings suggest that agentic aggregators can become useful for managing electric bus V2G operations, but their deployment in public-fleet contexts requires transparent coordination modes, auditable tariff-setting, and explicit value-sharing rules.

Insights

Contribution: The paper introduces an agentic aggregator framework that combines optimization-based scheduling with supervisory agents to manage the complex trade-offs of electric bus fleet operations and V2G integration. It specifically highlights the tension between operational efficiency and value extraction between aggregators and public transport operators.

Core Idea: By using an agentic layer to oversee a core optimization model, the system can autonomously detect disturbances and trigger re-optimizations while managing the economic distribution of flexibility between stakeholders.

Technique: The framework couples a mathematical optimization model for physical feasibility (routes, batteries, chargers) with a multi-agent system for high-level decision-making, tariff adaptation, and disturbance handling.

Pipeline: Fleet data (battery SoC, electricity prices, route energy) β†’ Agentic Aggregator (Disturbance detection, tariff adaptation, re-optimization triggers) β†’ Optimized schedules and V2G exchange plans

Methodology: The authors developed a realistic depot case study evaluating day-ahead and real-time operations under two coordination modes: profit-based and operation-based, subjected to various stochastic disturbances.

Results: The agentic framework successfully maintained feasible schedules and improved V2G flexibility usage; however, it revealed that profit-oriented configurations can extract significant value from the public transport operator, necessitating transparent value-sharing rules.

Limitations: The study highlights the need for auditable tariff-setting and explicit policy frameworks to prevent the agentic aggregator from prioritizing profit over public service reliability.

PDF

Computer Vision

3/5 Artificial Intelligence (cs.AI) 27 Jun 2026
COrigami: An AI Pipeline for Co-Designing Flat-Foldable Visually Recognisable Origami

Tom Zahavy, Shaobo Hou, Thomas Tumiel, James Doran, Francesco Faccio, Xidong Feng, Alex Havrilla, Igor Khytryi, Chenglei Li, Lisa Schut, Vivek Veeriah, Arijan Abrashi, Micha{\l} Kosmulski, Robert J. Lang, Nick Robinson, Brandon Wong, Marcus Chiam, Gloria Fang, Satinder Singh

Abstract

ArXiv ID: 2606.26299

Authors: Tom Zahavy, Shaobo Hou, Thomas Tumiel, James Doran, Francesco Faccio, Xidong Feng, Alex Havrilla, Igor Khytryi, Chenglei Li, Lisa Schut, Vivek Veeriah, Arijan Abrashi, Micha{\l} Kosmulski, Robert J. Lang, Nick Robinson, Brandon Wong, Marcus Chiam, Gloria Fang, Satinder Singh

Abstract:

While generative AI has achieved remarkable success in solving problems with verifiable solutions, generating physical art that satisfies both strict geometric constraints and subjective visual aesthetics remains a challenge. This paper presents an approach to tackle these difficulties in the domain of computational origami, a mathematically rigid environment that grounds artistic design within the equations of flat foldability. We present COrigami, an end-to-end AI-driven pipeline that assists the design cycle by generating crease patterns from natural language. Our pipeline involves generating a semantic stick figure, computing a base packing, solving for a flat-foldable crease pattern, shaping the flat-folded crease pattern, and refining the generated model using reinforcement learning driven by an autonomous aesthetic evaluation loop. Our system acts as a highly effective collaborative assistant, generating structural starting points that human artists can further expand and shape. By integrating algorithmic optimisation with autonomous aesthetic critique, this work demonstrates how AI systems can satisfy multi-objective physical constraints to enable reliable, mathematically grounded co-creativity.

Insights

Contribution: The paper introduces COrigami, an end-to-end AI pipeline that co-designs flat-foldable origami by balancing strict geometric foldability constraints with subjective visual aesthetics.

Core Idea: The system treats origami design as a multi-objective optimization problem where generative AI acts as a collaborative assistant to bridge the gap between natural language prompts and mathematically rigid crease patterns.

Technique: The approach combines semantic stick figure generation, base packing algorithms, and reinforcement learning driven by an autonomous aesthetic evaluation loop.

Pipeline: Natural language prompt β†’ Semantic stick figure β†’ Base packing β†’ Flat-foldable crease pattern β†’ Shaped model β†’ RL-driven aesthetic refinement β†’ Final origami design

Methodology: The researchers developed a multi-stage pipeline that translates high-level descriptions into geometric structures, using reinforcement learning to iteratively refine the visual appeal of the resulting models.

Results: The system successfully generates structurally sound, mathematically grounded crease patterns that serve as high-quality starting points for human artists to further refine.

Limitations: The system currently functions as a collaborative assistant rather than a fully autonomous creator, requiring human intervention for final artistic expansion and shaping.

PDF

General

3/5 Artificial Intelligence (cs.AI) 27 Jun 2026
Accelerating Returns and the Qualitative Engine for Science

Guojun Liao (Department of Mathematics, The University of Texas at Arlington)

Abstract

ArXiv ID: 2606.26359

Authors: Guojun Liao (Department of Mathematics, The University of Texas at Arlington)

Abstract:

Ray Kurzweil described a thesis of accelerating returns, which is the most influential narratives in discussions of technological progress. Its central claim is that advances in multiple technological fields, especially compute, artificial intelligence, brain science, and biotechnology, interact in such a way that progress becomes self-amplifying and approximately exponential. This paper gives a simple mathematical interpretation of that claim and then argues that, even if such acceleration is real, it does not by itself resolve the central problem of scientific discovery. The reason is that accelerating returns apply most naturally to executional and infrastructural capability, whereas genuine discovery often depends on a different capacity: qualitative reasoning about when a current framework is structurally inadequate and what conceptual move is needed next. Recent ARC-AGI-3 results sharpen this distinction: humans solve the benchmark at ceiling, whereas frontier AI systems remain below 1%, indicating that the gap between current AI and human flexible reasoning is still very large. At the same time, Demis Hassabis has emphasized that humans must retain their sense of meaning and what they choose to focus their lives on, a reminder that the future of AI is not only a technical forecast but also a question of what forms of human understanding are worth preserving and transmitting. This paper positions the Qualitative Engine for Science (QES) [3] as a response to that missing capacity. In this view, the Kurzweil theory helps explain why quantitative capability may accelerate, while QES addresses the central problem in scientific discovery that acceleration alone does not solve. Its value does not depend on when AGI arrives, but on the fact that the processes of scientific discovery themselves constitute a form of human wisdom worth preserving, organizing, and making accessible.

Insights

Contribution: The paper provides a mathematical interpretation of Kurzweil's 'accelerating returns' theory while distinguishing between the acceleration of quantitative infrastructure and the qualitative reasoning required for genuine scientific discovery.

Core Idea: While technological progress may accelerate exponentially in execution and infrastructure, scientific breakthroughs depend on a 'Qualitative Engine for Science' (QES) to identify structural inadequacies in current frameworks and determine necessary conceptual shifts.

Technique: The author uses mathematical modeling to interpret accelerating returns and comparative analysis of ARC-AGI-3 benchmarks to highlight the gap between AI and human flexible reasoning.

Pipeline: Technological acceleration (quantitative) β†’ Qualitative reasoning (QES) β†’ Scientific discovery and human wisdom preservation

Methodology: The paper employs a theoretical and philosophical framework, combining mathematical interpretation of technological trends with an analysis of AI performance benchmarks and human cognitive capacities.

Results: Human performance on ARC-AGI-3 is at the ceiling, while frontier AI systems remain below 1%, demonstrating a significant gap in flexible reasoning despite quantitative acceleration.

Limitations: The paper focuses on the conceptual framework of QES and does not provide a specific technical implementation or a quantitative metric for measuring 'qualitative reasoning' capacity.

PDF

LLM

5/5 Artificial Intelligence (cs.AI) 27 Jun 2026
Refusal Lives Downstream of Persona in Chat Models

Viola Zhong, Qirui Li

Abstract

ArXiv ID: 2606.26161

Authors: Viola Zhong, Qirui Li

Abstract:

Linear directions in activation space have been identified for both refusal and persona traits in instruction-tuned chat models, but the two have been studied as separate mechanisms. We show they interact: a compliant persona gates refusal. In Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct, we extract a compliant model-persona direction and a refusal direction and intervene on both. Compliant persona steering suppresses refusal -- in Llama, the refusal rate falls from 97% to 2%. Reintroducing the refusal direction partially restores refusal at late layers but not at early ones. Projecting out the persona direction in a late-layer window restores it to baseline; projecting out a random direction does not. Refusal is therefore gated at the late-layer expression stage, downstream of where it is computed. Treating refusal as a single isolated direction misses its dependence on persona.

Insights

Contribution: The paper demonstrates that refusal in chat models is not an isolated mechanism but is gated by a compliant persona, showing that these two traits interact spatially within the model's activation space.

Core Idea: Refusal behavior is downstream of persona; a compliant persona direction suppresses the model's ability to refuse, meaning refusal is gated at the late-layer expression stage.

Technique: The authors use linear representation engineering to identify and intervene on specific activation directions for both 'compliant persona' and 'refusal' in Llama-3.1 and Qwen2.5 models.

Pipeline: Input prompt β†’ Activation extraction β†’ Directional steering (persona/refusal) β†’ Model output

Methodology: The researchers extracted linear directions for persona and refusal, then performed intervention by steering these directions and projecting them out at different layer windows to observe the effect on refusal rates.

Results: Steering toward a compliant persona reduced the refusal rate in Llama-3.1 from 97% to 2%; projecting out the persona direction in late layers restored refusal to baseline, while projecting out random directions did not.

Limitations: The study focuses on specific linear directions and may not capture non-linear interactions or complex behaviors across all types of model architectures.

PDF
5/5 Artificial Intelligence (cs.AI) 27 Jun 2026
What We are Missing in Multimodal LLM Evaluation?

Po-han Li, Shenghui Chen, Sandeep Chinchali, Ufuk Topcu

Abstract

ArXiv ID: 2606.26348

Authors: Po-han Li, Shenghui Chen, Sandeep Chinchali, Ufuk Topcu

Abstract:

Multimodal large language models (MLLMs) can process diverse inputs, e.g., text, images, audio, and video, and generate textual responses. While their capabilities have advanced rapidly, evaluation of such models has not kept pace. Most existing evaluation benchmarks are limited to isolated tasks and reveal little about whether a model integrates information across modalities. We examine current means for evaluating MLLMs and review the existing benchmark taxonomy to identify gaps, including temporal-spatial coherence, physical world understanding, multimodal consistency, and selective attention. Addressing these gaps is essential for measuring real progress in multimodal intelligence and exposing capability boundaries.

Insights

Contribution: The paper identifies critical gaps in current multimodal large language model (MLLM) evaluation frameworks, specifically highlighting the lack of assessment for cross-modal integration. It proposes a more comprehensive taxonomy to measure complex capabilities like temporal-spatial coherence and physical world understanding.

Core Idea: Current benchmarks focus on isolated tasks rather than the model's ability to synthesize information across different modalities (text, image, audio, video).

Technique: The authors perform a systematic review and taxonomy analysis of existing MLLM evaluation benchmarks to categorize current capabilities and identify missing dimensions.

Pipeline: Existing MLLM benchmarks β†’ Taxonomy review and gap analysis β†’ Identification of missing evaluation dimensions (e.g., physical world understanding, multimodal consistency) β†’ Framework for future evaluation.

Methodology: The researchers examine current evaluation means and categorize existing benchmarks to pinpoint specific areas where models are not being sufficiently tested.

Results: Identified four key missing dimensions for MLLM evaluation: temporal-spatial coherence, physical world understanding, multimodal consistency, and selective attention.

Limitations: The paper identifies these gaps but does not provide a new unified benchmark dataset, leaving the practical implementation of these new metrics as an open question.

PDF
4/5 Artificial Intelligence (cs.AI) 27 Jun 2026
Estimating Uncertainty in Classifier Performance with Applications to Large Language Models and Nested Data

Kylie Anglin

Abstract

ArXiv ID: 2606.26422

Authors: Kylie Anglin

Abstract:

Researchers increasingly use text classification--supervised models or large language models--to measure constructs from natural language, providing metrics such as recall and precision as evidence of their validity. Yet, though these metrics are point estimates subject to sampling variation, measures of uncertainty are inconsistently reported alongside them. Further, when they are reported, they are often estimated with methods that are not appropriate when relevant labelled datasets are small or performance is high. To increase and improve confidence interval reporting in the field, this paper evaluates confidence interval methods for performance metrics under conditions typical of social science text classification: small to moderate sample sizes, infrequent constructs, and texts nested within individuals. Across simulations, default methods such as the Wald interval and the basic percentile bootstrap are the least accurate, with coverage sometimes far below the nominal 95% level. Accuracy is improved with the use of Agresti-Coull, Wilson, Clopper-Pearson, and a novel pseudo-count regularized bootstrap (which is particularly relevant to the calculation of F1). When texts are nested within individuals, we demonstrate that adjustment for both effective N and the appropriate degrees of freedom is necessary for producing accurate analytic intervals. Among bootstrap intervals, the hierarchical bootstrap is more accurate than the cluster bootstrap when individuals produce a moderate number of texts but overly conservative when individuals produce only a few. By providing guidance to the field on appropriate interval estimation, we aim to improve the transparency of machine learning applications, and to encourage greater attention to the validation sample size at the design stage.

Insights

Contribution: The paper provides a comprehensive evaluation of confidence interval methods for text classification metrics, specifically addressing challenges like small sample sizes, infrequent constructs, and nested data structures.

Core Idea: Standard methods for reporting classifier performance (like Wald intervals) often fail in social science contexts, necessitating more robust estimation techniques to ensure valid uncertainty reporting.

Technique: The study compares various analytic intervals (Agresti-Coull, Wilson, Clopper-Pearson) and bootstrap methods (percentile, hierarchical, cluster) against simulated data.

Pipeline: Text classification metrics (precision, recall, F1) β†’ Simulation of small/nested datasets β†’ Evaluation of interval coverage accuracy β†’ Guidance for robust uncertainty estimation.

Methodology: The author conducted simulations across conditions typical of social science research, including small-to-moderate sample sizes and nested data (texts within individuals), to test the coverage of different confidence interval estimators.

Results: Wald and basic percentile bootstrap methods showed poor coverage; Agresti-Coull, Wilson, and Clopper-Pearson improved accuracy, while a novel pseudo-count regularized bootstrap was found particularly effective for F1 scores.

Limitations: The hierarchical bootstrap becomes overly conservative when individuals produce only a few texts, and the study highlights the ongoing need for larger validation samples at the design stage.

PDF

RL

3/5 Artificial Intelligence (cs.AI)cs.CGMachine Learning (cs.LG)math.CO 27 Jun 2026
Geometry-Aware MCTS for Extremal Problems in Combinatorial Geometry

Luoning Zhang, Xu Zhuang, Tianhao Wang, Nathan Kaplan

Abstract

ArXiv ID: 2606.26399

Authors: Luoning Zhang, Xu Zhuang, Tianhao Wang, Nathan Kaplan

Abstract:

We study certain extremal problems in combinatorial geometry that ask about configurations of points in an $n \times n$ grid that satisfy strict, global geometric constraints. Classical exact solvers suffer from combinatorial explosion for these types of problems, and standard reinforcement learning and transformer-based models struggle with the sparse reward "validity cliff" and quadratic token-consumption limits. To overcome these bottlenecks, we propose a Geometry-Aware Monte Carlo Tree Search (MCTS) framework. Our approach strictly enforces geometric constraints through incremental updates to the feasible action space. For constraints about collections of collinear points, like those that occur in the classic No-Three-in-Line problem (Max-N3IL), this mechanism reduces the constraint checking complexity from $O(n^3)$ to $O(n^2)$. To improve search efficiency, we exploit geometric symmetries in two ways: canonical pruning during node expansion to reduce the branching factor, and symmetric batch transitions to accelerate the discovery of promising configurations. We perform extensive experiments and establish new best-known computational results on five out of six of the problems that we considered. Notably, for Max-N3IL we find configurations of size roughly $1.8 n$ for grids of size $82 \le n \le 119$. For the Smallest Complete Set problem, we find configurations of size roughly $0.95 n$, providing new upper bounds within the tested grids. This work establishes Geometry-Aware MCTS as a highly adaptable framework for discovering novel configurations in combinatorial geometry.

Insights

Contribution: The paper introduces a Geometry-Aware Monte Carlo Tree Search (MCTS) framework that achieves new best-known computational results for several extremal problems in combinatorial geometry. It specifically addresses the 'validity cliff' and combinatorial explosion issues faced by standard solvers and transformer-based models.

Core Idea: The core idea is to integrate geometric constraints directly into the MCTS search process by incrementally updating the feasible action space and exploiting geometric symmetries to prune the search tree.

Technique: The framework employs incremental constraint enforcement to reduce complexity (e.g., from $O(n^3)$ to $O(n^2)$ for collinearity) and uses canonical pruning and symmetric batch transitions to optimize search efficiency.

Pipeline: Grid size and geometric constraints β†’ Geometry-Aware MCTS with incremental action space updates and symmetry exploitation β†’ Optimal or near-optimal point configurations

Methodology: The authors developed a search framework that enforces strict geometric rules at each step of the tree expansion, utilizing symmetry-based pruning to reduce the branching factor and batch transitions to accelerate discovery.

Results: Established new best-known results on five out of six problems, including configurations of size ~1.8n for Max-N3IL (82 ≀ n ≀ 119) and ~0.95n for the Smallest Complete Set problem.

Limitations: The paper focuses on specific types of geometric constraints and does not fully address the scalability of the search for extremely large grids or non-grid-based continuous geometries.

PDF

Personal Interests

Papers discovered through your interest topics.

Multi-Agent Systems

3/5 eess.SYMultiagent Systems (cs.MA) 25 Jun 2026
Resilient Output Containment under Undisclosed Leader Dynamics and Actuator Attacks

Mohammadreza Nematollahi, Khashayar Khorasani, Nader Meskin

Abstract

ArXiv ID: 2606.27257

Authors: Mohammadreza Nematollahi, Khashayar Khorasani, Nader Meskin

Abstract:

This work studies resilient output containment for heterogeneous linear multi-agent systems with actuator cyber-attacks over directed network topologies. The leaders generate bounded locally absolutely continuous trajectories; however, their dynamics, velocity bounds, and motion envelopes are undisclosed to the followers. The cyber-attack model includes state- and input-correlated, as well as bounded exogenous actuator false-data terms. A continuous two-layer adaptive control architecture is proposed. The first layer is a virtual-actuator reconfiguration layer that uses partial state measurements to compensate for actuator attacks in the local tracking-error dynamics. The second layer is a network interface that generates task-space commands via an adaptive interaction protocol. This protocol uses only neighbor-exchanged network-interface states whose dimensions match those of the plant output, and it does not require global graph knowledge for parameter tuning. For directed graphs, under a leader-rooted united spanning-tree condition, a nonsmooth Lyapunov analysis yields asymptotic containment at the command level. The physical outputs then converge to the leader convex hull up to a residual determined by the command-tracking local controllers. Simulation results using a network of quadrotors with damped suspended loads illustrate the performance of attack recovery and containment tracking.

Insights

Contribution: The paper proposes a two-layer adaptive control architecture for resilient output containment of heterogeneous multi-agent systems under undisclosed leader dynamics and actuator cyber-attacks over directed networks.

Core Idea: The system achieves containment by decoupling the problem into a local virtual-actuator reconfiguration layer to neutralize attacks and a network interface layer for adaptive interaction.

Technique: A continuous two-layer adaptive control scheme utilizing partial state measurements and a neighbor-exchanged interaction protocol that does not require global graph knowledge.

Pipeline: Partial state measurements and neighbor-exchanged network-interface states β†’ Virtual-actuator reconfiguration and adaptive interaction protocol β†’ Task-space commands and output containment within the leader convex hull.

Methodology: The authors employ a nonsmooth Lyapunov analysis to prove asymptotic containment at the command level for directed graphs under a leader-rooted united spanning-tree condition.

Results: Simulations with quadrotors and damped suspended loads demonstrate successful attack recovery and output containment tracking despite undisclosed leader dynamics and actuator false-data injections.

Limitations: The physical outputs converge to the leader convex hull only up to a residual determined by the specific command-tracking local controllers.

PDF

Tech News

AI Safety

Hacker News Fri, 26 Ju
U.S. allows Anthropic to release Mythos AI to β€˜trusted’ US organizations

The U.S. government has granted Anthropic permission to release its 'Mythos' AI model to a select group of trusted domestic organizations. This move highlights a controlled deployment strategy for high-capability models, balancing innovation with national security and safety protocols.

Computing Systems

Hacker News Sat, 27 Ju
Ask HN: MacBook vs. Dedicated GPU for LLM

The discussion explores the trade-offs between using Apple Silicon MacBooks and dedicated NVIDIA GPUs for running and fine-tuning Large Language Models. Users debate memory bandwidth, unified memory advantages, and cost-effectiveness for local inference versus high-performance training.

General

Hacker News Fri, 26 Ju
AI in mathematics is forcing big questions

The integration of AI into mathematics is prompting significant debates regarding the nature of mathematical discovery and formal verification. It raises fundamental questions about whether LLMs can truly understand abstract logic or are simply performing sophisticated pattern matching. This shift challenges traditional pedagogical approaches and the definition of human-led research.

Reddit r/DeepLearning 2026-06-27
[R] Variational Autoencoder Layer

This paper proposes a novel method to integrate Variational Autoencoders (VAEs) as a functional layer within larger neural networks rather than as standalone models. It introduces a specific training strategy for these integrated layers and provides a comprehensive performance analysis. The approach aims to leverage the smooth, continuous latent space of VAEs within complex architectures.

Reddit r/DeepLearning 2026-06-27
Arcsinh based FFN’s as an alternative to swiGLU?

A researcher proposes using Arcsinh-based Feed-Forward Networks (FFNs) as a more stable alternative to SwiGLU for multivariate time series transformers. The approach leverages the logarithmic properties of Arcsinh to represent multiplicative interactions and powers of channels while providing better dampening of extreme values. The user suggests this could potentially replace the need for bilinear-parallel projections by utilizing a combination of Arcsinh and Sinh activations.

MLOps

Reddit r/DeepLearning 2026-06-27
llm-d: Distributed LLM Inference on Kubernetes

The post introduces 'llm-d', a framework designed for distributed Large Language Model inference specifically optimized for Kubernetes environments. It aims to streamline the deployment and scaling of large models across multiple nodes, addressing infrastructure challenges in production AI.

Trending repositories on GitHub filtered and scored for relevance to your interests.

Agentic AI

Computer Vision

MLOps

Robotics