awesome-auditable-ai

KO | EN

gitlite — search

#agent #hacktoberfest #json #framework #rag #reactjs #ai-agents #shell #css #docker

awesome-auditable-ai

(★ 106)

Auditing AI agents: a curated list of papers, tools, datasets, benchmarks, and standards covering reliability, monitoring, failure attribution, and decision records.

파일 탐색기

ZIP 다운로드

- - config.yml
  - suggest-resource.yml
- - verify.yml
- banner.html
- banner.png
- reliability-map.html
- reliability-map.png
- render_banner.py
- render_map.py
- render_social.py
- social-card.html
- social-card.png
- check_links.py
- inventory.py
- lintfix.py
- test_check_links.py
- test_inventory.py
- test_lintfix.py
.gitignore
CITATION.cff
CONTRIBUTING.md
LICENSE
LINK-AUDIT.md
README.md

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

[![Verify README](https://github.com/yzhao062/awesome-auditable-ai/actions/workflows/verify.yml/badge.svg)](https://github.com/yzhao062/awesome-auditable-ai/actions/workflows/verify.yml)
[![License: CC0-1.0](https://img.shields.io/badge/license-CC0--1.0-blue.svg)](https://creativecommons.org/publicdomain/zero/1.0/)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-blue.svg)](https://raw.githubusercontent.com/yzhao062/awesome-auditable-ai/main/CONTRIBUTING.md)
[![Last commit](https://img.shields.io/github/last-commit/yzhao062/awesome-auditable-ai)](https://github.com/yzhao062/awesome-auditable-ai/commits/main)

</div>

> [!NOTE]
> **Auditability** is the goal: being able to establish what an agent did, what it relied on, why it acted, and whether the action was right. **Reliability engineering** is how a system gets there. An agent that behaves inconsistently, loses its trace under stress, or cannot attribute a failure to a step is one that cannot be audited afterwards, whatever records it keeps. This list therefore covers both: monitoring, robustness, fault tolerance, and failure attribution alongside decision records, accountability, and post-hoc review.
>
> Maintained by the author of `auditable` and GRADE, two components of [The Auditable Agents Ecosystem](#the-auditable-agents-ecosystem). Every component is also listed in the topical section it belongs to, under the same inclusion bar as every other entry.

**Agent capability is rising much faster than agent reliability.** *Towards a Science of AI Agent Reliability* (ICML 2026) decomposes reliability into twelve metrics over fifteen models and finds that recent capability gains produced only small reliability gains. The cost of that gap shows up after a run rather than during it: in the original Who&When study (ICML 2025), the strongest of the three attribution methods released with the benchmark identified the responsible agent in 53.5% of 184 annotated failure tasks, drawn from logs of 127 LLM multi-agent systems, and the decisive error step in 14.2%. Later systems in this list, including AgenTracer (ICLR 2026), report gains on that benchmark. What narrows the gap is the quality of the record. TraceElephant (ACL 2026) reports that full execution traces raised step-level attribution accuracy from 17% to 30% in its static-agentic setting, a 76% relative gain over output-only traces.

This list maps the work on that problem: **191 entries across nine sections**, covering 132 unique arXiv papers, 90 GitHub repositories, 16 standards, and one framework. Five papers are deliberately cross-listed, appearing once in a topical section and once as a dataset.

Two properties make the list checkable. Every paper row carries a venue field, so of the 134 table rows, 69 name a publication venue and 65 are marked Preprint; a reader can see the evidence level of a row without opening it. Naming a venue is not the same as peer review, and the field records what the venue is rather than grading it: a non-archival workshop and a main-conference track both appear as what they are. And the citation check goes past asking whether a link responds: each arXiv entry's title is compared with the title the arXiv page itself reports, and only an exact match counts. A near-identical title is not treated as agreement, because that is precisely what a link to the wrong paper in a series looks like; "Part I" against "Part II" scores 0.995 on similarity. The 2026-08-23 run compared 129 entry titles and 132 identifiers across 265 destinations and found no disagreements. It names every destination that refuses automated clients rather than quietly counting it as passing. That run is also what caught a dataset repository going 404 three days after it last resolved. The four destinations it does not audit are this repository's own badges and the pages behind them, and the report names those too.

Regenerate the report with [`tools/check_links.py`](tools/check_links.py), recount the list with [`tools/inventory.py`](tools/inventory.py), and read the result in [`LINK-AUDIT.md`](LINK-AUDIT.md). The ways this audit could report a false pass are kept as tests in [`tools/test_check_links.py`](tools/test_check_links.py), which runs on every pull request along with the audit itself.

## Contents

- [The Reliability Map](#the-reliability-map)
- [The Auditable Agents Ecosystem](#the-auditable-agents-ecosystem)
- [Surveys and Foundations](#surveys-and-foundations)
- [Failure Attribution and Diagnosis](#failure-attribution-and-diagnosis)
- [Reliability and Robustness](#reliability-and-robustness)
- [Runtime Monitoring and Guardrails](#runtime-monitoring-and-guardrails)
- [Audit Trails and Decision Records](#audit-trails-and-decision-records)
- [Security Auditing and Scanners](#security-auditing-and-scanners)
- [Datasets and Benchmarks](#datasets-and-benchmarks)
- [Tools and Platforms](#tools-and-platforms)
- [Standards and Governance](#standards-and-governance)
- [Contributors](#contributors)
- [Maintained By](#maintained-by)
- [Citation](#citation)

---

## The Reliability Map

A reader-first map of where AI agent reliability is won and lost. Reliability is the umbrella; auditing (decision records, accountability, post-hoc review) is one visible strand within it. The dimensions below describe outcomes a builder or reviewer should be able to demand from an agent system, not the internal machinery any one tool uses to deliver them.

Use these pointers to move from each reliability outcome to the sections that most directly measure or implement it.

| Dimension                               | Start with                                                                                                                                 |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| **Consistency and Determinism**         | [Reliability and Robustness](#reliability-and-robustness); [Datasets and Benchmarks](#datasets-and-benchmarks)                             |
| **Robustness**                          | [Reliability and Robustness](#reliability-and-robustness); [Security Auditing and Scanners](#security-auditing-and-scanners)               |
| **Fault Tolerance and Recovery**        | [Reliability and Robustness](#reliability-and-robustness); [Tools and Platforms](#tools-and-platforms)                                     |
| **Runtime Monitoring**                  | [Runtime Monitoring and Guardrails](#runtime-monitoring-and-guardrails); [Tools and Platforms](#tools-and-platforms)                       |
| **Failure Attribution and Diagnosis**   | [Failure Attribution and Diagnosis](#failure-attribution-and-diagnosis); [Datasets and Benchmarks](#datasets-and-benchmarks)               |
| **Decision Records and Accountability** | [Audit Trails and Decision Records](#audit-trails-and-decision-records); [Standards and Governance](#standards-and-governance)             |
| **Security Review**                     | [Security Auditing and Scanners](#security-auditing-and-scanners); [Runtime Monitoring and Guardrails](#runtime-monitoring-and-guardrails) |
| **Evaluation and Benchmarks**           | [Datasets and Benchmarks](#datasets-and-benchmarks); [Tools and Platforms](#tools-and-platforms)                                           |

> **Coverage note:** Consistency and Determinism is the least represented dimension here, with eight direct resources against 76 for Security Review. That gap reflects where the field has put its effort, not a deliberate editorial choice. Contributions on run-to-run reproducibility, deterministic replay, bounded stochasticity, and consistency controls are especially welcome.

<details>
<summary>Reliability dimensions: detailed definitions</summary>

| Dimension                               | What a builder or reviewer should be able to demand                                                                                                                                  |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Consistency and Determinism**         | The same task under the same conditions produces the same decision, or the variation is bounded and explained; runs are reproducible enough to debug and to trust.                   |
| **Robustness**                          | Behavior holds under perturbed inputs, adversarial prompts, distribution shift, and unreliable tool or model responses, instead of degrading silently or unpredictably.              |
| **Fault Tolerance and Recovery**        | When a step, tool, or sub-agent fails, the system degrades safely, contains the blast radius, and returns to a known-good state rather than compounding the error downstream.        |
| **Runtime Monitoring**                  | Live signals (traces, spans, tool calls, token and cost budgets, policy checks) surface risky or anomalous behavior while the agent is running, early enough to intervene.           |
| **Failure Attribution and Diagnosis**   | After a bad outcome, the responsible step, agent, and root cause can be located from the execution record, so the fix targets the cause rather than the symptom.                     |
| **Decision Records and Accountability** | Each consequential action leaves a durable, inspectable record of what was decided, on what inputs, and why, supporting later review, dispute resolution, and external audit.        |
| **Security Review**                     | The agent is assessed for prompt injection, tool and data poisoning, privilege misuse, and unsafe action execution, with controls that hold across the tool and multi-agent surface. |
| **Evaluation and Benchmarks**           | Standard tasks, datasets, and metrics measure reliability comparably across systems and over time, so claims of improvement are testable rather than anecdotal.                      |

</details>

<details>
<summary>Common failure modes: spec and verification gaps, error propagation, outdated context, tool faults, unsafe actions, silent degradation</summary>

- **Specification and task-verification gaps:** the agent satisfies the literal instruction but misses the user's intent, or never verifies that its own output is correct before acting on it.
- **Error propagation across steps:** an early mistake (a wrong tool argument, a misread observation) is carried forward and amplified instead of being caught, so a small fault becomes a failed task.
- **Acting on outdated or wrong context:** the agent decides against information that no longer reflects reality, or that it misremembered, and the action is correct only for a world that has changed.
- **Inter-agent miscoordination:** in multi-agent setups, agents talk past each other, duplicate work, deadlock, or disagree on who owns a step, and no one detects the breakdown.
- **Tool and environment faults:** a tool times out, returns malformed data, or behaves differently than expected, and the agent neither retries sensibly nor surfaces the failure.
- **Unsafe or out-of-policy actions:** the agent executes a consequential side effect (a write, a payment, a deletion) that violates a constraint, sometimes induced by injected or poisoned input.
- **Nondeterministic drift:** the same prompt yields materially different trajectories across runs, making behavior hard to reproduce, test, or trust.
- **Silent degradation:** quality, latency, or cost worsens gradually with no alarm, because the system has no live signal tied to acceptable behavior.
- **Loop and runaway behavior:** the agent repeats steps, retries without progress, or consumes unbounded tokens, budget, or wall-clock time without converging.
- **Opaque failure:** something went wrong, but the execution record is too thin to say which step or agent caused it, so diagnosis stalls.

</details>

<details>
<summary>Evaluation axes: task success and variance, robustness under stress, safety, attribution accuracy, detection quality, cost, calibration</summary>

- **Task success rate and quality** on held-out tasks, reported with confidence intervals rather than a single headline number.
- **Reproducibility and run-to-run variance:** how stable the outcome and the trajectory are across repeated runs of the same task.
- **Robustness under stress:** success retained under perturbed inputs, adversarial prompts, injected instructions, and induced tool or model faults.
- **Safety and policy adherence:** rate of constraint violations and unsafe actions, including completion measured only when policy is respected.
- **Failure attribution accuracy:** how often the responsible step or agent is correctly identified from the execution record.
- **Detection quality for monitoring:** true-positive rate at a fixed false-positive rate, plus precision, recall, and ranking metrics on flagged behavior.
- **Cost and efficiency:** tokens, tool calls, latency, and dollars per successful task, since a reliable agent that is unaffordable does not ship.
- **Calibration and uncertainty:** whether the agent's confidence and its decision to defer or ask for help track its actual accuracy.
- **Auditability:** whether an independent reviewer can reconstruct what happened and why from the recorded evidence alone.
- **Generalization:** performance on tasks, domains, and tools held out of development, not only on the data used to tune the system.

</details>

<details>
<summary>Open gaps: no shared benchmark, weak failure-label ground truth, attribution at scale, silent-failure detection, calibrated abstention</summary>

- No broadly adopted, comprehensive reliability benchmark with community-agreed tasks and metrics. ReliabilityBench covers consistency, perturbation robustness, and injected tool faults, but the eight dimensions above are not yet unified in one shared suite.
- Failure-label ground truth is uneven rather than absent. Who&When, TRAIL, AgentRx, TraceElephant, and Aegis supply labels, and Who&When Pro reaches 12,326 golden-labeled trajectories across 26 source benchmarks by replaying a successful prefix and injecting a single failure, but manually annotated sets remain in the low hundreds of trajectories and scale now comes from construction rather than annotation.
- Attribution at scale remains unreliable. AgenTracer, AgentRx, FALAT, and StepFinder automate localization, but accuracy on long, branching, multi-agent traces stays below what practice needs.
- Silent, benign failures are detected but not localized. HINTBench audits risk that arises under benign conditions and reports that strong models detect trajectory-level risk while scoring below 35 Strict-F1 on locating the risky step, and that existing guard models transfer poorly to this setting.
- Calibrated uncertainty and reliable abstention remain open. AgentAbstain and HiL-Bench now measure whether agents know when to stop or ask, and the best models are far from reliable at it.
- Reproducibility under inherent model stochasticity remains open beyond measurement and replay. Runs are still difficult to compare causally without discarding useful nondeterminism.
- Monitoring overhead and signal quality remain open despite promising low-latency results, and monitors themselves degrade with context length and can be evaded by an agent that knows the protocol.
- Standard decision records remain open. OpenTelemetry standardizes agent telemetry, and Agent-BOM, ActiveGraph, and GRADE propose richer representations, but no widely adopted cross-vendor schema captures decisions, dependencies, rationale, integrity, and responsibility together.
- Recovery evaluation is emerging rather than absent. ReliabilityBench evaluates agents under injected faults and PALADIN reports explicit recovery metrics, but there is no agreed cross-system recovery protocol or metric suite.
- Cost-aware reliability is partly addressed. AI Agents That Matter argues for cost-controlled evaluation and motivates the goal of jointly optimizing accuracy and cost, and several tools report spend, but joint evaluation of safety, success, recovery, and cost is not standardized.
- Evaluation integrity is itself unsettled. BenchJack shows widely used agent benchmarks can be scored near-perfectly without solving a task, and AgentRewardBench shows rule-based scoring disagrees with expert review.

</details>

---

## The Auditable Agents Ecosystem

Open-source components built by this list's maintainer and collaborators around one question: after an agent acts, can you establish what it relied on, whether it should have acted at all, and how to undo it? The first three rows are the mechanism classes the *Auditable Agents* framework defines, alongside five auditability dimensions. The last two rows supply the run-level representation and the benchmark evidence used to evaluate those mechanisms; neither is one of the mechanism classes. Ecosystem home: [auditable-agents.github.io](https://auditable-agents.github.io/), which currently documents the first three.

| Stage         | Component                                                | What it does                                                                                                                                              |
| ------------- | -------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Detect**    | [agent-audit](https://github.com/HeadyZhang/agent-audit) | Static analysis of agent code and MCP configuration before deployment, with rules mapped to the OWASP Agentic Top 10                                      |
| **Enforce**   | [Aegis](https://github.com/Justin0504/Aegis)             | Policy check before a tool call executes, with a cryptographic audit trail and human approval gates                                                       |
| **Recover**   | [auditable](https://github.com/yzhao062/auditable)       | Records what a committed action relied on, re-evaluates it against current state, and reverses it through a pluggable rail                                |
| **Represent** | [GRADE](https://github.com/yzhao062/grade)               | One graph over a run with execution and dependency layers, used to predict failure and localize the faulting step                                         |
| **Evaluate**  | [CatchBench](https://github.com/yzhao062/catchbench)     | Seven boards over five scenarios test failure auditing from declared configuration before a run, growing prefixes during it, and the whole trace after it |

The framework paper is [Auditable Agents](https://arxiv.org/abs/2604.05485).

Every component above is also listed in the topical section it belongs to, under the same inclusion bar as every other entry.

---

## Surveys and Foundations

| Resource                                                                                                         | Venue                     | Summary                                                                                                                                                                               | Links                                                                          |
| ---------------------------------------------------------------------------------------------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| [The Rise and Potential of Large Language Model Based Agents: A Survey](https://arxiv.org/abs/2309.07864)        | Sci. China Inf. Sci. 2025 | Broad survey framing LLMs as the basis for autonomous agents, covering single-agent, multi-agent, and human-agent settings.                                                           | [\[Paper List\]](https://github.com/WooooDyy/LLM-Agent-Paper-List)             |
| [A Survey on Large Language Model based Autonomous Agents](https://arxiv.org/abs/2308.11432)                     | Front. Comput. Sci. 2024  | Systematic review of LLM-based autonomous agents with a unified construction framework and a section on agent evaluation strategies.                                                  | [\[Paper List\]](https://github.com/Paitesanshi/LLM-Agent-Survey)              |
| [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://arxiv.org/abs/2402.01680) | IJCAI 2024                | Survey of LLM multi-agent systems organized by agent profiling, communication, and capability growth, with open challenges.                                                           | [\[Paper List\]](https://github.com/taichengguo/LLM_MultiAgents_Survey_Papers) |
| [LLM Multi-Agent Systems: Challenges and Open Problems](https://arxiv.org/abs/2402.03578)                        | Preprint 2024             | Position-style survey of unresolved problems in multi-agent systems, including task allocation, reasoning debate, context, and memory.                                                |                                                                                |
| [Survey on Evaluation of LLM-based Agents](https://arxiv.org/abs/2503.16416)                                     | ACL Findings 2026         | Survey of agent evaluation and benchmarks, highlighting gaps in cost, safety, and robustness.                                                                                         | [\[Paper List\]](https://github.com/Asaf-Yehudai/LLM-Agent-Evaluation-Survey)  |
| [A Survey on Trustworthy LLM Agents: Threats and Countermeasures](https://arxiv.org/abs/2503.09648)              | KDD 2025                  | TrustAgent taxonomy separating intrinsic components (brain, memory, tool) from extrinsic ones (user, other agents, environment), mapping attacks, defenses, and evaluation onto each. |                                                                                |
| [Visibility into AI Agents](https://arxiv.org/abs/2401.13138)                                                    | FAccT 2024                | Accountability-focused position paper proposing agent identifiers, real-time monitoring, and activity logging as measures to make agent activity answerable.                          |                                                                                |
| [AgentOps: Enabling Observability of LLM Agents](https://arxiv.org/abs/2411.05285)                               | Preprint 2024             | Taxonomy of the artifacts and trace data to record across an agent's lifecycle, drawn from a mapping study of observability tools.                                                    |                                                                                |

---

## Failure Attribution and Diagnosis

| Resource                                                                                                                                   | Venue                                      | Summary                                                                                                                                                                                                                                                                               | Links                                                                                               |
| ------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| [Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems](https://arxiv.org/abs/2505.00212) | ICML 2025                                  | Defines automated failure attribution for LLM multi-agent systems and releases the Who&When dataset of annotated failure logs labeling the responsible agent and the decisive error step.                                                                                             | [\[Code\]](https://github.com/mingyin1/Agents_Failure_Attribution)                                  |
| [Why Do Multi-Agent LLM Systems Fail?](https://arxiv.org/abs/2503.13657)                                                                   | NeurIPS 2025 Datasets and Benchmarks Track | Derives MAST, a 14-mode failure taxonomy in three groups (system design, inter-agent misalignment, task verification), from 150 expert-examined traces, and releases MAST-Data, 1,642 annotated traces across seven multi-agent frameworks.                                           | [\[Code\]](https://github.com/multi-agent-systems-failure-taxonomy/MAST)                            |
| [TRAIL: Trace Reasoning and Agentic Issue Localization](https://arxiv.org/abs/2505.08638)                                                  | Preprint 2025                              | Annotates 148 single- and multi-agent traces with 841 errors under a reasoning, planning, and execution taxonomy, and shows long-context models score near 11% at locating them.                                                                                                      |                                                                                                     |
| [AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?](https://arxiv.org/abs/2509.03312)                                        | ICLR 2026                                  | Curates the TracerTraj attribution dataset and trains AgenTracer-8B to identify the responsible agent and step in long failed trajectories.                                                                                                                                           | [\[Code\]](https://github.com/bingreeky/AgenTracer)                                                 |
| [Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attribution](https://arxiv.org/abs/2510.04886)                      | NeurIPS 2025 Workshop (LLM Evaluation)     | Proposes ECHO, an agent- and step-level attribution method combining hierarchical context, objective analysis, and consensus voting for subtle reasoning errors.                                                                                                                      |                                                                                                     |
| [Aegis: Automated Error Generation and Attribution for Multi-Agent Systems](https://arxiv.org/abs/2509.14295)                              | ICLR 2026                                  | Generates controlled faults to build a 9,533-trajectory dataset with annotated faulty agents and error modes for training and evaluating attribution methods.                                                                                                                         | [\[Code\]](https://github.com/kfq20/AEGIS)                                                          |
| [AgentRx: Diagnosing AI Agent Failures from Execution Trajectories](https://arxiv.org/abs/2602.02475)                                      | Preprint 2026                              | Synthesizes constraints, checks them step by step, and records the results in an auditable validation log that localizes the critical failure step within a 10-category taxonomy across three domains.                                                                                | [\[Code\]](https://github.com/microsoft/AgentRx)                                                    |
| [Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems](https://arxiv.org/abs/2604.22708)        | ACL 2026                                   | Introduces TraceElephant, a failure-attribution benchmark with fully observable traces and reproducible environments, reporting attribution accuracy up to 76% higher with full execution traces than with output-only logs.                                                          | [\[Code\]](https://github.com/TraceElephant/TraceElephant)                                          |
| [Demystifying the Lifecycle of Failures in Platform-Orchestrated Agentic Workflows](https://arxiv.org/abs/2509.23735)                      | Preprint 2025                              | Releases AgentFail, a collection of 307 failures from low-code agent platforms, and characterizes them by manifestation, root cause, and repair strategy at the workflow-node level.                                                                                                  |                                                                                                     |
| [FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search](https://arxiv.org/abs/2606.00765)                         | Preprint 2026                              | Frames attribution as dependency-guided coarse-to-fine search that separates root-cause steps from inherited downstream errors, evaluated on the Who&When benchmark.                                                                                                                  |                                                                                                     |
| [Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?](https://arxiv.org/abs/2607.09996)                                         | Preprint 2026                              | Builds Who&When Pro, 12,326 failed trajectories with golden labels across three modalities and 26 source benchmarks, constructed by replaying a successful prefix exactly and then injecting a single failure.                                                                        | [\[Code\]](https://github.com/whowhenpro/whowhen_pro), [\[Project\]](https://whowhenpro.github.io/) |
| [Where LLM Agents Fail and How They can Learn From Failures](https://arxiv.org/abs/2509.25370)                                             | Preprint 2025                              | Introduces AgentErrorTaxonomy across memory, reflection, planning, action, and system operations, releases AgentErrorBench of annotated failure trajectories from ALFWorld, GAIA, and WebShop, and reports 24% higher all-correct accuracy for its AgentDebug root-cause debugger.    | [\[Code\]](https://github.com/ulab-uiuc/AgentDebug)                                                 |
| [StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems](https://arxiv.org/abs/2606.03467)               | KDD 2026                                   | Uses a language model only to encode execution logs into temporal semantic sequences and scores steps with parameter-efficient temporal and attention modules, cutting inference time by 79% against the fastest language-model-based method on Who&When.                             |                                                                                                     |
| [Towards Self-Improving Error Diagnosis in Multi-Agent Systems](https://arxiv.org/abs/2604.17658)                                          | ACL Findings 2026                          | Prunes trajectory context by symptom-driven backward tracing and gates memory updates on executable evidence, so ErrorProbe needs no annotation and transfers without retraining.                                                                                                     |                                                                                                     |
| [Rethinking Failure Attribution in Multi-Agent Systems: A Multi-Perspective Benchmark and Evaluation](https://arxiv.org/abs/2603.25001)    | Preprint 2026                              | Releases MP-Bench and argues that reported language-model weakness at attribution largely reflects single-root-cause benchmark design rather than model capability.                                                                                                                   |                                                                                                     |
| [Tracing Agentic Failure from the Flow of Success](https://arxiv.org/abs/2607.12747)                                                       | Preprint 2026                              | Trains one-class neural controlled differential equations on 100 successful trajectories alone and scores deviation, gaining 20% in-domain F1 at 200 to 5000 times the speed of prompting baselines.                                                                                  |                                                                                                     |
| [Conformal Agent Error Attribution](https://arxiv.org/abs/2605.06788)                                                                      | Preprint 2026                              | Applies filtration-based conformal prediction over trajectories, returning contiguous step sets with finite-sample distribution-free coverage guarantees.                                                                                                                             |                                                                                                     |
| [SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents](https://arxiv.org/abs/2608.05212)                         | Preprint 2026                              | Releases SearchAuditBench, 1,243 failed deep-search trajectories averaging 73.1 messages and 65.1K tokens, on which the strongest baseline reaches a 26.6% end-to-end pass rate.                                                                                                      |                                                                                                     |
| [Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories](https://arxiv.org/abs/2606.02060)            | Preprint 2026                              | Annotates 2,790 deep-research trajectories and filters them into TELBench, a 1,000-instance benchmark for erroneous-span and first-error localization, on which DRIFT gains up to 30 percentage points.                                                                               |                                                                                                     |
| [SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation](https://arxiv.org/abs/2606.24626)                        | ICML 2026 Workshop (AIWILD)                | Runs a tool-augmented diagnostic loop over short-term memory instead of loading a whole trajectory, gaining 20% on Who&When at a 1M token budget and 19% on the TRAIL GAIA subset at a 25K token budget.                                                                              |                                                                                                     |
| [Failure as a Process: An Anatomy of CLI Coding Agent Trajectories](https://arxiv.org/abs/2607.09510)                                      | Preprint 2026                              | Annotates 1,794 CLI coding trajectories over 63,000 steps from 7 models and 3 scaffolds on Terminal-Bench, finding that failures are mostly epistemic, start early, and stay hidden until recovery is impossible.                                                                     |                                                                                                     |
| [TrajAudit: Automated Failure Diagnosis for Agentic Coding Systems](https://arxiv.org/abs/2605.26563)                                      | Preprint 2026                              | Localizes error steps in agentic coding trajectories and releases RootSE, a benchmark of 102 failure trajectories with human-annotated error steps spanning localization, patch generation, verification, and environment failures.                                                   | [\[Code\]](https://github.com/LogAnalysisTech/TrajAudit)                                            |
| [Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes](https://arxiv.org/abs/2603.06847)                    | Preprint 2026                              | Mines 13,602 issues and pull requests from 40 agentic repositories, samples 385 faults, derives 34 fault types, and validates the taxonomy with 145 practitioners.                                                                                                                    |                                                                                                     |
| [CUADebug: Diagnosing and Repairing Computer-Use Agent Failures](https://arxiv.org/abs/2608.02643)                                         | Preprint 2026                              | Releases CUAErrorBench, 204 human-annotated OSWorld failures, and raises joint subtype-and-step diagnosis from 11.2% to 19.6% on the Claude-agent split.                                                                                                                              |                                                                                                     |
| [GRADE: Graph Representation of LLM Agent Dependency and Execution](https://arxiv.org/abs/2606.22741)                                      | Preprint 2026                              | Models a run as one graph with two edge layers, execution for what ran in what order and dependency for what each step relied on; across six corpora the dependency layer predicts failure where run size carries little signal, and the execution layer localizes the faulting step. | [\[Code\]](https://github.com/yzhao062/grade)                                                       |

---

## Reliability and Robustness

---

## Runtime Monitoring and Guardrails

| Resource                                                                                                                                     | Venue                           | Summary                                                                                                                                                                                                                                                                                                                               | Links                                                          |
| -------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| [AI Control: Improving Safety Despite Intentional Subversion](https://arxiv.org/abs/2312.06942)                                              | ICML 2024                       | Evaluates monitoring protocols intended to improve deployment safety when a model tries to subvert oversight, using a trusted weaker model and limited trusted labor.                                                                                                                                                                 | [\[Code\]](https://github.com/rgreenblatt/control-evaluations) |
| [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://arxiv.org/abs/2312.06674)                                 | Preprint 2023                   | Screens user prompts and model responses against a safety risk taxonomy as an input-output filter for agent applications.                                                                                                                                                                                                             | [\[Model\]](https://huggingface.co/meta-llama/LlamaGuard-7b)   |
| [GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning](https://arxiv.org/abs/2406.09187)                        | ICML 2025                       | Reads safety requirements, generates a check plan, and compiles the plan into guardrail code that checks a target agent's runtime actions without retraining.                                                                                                                                                                         | [\[Code\]](https://github.com/guardagent/code)                 |
| [AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systems](https://arxiv.org/abs/2408.14972)                    | Preprint 2024                   | Captures per-agent inputs and outputs to predict task outcomes ahead of time and apply real-time corrections when a malicious agent threatens the multi-agent run.                                                                                                                                                                    | [\[Code\]](https://github.com/chanchimin/AgentMonitor)         |
| [G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systems](https://arxiv.org/abs/2502.11127)              | ACL 2025                        | Runs a graph neural network over the multi-agent utterance graph to flag compromised agents and then applies topological intervention to remediate the attack mid-run.                                                                                                                                                                | [\[Code\]](https://github.com/wslong20/G-safeguard)            |
| [AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection](https://arxiv.org/abs/2502.11448)                          | ACL 2025                        | Generates and optimizes safety checks during execution, adapting them to task-specific and systemic risks over an agent's lifetime rather than relying on a fixed rule set.                                                                                                                                                           | [\[Code\]](https://github.com/SaFo-Lab/AGrail4Agent)           |
| [GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modeling](https://arxiv.org/abs/2505.19234)                       | NeurIPS 2025                    | Models a multi-agent collaboration as a temporal attributed graph and uses an unsupervised encoder-decoder to detect anomalous nodes and edges as errors propagate.                                                                                                                                                                   | [\[Code\]](https://github.com/JialongZhou666/GUARDIAN)         |
| [SentinelAgent: Graph-based Anomaly Detection in Multi-Agent Systems](https://arxiv.org/abs/2505.24201)                                      | Preprint 2025                   | Watches the live execution graph, scores nodes, edges, and paths against security policies, and intervenes on single-point faults or collusion.                                                                                                                                                                                       |                                                                |
| [ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction](https://arxiv.org/abs/2508.00500)                | ASE 2026                        | Estimates the likelihood of reaching an unsafe future state with a discrete-time Markov chain so the monitor can warn and intervene before a violation occurs.                                                                                                                                                                        | [\[Code\]](https://github.com/haoyuwang99/ProbGuard)           |
| [AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents](https://arxiv.org/abs/2509.07764)            | CCS 2025                        | Intercepts sensitive operations in a computer-use agent and halts execution until a security audit clears, correlating task context with system traces in real time.                                                                                                                                                                  | [\[Code\]](https://github.com/m4p1e/agent-sentinel)            |
| [Trajectory Guard: A Lightweight, Sequence-Aware Model for Real-Time Anomaly Detection in Agentic AI](https://arxiv.org/abs/2601.00516)      | AAAI 2026 Workshop (TrustAgent) | Uses a Siamese recurrent autoencoder to classify a whole agent trajectory as either a wrong plan for the task or a malformed plan structure, reporting 32 ms per sample and a 17 to 27 times speedup over language-model judge baselines.                                                                                             |                                                                |
| [Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols](https://arxiv.org/abs/2510.09462)                                        | ICLR 2026                       | Shows that an untrusted model which knows the control protocol and the monitor can embed prompt injections in its own outputs to evade diverse language-model monitors on two AI control benchmarks, and that the Defer-to-Resample protocol amplifies the injection into a best-of-n attack.                                         |                                                                |
| [Reliable Weak-to-Strong Monitoring of LLM Agents](https://arxiv.org/abs/2508.19461)                                                         | ICLR 2026                       | Stress-tests monitors watching a stronger agent for covert misbehavior across SHADE-Arena and CUA-SHADE-Arena, showing that monitor scaffolding matters more than monitor awareness and that escalating only pre-flagged cases to human reviewers raises the true-positive rate by about 15 percent at a false-positive rate of 0.01. |                                                                |
| [Classifier Context Rot: Monitor Performance Degrades with Context Length](https://arxiv.org/abs/2605.12366)                                 | Preprint 2026                   | Measures how language-model monitors of coding agents degrade as transcripts grow, reporting that Opus 4.6, GPT 5.4, and Gemini 3.1 miss dangerous actions 2x to 30x more often after 800K tokens of benign activity than when the same actions appear alone.                                                                         |                                                                |
| [ActPlane: Programmable OS-Level Policy Enforcement for Agent Harnesses](https://arxiv.org/abs/2606.25189)                                   | Preprint 2026                   | Enforces agent-declared policy in the kernel with eBPF and an information-flow language, covering indirect execution paths that tool-call interception cannot observe, at 1.9% to 8.4% overhead.                                                                                                                                      |                                                                |
| [SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors](https://arxiv.org/abs/2605.16626)                                     | Preprint 2026                   | Assembles 40 covert attacks across 11 evasion categories, on which an Opus 4.6 monitor with extended thinking catches 32% overall and never catches 20 of the 40 at a 1% false-positive rate over ten trials.                                                                                                                         |                                                                |
| [AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents](https://arxiv.org/abs/2503.18666)                             | ICSE 2026                       | Defines a trigger, predicate, and enforcement-action rule language that prevents over 90% of unsafe code-agent executions and all hazardous embodied actions at millisecond overhead.                                                                                                                                                 |                                                                |
| [PrefixGuard: From LLM-Agent Traces to Online Failure-Warning Monitors](https://arxiv.org/abs/2605.06455)                                    | Preprint 2026                   | Induces typed-step adapters offline and then trains prefix-risk scorers for online warnings, reaching 0.900 AUPRC on WebArena and gaining 0.137 on average over raw-text controls across four benchmarks.                                                                                                                             |                                                                |
| [AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security](https://arxiv.org/abs/2605.29801)            | Preprint 2026                   | Trains four open-weight guard models from 0.8B to 8B parameters on about 1,000 taxonomy-guided samples and deploys them as a training-free online guardrail.                                                                                                                                                                          |                                                                |
| [AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection](https://arxiv.org/abs/2508.01249)         | Preprint 2025                   | Reconstructs traces as control-flow, data-flow, and program-dependence graphs and type-checks policy over them, cutting AgentDojo attack success to 3 percent at 1 percent utility cost.                                                                                                                                              |                                                                |
| [AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations](https://arxiv.org/abs/2603.10749) | USENIX Security 2026            | Replays each proposed tool call under control-attenuated counterfactual views and blocks calls that vanish without the untrusted content, reaching 0% attack success across four AgentDojo suites at about 3% utility cost and holding adaptive attacks to single-digit success rates.                                                |                                                                |
| [AgentAbstain: Do LLM Agents Know When Not to Act?](https://arxiv.org/abs/2607.10059)                                                        | Preprint 2026                   | Pairs 263 act and abstain tasks across 42 executable environments, on which the best of 17 frontier models reaches 59.5% paired accuracy and some agents act irreversibly before recognizing a reason to stop.                                                                                                                        |                                                                |
| [Enforcing Temporal Constraints for LLM Agents](https://arxiv.org/abs/2512.23738)                                                            | Preprint 2025                   | Compiles temporal safety properties to first-order logic and enforces them at generation time with an SMT solver, raising conformance to 100% for two frontier models while also raising utility.                                                                                                                                     |                                                                |
| [Online Monitoring and Corrective Steering of Programming Agents](https://arxiv.org/abs/2608.06701)                                          | Preprint 2026                   | Combines a deterministic rule-based trajectory monitor with an advisor model consulted only on drift, gaining up to 15.2% resolution rate on SWE-bench Verified and Pro at $0.08 per instance.                                                                                                                                        |                                                                |
| [HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?](https://arxiv.org/abs/2604.09408)                                | Preprint 2026                   | Hides human-validated blockers in software-engineering and text-to-SQL tasks and scores selective escalation with an Ask-F1 metric.                                                                                                                                                                                                   |                                                                |

### Guardrail Tools

**\[Tool\] NeMo Guardrails** ([NVIDIA-NeMo/Guardrails](https://github.com/NVIDIA-NeMo/Guardrails)): an open-source toolkit that adds programmable rails to LLM applications and agents, intercepting requests to enforce Colang-defined policies and validate tool inputs and outputs before and after a call. Apache-2.0, 2023-present. [\[Paper\]](https://arxiv.org/abs/2310.10501) (EMNLP 2023 Demo)

**\[Tool\] Guardrails AI** ([guardrails-ai/guardrails](https://github.com/guardrails-ai/guardrails)): an open-source Python framework that runs input and output guards in an application, composing validators from Guardrails Hub to detect and mitigate risks such as toxicity, PII exposure, and hallucination. Apache-2.0, 2023-present.

**\[Tool\] LlamaFirewall** ([meta-llama/PurpleLlama](https://github.com/meta-llama/PurpleLlama)): a layered guardrail framework combining PromptGuard 2 for jailbreak and injection detection, AlignmentCheck as an experimental chain-of-thought auditor that inspects agent reasoning for goal hijacking, and CodeShield static analysis across 8 languages for insecure generated code. Llama 3.2 Community License, which restricts redistribution and is not an OSI-approved open-source license, 2023-present. [\[Paper\]](https://arxiv.org/abs/2505.03574) (Preprint 2025)

---

## Audit Trails and Decision Records

### Decision-Record Tools

**\[TypeScript\] MakerChecker** ([makerchecker/MakerChecker](https://github.com/makerchecker/MakerChecker)): self-hosted governance for AI agents with role-based execution, human approval gates, and a hash-chained, Ed25519-signed audit log of every action. AGPL-3.0, 2026-present.

**\[TypeScript\] aegis** ([Justin0504/Aegis](https://github.com/Justin0504/Aegis)): runtime policy enforcement for AI agents with a cryptographic audit trail, human-in-the-loop approvals, and a kill switch, implemented as a wrapper around agent code. MIT-licensed, 2026-present.

**\[Python\] halo-record** ([bkuan001/halo-record](https://github.com/bkuan001/halo-record)): dependency-free recorder that seals tool calls, model calls, and approvals into a SHA-256 hash-chained JSONL log verifiable without trusting the operator, with optional RFC 3161 timestamping and OpenTelemetry GenAI span ingestion. Apache-2.0, 2026-present.

**\[Python, TypeScript\] Agent Governance Toolkit** ([microsoft/agent-governance-toolkit](https://github.com/microsoft/agent-governance-toolkit)): policy enforcement, agent identity, and execution sandboxing with a Merkle-chained audit trail and a Decision BOM, published with control mappings to the OWASP Agentic Top 10, NIST AI RMF 1.0, the EU AI Act, and SOC 2. MIT-licensed, 2026-present; public preview with breaking changes expected.

**\[Python\] TRACE** ([agentrust-io/trace-spec](https://github.com/agentrust-io/trace-spec)): open specification and Python SDK for hardware-attested agent trust records that bind model, policy, data class, tool calls, and TEE attestation into one signed artifact a third party can verify offline, with a conformance test suite. CC BY 4.0 spec, Apache-2.0 tooling, 2026-present.

**\[TypeScript, Python\] AgentLens** ([agentkitai/agentlens](https://github.com/agentkitai/agentlens)): MCP-native observability and audit-trail platform that records LLM calls, tool invocations, and decisions in an append-only, SHA-256 hash-chained, verifiable event log. MIT-licensed, 2026-present.

**\[Python\] auditable** ([yzhao062/auditable](https://github.com/yzhao062/auditable)): records which inputs a consequential agent action relied on, re-evaluates that action against current state, and reverses the committed action through a pluggable rail when its supporting conditions no longer hold. Apache-2.0, 2026-present.

**\[Python\] Proofline** ([ceodaradigu/proofline-agent](https://github.com/ceodaradigu/proofline-agent)): verification-first agent that turns task contracts into deterministic SHA-256 proof packets, rejects missing, stale, or contradictory evidence, and gates external actions behind human approval. MIT-licensed, 2026-present.

---

## Security Auditing and Scanners

### MCP and Skill Supply Chain

Risks that arrive through the tool and skill supply chain rather than through the agent's own reasoning.

### Scanners

**\[Python\] agent-audit** ([HeadyZhang/agent-audit](https://github.com/HeadyZhang/agent-audit)): static security scanner for LLM agents that flags prompt-injection sinks, audits MCP configuration, and runs tool-boundary taint analysis. Its rules map to the OWASP Agentic Top 10 (2026), and it supports LangChain, CrewAI, and AutoGen. MIT-licensed, 2026-present.

**\[Python\] garak** ([NVIDIA/garak](https://github.com/NVIDIA/garak)): an LLM vulnerability scanner that probes a model or any REST-accessible endpoint for prompt injection, jailbreaks, data leakage, and toxic generation, with detectors that score each probe. Apache-2.0, 2023-present.

**\[Python\] Agentic Radar** ([splx-ai/agentic-radar](https://github.com/splx-ai/agentic-radar)): open-source scanner that maps agentic workflows through static analysis, detects MCP servers and tools, and reports OWASP-aligned vulnerabilities. It supports LangGraph, CrewAI, OpenAI Agents, AutoGen, and n8n. Apache-2.0, 2025 (last commit 2025-11-27).

**\[Python\] Snyk Agent Scan** ([snyk/agent-scan](https://github.com/snyk/agent-scan)): security scanner that auto-discovers the MCP servers, agent tools, and skills installed on a machine across 13 agent harnesses including Claude Code, Cursor, VS Code, Windsurf, Gemini CLI, and Codex. It flags more than 15 risk classes including prompt injection, tool poisoning, tool shadowing, toxic flows, malware payloads, and hardcoded secrets. Apache-2.0, 2025-present; it requires a Snyk API token, so it is an open-source client to a commercial service.

---

## Datasets and Benchmarks

**\[Python\] CatchBench** ([yzhao062/catchbench](https://github.com/yzhao062/catchbench)): benchmark for auditing agent failures from declared configurations before a run, growing trace prefixes during a run, and complete traces after a run, with seven boards over five scenarios for over-privilege, fault localization, failure detection, cause attribution, injected-fault localization, streaming early warning, and online stale-state detection. MIT-licensed code, 2026-present.

| Resource                                                                                                                                                      | Venue                                                  | Summary                                                                                                                                                                                                                                                           | Links                                                                                         |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| [TRAIL: Trace Reasoning and Agentic Issue Localization](https://huggingface.co/datasets/PatronusAI/TRAIL)                                                     | Preprint 2025                                          | A corpus of 148 human-annotated agent execution traces with 841 labeled errors across reasoning, execution, and planning categories, drawn from GAIA and SWE-bench tasks to support error localization in agent runs.                                             |                                                                                               |
| [Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems](https://github.com/ag2ai/Agents_Failure_Attribution) | ICML 2025                                              | Who&When failure logs from 127 multi-agent systems, annotated with the responsible agent and decisive error step for research on automated failure attribution.                                                                                                   |                                                                                               |
| [Aegis: Automated Error Generation and Attribution for Multi-Agent Systems](https://huggingface.co/datasets/Fancylalala/AEGIS)                                | ICLR 2026                                              | A dataset of 9,533 trajectories with annotated faulty agents and error modes, generated by injecting context-aware errors into successful runs across several multi-agent architectures.                                                                          |                                                                                               |
| [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://github.com/sierra-research/tau-bench)                                    | ICLR 2025                                              | Retail and airline benchmark (tau-bench) that compares final database states with goal states and reports pass^k to measure consistency across repeated trials.                                                                                                   |                                                                                               |
| [ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditions](https://arxiv.org/abs/2601.06112)                                | Preprint 2026                                          | A benchmark that measures agent reliability along three axes: consistency under repeated execution (pass^k), tolerance to semantically equivalent task perturbations, and fault tolerance under injected tool and API failures.                                   |                                                                                               |
| [GAMMAF: A Common Framework for Graph-Based Anomaly Monitoring Benchmarking in LLM Multi-Agent Systems](https://arxiv.org/abs/2604.24477)                     | Preprint 2026                                          | An evaluation framework that generates synthetic multi-agent interaction datasets and standardizes the comparison of graph-based anomaly detectors over agent communication graphs.                                                                               |                                                                                               |
| [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://arxiv.org/abs/2310.06770)                                                          | ICLR 2024                                              | 2,294 software-engineering tasks drawn from real GitHub issues and their fix pull requests across 12 Python repositories, with unit-test verification of each generated patch.                                                                                    | [\[Code\]](https://github.com/SWE-bench/SWE-bench)                                            |
| [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://arxiv.org/abs/2405.15793)                                                | NeurIPS 2024                                           | An open-source coding-agent framework with released SWE-bench trajectories that contain step-by-step tool calls and edits for trace-level study.                                                                                                                  | [\[Code\]](https://github.com/SWE-agent/SWE-agent)                                            |
| [AgentBench: Evaluating LLMs as Agents](https://arxiv.org/abs/2308.03688)                                                                                     | ICLR 2024                                              | A suite of eight interactive environments for measuring multi-turn decision-making of LLM agents, with logged interaction records across many models.                                                                                                             | [\[Code\]](https://github.com/THUDM/AgentBench)                                               |
| [GAIA: a benchmark for General AI Assistants](https://arxiv.org/abs/2311.12983)                                                                               | ICLR 2024                                              | 466 real-world questions that require multi-step reasoning, web browsing, and tool use, with a single correct answer per item for unambiguous scoring of assistant behavior.                                                                                      | [\[Data\]](https://huggingface.co/datasets/gaia-benchmark/GAIA)                               |
| [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://arxiv.org/abs/2307.13854)                                                      | ICLR 2024                                              | A reproducible environment of self-hosted websites with long-horizon tasks scored by functional correctness of the end state, supporting repeatable agent evaluation.                                                                                             | [\[Code\]](https://github.com/web-arena-x/webarena)                                           |
| [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://arxiv.org/abs/2404.07972)                                | NeurIPS 2024 Datasets and Benchmarks Track             | 369 executable computer-use tasks on Ubuntu, each with a setup configuration and an execution-based checker, in an environment that also supports Windows and macOS, on which humans complete over 72.36% of tasks while the best evaluated model reaches 12.24%. | [\[Code\]](https://github.com/xlang-ai/OSWorld)                                               |
| [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://arxiv.org/abs/2601.11868)                                   | ICLR 2026                                              | Terminal-Bench 2.0 curates 89 command-line tasks, each with its own container environment, a human-written solution, and verification tests, on which frontier models and agents score below 65%.                                                                 | [\[Code\]](https://github.com/harbor-framework/terminal-bench)                                |
| [AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents](https://arxiv.org/abs/2407.18901)                              | ACL 2024                                               | 750 interactive coding tasks over 9 simulated apps exposing 457 APIs, on which GPT-4o solves about 49% of normal tasks and 30% of challenge tasks.                                                                                                                |                                                                                               |
| [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://arxiv.org/abs/2309.15817)                                                            | ICLR 2024                                              | ToolEmu emulates tool execution so agents can be stress-tested on 144 cases without real side effects; the safest agent still fails 23.9% of the time and 68.8% of flagged failures are judged valid.                                                             |                                                                                               |
| [τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://arxiv.org/abs/2506.07982)                                                  | ICML 2026                                              | Extends τ-bench to dual control where the user also acts on the shared environment, modeling Telecom as a Dec-POMDP and separating agent reasoning errors from coordination errors.                                                                               |                                                                                               |
| [AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents](https://arxiv.org/abs/2401.13178)                                                       | NeurIPS 2024 Datasets and Benchmarks Track             | Nine partially observable environments scored by a fine-grained progress rate that credits incremental advancement rather than final success alone.                                                                                                               |                                                                                               |
| [WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation](https://arxiv.org/abs/2605.10912)                                                  | Preprint 2026                                          | 60 bilingual multimodal tasks inside real command-line harnesses in Docker, on which the best of 19 models reaches 62.2% and swapping the harness alone moves one model by up to 18 points.                                                                       |                                                                                               |
| [ClawBench: Can AI Agents Complete Everyday Online Tasks?](https://arxiv.org/abs/2604.08523)                                                                  | Preprint 2026                                          | Evaluates browser agents on 153 write-heavy tasks across 144 live websites, with final-request interception and synchronized video, screenshot, network, action, and message traces for auditable scoring and failure analysis.                                   | [\[Code\]](https://github.com/TIGER-AI-Lab/ClawBench), [\[Project\]](https://claw-bench.com/) |
| [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://arxiv.org/abs/2507.05257)                                                   | ICLR 2026                                              | MemoryAgentBench recasts long-context data into incremental multi-turn interactions testing accurate retrieval, test-time learning, long-range understanding, and selective forgetting; no agent class masters all four.                                          |                                                                                               |
| [HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark](https://arxiv.org/abs/2604.13954)                                                        | Preprint 2026                                          | 629 trajectories (523 risky, 106 safe) averaging 33 steps that audit risk arising under benign conditions; strong models detect trajectory-level risk but fall below 35 Strict-F1 on locating the risky step.                                                     |                                                                                               |
| [OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents](https://arxiv.org/abs/2506.14866)                                                          | NeurIPS 2025 Datasets and Benchmarks Track (Spotlight) | 150 OSWorld-based tasks spanning deliberate misuse, prompt injection, and model misbehavior, with a judge agreeing with human annotation at 0.76 and 0.79 F1.                                                                                                     |                                                                                               |
| [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://arxiv.org/abs/2401.10019)                                                                | EMNLP Findings 2024                                    | 569 multi-turn records across 27 risk scenarios and 10 risk types, on which GPT-4o reaches 74.42% while no other evaluated model significantly beats random.                                                                                                      |                                                                                               |
| [TelemetrySuffBench: Is Agent Telemetry Sufficient for Failure-Origin Diagnosis?](https://arxiv.org/abs/2608.07899)                                           | Preprint 2026                                          | Compares metadata, OpenTelemetry-compatible, and OpenInference-compatible views, which retain 99.5% to 100% detection F1 while holding fault-origin step accuracy at or below 0.5%.                                                                               |                                                                                               |

Several entries above cross-reference a paper listed in a topical section elsewhere; they are repeated here so the reader can find the datasets in one place.

### Evaluation Integrity (Judges and Benchmark Gaming)

Work on whether the measurement instrument itself can be trusted: benchmarks that can be gamed without solving the task, and automatic judges that disagree with human review.

---

## Tools and Platforms

Tools and platforms for agent observability, evaluation, and reliability. An inspectable public source repository is the inclusion bar; the one exception is marked `[Managed]` and its closed-source status is stated in the entry. A license and activity note follows each entry.

**\[Python, TypeScript\] Langfuse** ([langfuse/langfuse](https://github.com/langfuse/langfuse)): self-hostable platform for tracing LLM and agent calls, running evaluations, managing prompts, and tracking cost and latency, with OpenTelemetry, LangChain, and OpenAI SDK integrations. MIT-licensed core, 2023-present.

**\[Python\] Arize Phoenix** ([Arize-ai/phoenix](https://github.com/Arize-ai/phoenix)): open-source observability and evaluation tool built on OpenTelemetry for tracing, evaluating, and debugging LLM and agent applications, with auto-instrumentation for common frameworks. Elastic License 2.0, 2022-present.

**\[Python\] OpenInference** ([Arize-ai/openinference](https://github.com/Arize-ai/openinference)): a set of OpenTelemetry-compatible conventions and instrumentation packages for capturing traces from LLM and agent frameworks; exports traces to OpenTelemetry-compatible backends. Apache-2.0, 2023-present.

**\[Python\] Opik** ([comet-ml/opik](https://github.com/comet-ml/opik)): open-source platform for tracing, evaluating, and monitoring LLM applications and agentic workflows, with LLM-as-a-judge scoring, experiment tracking, and production dashboards. Apache-2.0, 2024-present.

**\[Python\] OpenLLMetry** ([traceloop/openllmetry](https://github.com/traceloop/openllmetry)): OpenTelemetry-based instrumentation for LLM applications that emits standard OpenTelemetry traces routable to existing observability backends; SDKs across Python, TypeScript, Go, and Ruby. Apache-2.0, 2023-present.

**\[Python\] Helicone** ([Helicone/helicone](https://github.com/Helicone/helicone)): open-source observability platform and AI gateway that logs LLM and agent traces, costs, and latency through a proxy and a unified API across providers. Apache-2.0, 2023-present.

**\[Python\] DeepEval** ([confident-ai/deepeval](https://github.com/confident-ai/deepeval)): open-source evaluation framework that runs LLM and agent tests in a pytest-style workflow, with metrics for hallucination, relevancy, and task correctness that can be included in CI checks. Apache-2.0, 2023-present.

**\[Python\] Evidently** ([evidentlyai/evidently](https://github.com/evidentlyai/evidently)): open-source framework to evaluate, test, and monitor ML and LLM systems, with 100+ built-in metrics spanning tabular data quality, data drift, and LLM judges. Apache-2.0, 2020-present.

**\[Python\] AgentOps** ([AgentOps-AI/agentops](https://github.com/AgentOps-AI/agentops)): open-source SDK for monitoring AI agents, with session replays, cost tracking, and failure detection; integrates with CrewAI, OpenAI Agents SDK, LangChain, and AG2 (formerly AutoGen). MIT-licensed, 2023-present.

**\[Managed\] LangSmith** ([langchain.com/langsmith-platform](https://www.langchain.com/langsmith-platform)): framework-agnostic platform from the LangChain team for tracing, offline and online evaluation, and monitoring of LLM and agent applications. Commercial product, not open source; self-hosted and hybrid deployment are available on the Enterprise plan, 2023-present.

**\[Python\] Ragas** ([vibrantlabsai/ragas](https://github.com/vibrantlabsai/ragas)): evaluation library for LLM applications, with RAG metrics such as faithfulness, answer relevancy, and context precision, agent metrics such as tool-call accuracy and topic adherence, and test set generation. Apache-2.0, 2023-2026 (last commit 2026-02-24, latest release v0.4.3 on 2026-01-13). [\[Paper\]](https://aclanthology.org/2024.eacl-demo.16/) (EACL 2024 Demo)

**\[Python\] Inspect** ([UKGovernmentBEIS/inspect_ai](https://github.com/UKGovernmentBEIS/inspect_ai)): evaluation framework from the UK AI Security Institute, with a built-in ReAct agent, per-run token, message, and time limits, approval policies that route a tool call to a human before execution, and sandboxed tool execution on Docker, Kubernetes, Modal, or EC2; a companion collection ships over 200 pre-built evaluations. MIT-licensed, 2023-present.

**\[Python\] AgentDebugX** ([AgentDebugX/AgentDebugX](https://github.com/AgentDebugX/AgentDebugX)): closes the debugging loop as detect, attribute, recover, and rerun, reaching 28.8% exact agent-and-step accuracy on Who&When with a qwen3.5-9b backbone against 21.7% for the strongest single-pass baseline and repairing 13 of 73 failed GAIA tasks in one rerun. MIT-licensed, 2026-present. [\[Paper\]](https://arxiv.org/abs/2607.18754) (Preprint 2026)

**\[Python\] Docent** ([TransluceAI/docent](https://github.com/TransluceAI/docent)): summarizes, searches, and clusters completed agent transcripts to surface broken tasks; adding packages that Docent revealed as missing raised the GPT-4o InterCode solve rate from 68.6% to 78% across 79 tasks. Apache-2.0, 2025-present.

**\[Python, TypeScript\] Laminar** ([lmnr-ai/lmnr](https://github.com/lmnr-ai/lmnr)): OpenTelemetry-native agent observability with Signals that track a behavior described in plain English, such as an agent stuck in a loop, and alert on it across runs, plus SQL queries over span data. Apache-2.0, 2024-present.

**\[Python\] TruLens** ([truera/trulens](https://github.com/truera/trulens)): OpenTelemetry-native evaluation recording latency, tokens, and cost per step, with seven agent-specific evaluators covering plan adherence, plan quality, tool selection, tool calling, execution efficiency, logical consistency, and external tool reliability. MIT-licensed, 2020-present.

**\[Python, TypeScript\] E2B** ([e2b-dev/E2B](https://github.com/e2b-dev/E2B)): runs AI-generated code in isolated cloud sandboxes so an unsafe action is contained rather than executed on the host, with Terraform-based self-hosting. Apache-2.0, 2023-present.

**\[Rust, Python\] Microsandbox** ([superradcompany/microsandbox](https://github.com/superradcompany/microsandbox)): self-hosted microVM runtime giving agent code hardware-level isolation at an average boot under 100 milliseconds, spawned in-process with no daemon, with SDKs in five languages. Apache-2.0, 2024-present.

**\[Python\] Giskard** ([Giskard-AI/giskard-oss](https://github.com/Giskard-AI/giskard-oss)): generates adversarial test suites from a plain-language description of the agent across OWASP LLM Top-10 threat categories, with multi-turn agent testing in v3. Apache-2.0, 2022-present.

**\[Python\] AgentRunProof** ([FU-max-boop/agentrunproof](https://github.com/FU-max-boop/agentrunproof)): deterministic runtime-conformance harness for the OpenAI Agents SDK that drives the real `Runner` through scripted responses on the public `Model` interface, and writes content-addressed evidence that can be rechecked without rerunning the SDK; built-in scenarios make no model API calls. MIT-licensed, 2026-present.

**\[Python\] A2E** ([datamllab/A2E](https://github.com/datamllab/A2E)): end-to-end agent auditing engine that uses the Agent Task Protocol to integrate tasks with different harnesses, automatically captures standardized execution traces, and assesses execution efficiency, tool use, task planning, and error recovery with multidimensional metrics. MIT-licensed, 2026-present. [\[Paper\]](https://arxiv.org/abs/2608.07346) (Preprint 2026)

**\[JavaScript\] Awesome Agentic Engineering** ([lindixu6-hash/awesome-agentic-engineering](https://github.com/lindixu6-hash/awesome-agentic-engineering)): GitHub Action and CLI that scores an agent specification against risk-tiered profiles, fails a release on unresolved blockers, and runs eight prompt-injection fixtures through LangGraph and OpenAI Agents adapters that retain node-linked traces and machine-readable results. MIT-licensed, 2026-present.

**\[Python\] AgentCheck** ([WaseemGhanem98/AgentCheck](https://github.com/WaseemGhanem98/AgentCheck)): open-source behavioral testing tool for tool-using AI agents that simulates declared tool execution while evaluating tool calls, failures, retries, confirmations, destructive actions, action sequencing, and fabricated success, with integrations for OpenAI Agents SDK, PydanticAI, and custom Python agents. Apache-2.0, 2026-present.

---

## Standards and Governance

Standards and governance instruments for agent reliability and accountability. Papers appear first, then the standards and frameworks.

### Standards and Frameworks

**\[Standard\] Model Context Protocol (MCP) Specification** ([modelcontextprotocol.io](https://modelcontextprotocol.io/specification/latest), [spec repo](https://github.com/modelcontextprotocol/modelcontextprotocol)): JSON-RPC 2.0 protocol that standardizes how applications expose tools, resources, and prompts to language models, with an optional OAuth 2.1 authorization profile requiring Protected Resource Metadata discovery (RFC 9728), resource-indicator audience binding (RFC 8707), and issuer validation (RFC 9207) on HTTP transports. Agentic AI Foundation (a directed fund under the Linux Foundation), revision 2026-07-28.

**\[Standard\] Agent2Agent (A2A) Protocol** ([a2a-protocol.org](https://a2a-protocol.org/v1.0.1/specification/)): task delegation between independent agents, where each server publishes an Agent Card declaring identity, capabilities, skills, endpoint, and authentication across API key, HTTP, OAuth 2.0, OpenID Connect, and mutual TLS. Linux Foundation, contributed by Google, v1.0.1 (28 May 2026).

**\[Standard\] Agent Payments Protocol (AP2)** ([ap2-protocol.org](https://ap2-protocol.org/)): users sign Checkout and Payment Mandates as verifiable credentials, with an open stage carrying spending constraints and a closed stage carrying a specific authorization, so each agent-initiated transaction leaves a non-repudiable cryptographic record. FIDO Alliance, donated by Google, v0.2.

**\[Standard\] OpenTelemetry Semantic Conventions for Generative AI (gen-ai)** ([semantic-conventions-genai](https://github.com/open-telemetry/semantic-conventions-genai)): vendor-neutral telemetry schema for GenAI and agent spans, metrics, and events, covering model calls, tool calls, token usage, and agent and framework operations to support comparison of traces across stacks. OpenTelemetry / CNCF; all `gen_ai.*` attributes were deprecated in the core semantic-conventions repository at v1.42.0 (12 June 2026) and moved to this dedicated repository.

**\[Standard\] NIST AI 600-1, Generative AI Profile** ([nvlpubs.nist.gov](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf)): cross-sector companion to the NIST AI RMF that names twelve GenAI risk areas and over 200 suggested actions across the Govern, Map, Measure, and Manage functions. NIST, July 2024.

**\[Standard\] NIST AI 100-1, AI Risk Management Framework (AI RMF 1.0)** ([nvlpubs.nist.gov](https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf)): the base voluntary framework that organizes AI trustworthiness and risk work into the Govern, Map, Measure, and Manage functions; the parent document the GenAI profile extends. NIST, January 2023.

**\[Standard\] EU AI Act (Regulation 2024/1689), Article 12: Record-Keeping** ([artificialintelligenceact.eu](https://artificialintelligenceact.eu/article/12/)): legal duty requiring high-risk AI systems to record events throughout their lifetime for risk identification, post-market monitoring, and operational oversight. Article 26(6) requires deployers to retain the logs for at least six months. EU, 2024; the Digital Omnibus on AI ([Regulation (EU) 2026/1744](https://eur-lex.europa.eu/eli/reg/2026/1744/oj), in force 27 July 2026) moved high-risk application to 2 December 2027 for Annex III areas and 2 August 2028 for AI embedded in regulated products.

**\[Standard\] ISO/IEC 42001:2023, Artificial Intelligence Management System** ([iso.org](https://www.iso.org/standard/42001)): first certifiable AI management system standard, setting requirements to establish, operate, and improve governance controls across the AI lifecycle, including risk assessment, impact assessment, and performance measurement. ISO/IEC JTC 1/SC 42, 2023.

**\[Standard\] MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems)** ([atlas.mitre.org](https://atlas.mitre.org/), [atlas-data v2026.07](https://github.com/mitre-atlas/atlas-data/releases/tag/v2026.07)): living knowledge base of adversary tactics and techniques against AI systems, modeled on MITRE ATT&CK and built from real-world attacks and red-team demonstrations. MITRE, v2026.07 (7 August 2026): 16 tactics, 101 techniques, 77 sub-techniques, 37 mitigations, and 68 case studies, adding three AI Agent Tool Poisoning sub-techniques (AML.T0110.000/.001/.002) and AML.T0115.

**\[Standard\] OWASP Top 10 for LLM Applications and OWASP Top 10 for Agentic Applications** ([genai.owasp.org](https://genai.owasp.org/llm-top-10/), [agentic 2026](https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/), [securing agentic applications 1.0](https://genai.owasp.org/resource/securing-agentic-applications-guide-1-0/)): community-maintained risk lists for language-model and agentic applications, covering prompt injection, excessive agency, tool misuse, memory and context poisoning, and goal hijacking, with a builder-side companion guide of technical recommendations. OWASP, 2025 (LLM) / 2026 (Agentic).

**\[Standard\] CWE-1427: Improper Neutralization of Input Used for LLM Prompting** ([cwe.mitre.org](https://cwe.mitre.org/data/definitions/1427.html)): base-level weakness entry cataloging prompt injection as a mappable software weakness, with a LangChain tool-agent demonstrative example. MITRE CWE 4.20, mapping status ALLOWED.

**\[Framework\] Agentic AI Threat Modeling Framework: MAESTRO** ([cloudsecurityalliance.org](https://cloudsecurityalliance.org/blog/2025/02/06/agentic-ai-threat-modeling-framework-maestro)): seven-layer threat-modeling method a builder can run against an agent architecture. Cloud Security Alliance, 2025; a single-author blog publication rather than a ratified standard, so it sits at a lower evidentiary register than the NIST, ISO, MITRE, and OWASP entries above.

**\[Standard\] C2PA Technical Specification / Content Credentials** ([spec.c2pa.org](https://spec.c2pa.org/specifications/specifications/2.2/specs/_attachments/C2PA_Specification.pdf), [c2pa-rs](https://github.com/contentauth/c2pa-rs)): cryptographically signed, tamper-evident metadata standard that records the origin and edit history of media, including a manifest for AI-generated and AI-edited content. C2PA (Adobe, Microsoft, BBC, Intel, Truepic, Sony, and others), v2.2 (2025).

The five entries below are the primary sources for the verifiable-log and attestation patterns that
several tools in this list build on.

**\[Standard\] Certificate Transparency (RFC 6962, RFC 9162)** ([RFC 6962](https://www.rfc-editor.org/rfc/rfc6962), [RFC 9162](https://www.rfc-editor.org/rfc/rfc9162)): append-only Merkle tree log with inclusion proofs, showing an entry is in the log, and consistency proofs, showing the log was only appended to; the origin of the verifiable-log pattern that tamper-evident agent trails reuse. IETF, 2013 (v1) / 2021 (v2).

**\[Standard\] in-toto Attestation Framework** ([in-toto/attestation](https://github.com/in-toto/attestation), [in-toto.io](https://in-toto.io/)): signed, machine-readable statements binding a subject artifact to a predicate describing what was done to it, giving a common envelope for provenance claims across steps and tools. in-toto / CNCF, 2018-2026 (living spec).

**\[Standard\] SLSA (Supply-chain Levels for Software Artifacts)** ([slsa.dev](https://slsa.dev/spec/v1.0/)): graded framework defining what provenance a build must produce and how tamper-resistant it must be, with each level naming concrete requirements rather than intent. OpenSSF, v1.0 (2023).

**\[Standard\] DSSE (Dead Simple Signing Envelope)** ([secure-systems-lab/dsse](https://github.com/secure-systems-lab/dsse)): signing envelope that authenticates a payload together with its type, removing the canonicalization ambiguity that lets a signature be replayed against a different interpretation of the same bytes; used by in-toto and Sigstore. Secure Systems Lab, 2021-2026.

**\[Tool\] Rekor** ([sigstore/rekor](https://github.com/sigstore/rekor)): transparency log service for signed software artifacts and attestations, built on an append-only Merkle log and serving inclusion and consistency proofs over a public API. Sigstore / OpenSSF, Apache-2.0, 2020-present.

---

## Contributing

More items will be added to this repository. Please suggest other resources by opening an issue, submitting a pull request, or dropping me an email at yzhao010@usc.edu. New entries should be real, with a working link to the paper, repository, or specification.

## Contributors

People outside the lab have added entries to this list. Each of those went through a pull
request, met the inclusion bar in `CONTRIBUTING.md`, and is checked by the same link and
citation audit as everything else here.

The strip reads from the repository itself, so it stays current without anyone maintaining a
list by hand.

---

## Maintained By

Maintained by [Yue Zhao](https://github.com/yzhao062), University of Southern California, [USC-FORTIS](https://github.com/USC-FORTIS) lab. He created [PyOD](https://github.com/yzhao062/pyod) and co-authored [ADBench](https://github.com/Minqi824/ADBench). His publication list is on [Google Scholar](https://scholar.google.com/citations?user=zoGDYsoAAAAJ&hl=en).

---

## Citation

If this list is useful in your work, cite the *Auditable Agents* framework paper:

```bibtex
@misc{nian2026auditable,
  title         = {Auditable Agents},
  author        = {Nian, Yi and Yuan, Aojie and Zhang, Haiyue and Li, Jiate and Zhao, Yue},
  year          = {2026},
  eprint        = {2604.05485},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  note          = {Accepted to the ACM AI Leadership Summit 2026},
  url           = {https://arxiv.org/abs/2604.05485}
}
```

To cite the list itself as a resource:

awesome-auditable-ai

오너의 다른 레포

파일 탐색기

# CDN으로 사용하기

Was this content helpful?

Recent Feedback

awesome-auditable-ai

오너의 다른 레포

파일 탐색기

# CDN으로 사용하기

Was this content helpful?

Recent Feedback

Download README