DRIFT - GitLite

KO | EN

gitlite — search

#list #framework #developer-tools #javascript #claude-code #api #swift #linux #tailwindcss #animation

DRIFT

(★ 23)

Design for Error Detection in Deep-Research Agents Trajectories.

파일 탐색기

ZIP 다운로드

- README.md
- index.html
- styles.css
- USAGE.md
- decrypt_telbench.sh
- - __init__.py
  - __main__.py
  - cli.py
  - client.py
  - data.py
  - env.py
  - evaluate.py
  - io.py
  - models.py
  - prompts.py
  - runner.py
  - sanitize.py
  - span_store.py
  - utils.py
- test_release_contract.py
.gitignore
pyproject.toml
README.md

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

명령어 용어집

이 문서에서 사용된 명령어를 모아봤습니다. 낯선 명령어가 있다면 펼쳐서 확인해보세요.

🔍

`git clone`

설명 보기 ▼

Clone an existing repository.

git clone {{remote_repository_location}} {{path/to/directory}}

Clone an existing repository into a new directory (the default directory is the repository name):

git clone --recursive {{remote_repository_location}}

Clone an existing repository and its submodules:

git clone {{[-n|--no-checkout]}} {{remote_repository_location}}

Clone only the `.git` directory of an existing repository:

🔍

`python`

설명 보기 ▼

Python 언어 인터프리터.
더 많은 정보: <https://docs.python.org/using/cmdline.html>.

python

REPL(대화형 셸) 시작:

python {{path/to/file.py}}

특정 Python 파일 실행:

python -i {{path/to/file.py}}

특정 Python 파일 실행 후 REPL 시작:

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

> **Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories**

**NJU-LINK Team, Nanjing University · JIUTIAN Research**

<p align="center">
  <a href="https://arxiv.org/abs/2606.02060"><img src="https://img.shields.io/badge/arXiv-2606.02060-b31b1b.svg" alt="arXiv"></a>
  <a href="https://nju-link.github.io/DRIFT/"><img src="https://img.shields.io/badge/Project-Page-4f7cba.svg" alt="Project Page"></a>
  <a href="https://huggingface.co/datasets/NJU-LINK/TELBench"><img src="https://img.shields.io/badge/HuggingFace-TELBench-f6c343.svg" alt="TELBench"></a>
  <a href="https://github.com/NJU-LINK/DRIFT"><img src="https://img.shields.io/badge/GitHub-Code-24292f.svg" alt="GitHub"></a>
  <a href="docs/USAGE.md"><img src="https://img.shields.io/badge/Docs-Usage-6a8f5f.svg" alt="Usage Docs"></a>
</p>

DRIFT is a claim-centric auditing framework for diagnosing deep-research agent
trajectories. Instead of only checking whether the final answer is correct,
DRIFT localizes the semantic spans where an agent first makes, reuses, or
finalizes harmful unsupported claims.

We release:

- **TELBench**, a 1,000-instance benchmark for span-level error localization in
  real deep-research trajectories.
- **DRIFT**, a clean runner with two comparable settings: `bare` full-context
  prediction and the full claim-centric `drift` pipeline.
- **Project page**, figures, prompts, and reproducibility utilities.

<p align="center">
  <img src="docs/assets/images/mechanism_analysis.png" alt="TELBench mechanism analysis" width="92%">
</p>

## What We Study

Deep-research agents solve tasks through long trajectories of search, tool use,
evidence inspection, hypothesis testing, and answer synthesis. Final-answer
accuracy says whether the agent succeeds, but not which step made the trajectory
unreliable. TELBench turns this into a process-level task: given only the
question and ordered raw semantic spans, predict the harmful error span ids.

<p align="center">
  <img src="docs/assets/images/drift_overview.png" alt="DRIFT architecture" width="92%">
</p>

## DRIFT in Brief

DRIFT audits trajectories as claim graphs rather than independent spans:

1. **Claim Keeper** records decision-critical claims and when they become
   consequential.
2. **Support Seeker** uses graph-grep raw-span access to check whether those
   claims are supported by trajectory evidence.
3. **Dependency Tracer** localizes spans where unsupported claims become harmful
   commitments.

Model prompts receive only the task question and ordered raw span text. Gold
labels, annotations, judge results, span types, manual notes, and generated
summaries are stripped before prompting.

## Quick Start

```bash
git clone https://github.com/NJU-LINK/DRIFT.git
cd DRIFT
python -m pip install -e .
python -m pip install -U huggingface_hub

hf download NJU-LINK/TELBench \
  --repo-type dataset \
  --local-dir data \
  --include "TELBench.jsonl.enc" \
  --include "TELBench.jsonl.enc.sha256" \
  --include "TELBench.jsonl.sha256" \
  --include "TELBench.passphrase.txt"

bash scripts/decrypt_telbench.sh

cat > .env <<'EOF'
API_URL=https://example.com/codex
API_KEY=your_api_key_here
EOF

drift \
  --setting drift \
  --input data/TELBench.jsonl \
  --model gpt-5.4 \
  --api-type responses \
  --env-file .env \
  --outdir runs/telbench_gpt54 \
  --workers 8

drift-eval \
  --gold data/TELBench.jsonl \
  --pred runs/telbench_gpt54/drift/gpt-5.4/summary.json \
  --output runs/telbench_gpt54/drift/gpt-5.4/eval.json
```

For complete API, data, and evaluation details, see [docs/USAGE.md](docs/USAGE.md).
For the TELBench JSONL schema, see [data/README.md](data/README.md).

## Project Structure

```text
DRIFT/
├── data/
│   └── README.md              # download TELBench artifacts from Hugging Face
├── docs/
│   ├── index.html
│   ├── styles.css
│   ├── USAGE.md
│   └── assets/
├── scripts/
│   └── decrypt_telbench.sh
├── src/drift_open/
│   ├── cli.py
│   ├── client.py
│   ├── data.py
│   ├── evaluate.py
│   ├── prompts.py
│   ├── runner.py
│   └── span_store.py
├── tests/
└── pyproject.toml
```

## Links

- Project page: [https://nju-link.github.io/DRIFT/](https://nju-link.github.io/DRIFT/)
- Usage guide: [docs/USAGE.md](docs/USAGE.md)
- Data guide: [data/README.md](data/README.md)
- Code package: [src/drift_open](src/drift_open)

## Citation

```bibtex
@misc{wang2026deepresearchagentswrongspanlevel,
      title={Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories}, 
      author={Jiaming Wang and Ziteng Feng and Jiangtao Wu and Ruihao Li and Qianqian Xie and Yuxiang Ren and He Zhu and Xueming Han and Fanyu Meng and Junlan Feng and Jiaheng Liu},
      year={2026},
      eprint={2606.02060},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2606.02060}, 
}
```