user-as-code - GitLite

KO | EN

gitlite — search

#devops #pytorch #nodejs #open-source #security #c #list #hacktoberfest #ai #golang

user-as-code

(★ 12)

Executable memory for personalized agents: represent a user's memory as typed Python state + constraints an interpreter can run, instead of retrieved text. Code, experiments, and reference implementation for the paper (arXiv:2606.16707).

user-as-code 최신버젼 다운로드

ZIP 다운로드

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일 경로 직접 입력

링크

예시

명령어 용어집

이 문서에서 사용된 명령어를 모아봤습니다. 낯선 명령어가 있다면 펼쳐서 확인해보세요.

🔍

`done`

설명 보기 ▼

`for`, `while`, `select`, `until` 루프의 끝을 표시하는 쉘 키워드.

tldr for

`for` 키워드 문서 확인:

tldr while

`while` 키워드 문서 확인:

tldr select

`select` 키워드 문서 확인:

🔍

`npm install`

설명 보기 ▼

Install Node packages.

npm {{[i|install]}}

Install dependencies listed in `package.json`:

npm {{[i|install]}} {{package_name}}@{{version}}

Download a specific version of a package and add it to the list of dependencies in `package.json`:

npm {{[i|install]}} {{package_name}} {{[-D|--save-dev]}}

Download the latest version of a package and add it to the list of dev dependencies in `package.json`:

🔍

`npm run`

설명 보기 ▼

Run a script.

npm run

List available scripts:

npm run {{script_name}}

Run a script:

npm run {{script_name}} -- {{argument}} {{--option}}

Pass arguments to a script:

🔍

`python`

설명 보기 ▼

Python 언어 인터프리터.
더 많은 정보: <https://docs.python.org/using/cmdline.html>.

python

REPL(대화형 셸) 시작:

python {{path/to/file.py}}

특정 Python 파일 실행:

python -i {{path/to/file.py}}

특정 Python 파일 실행 후 REPL 시작:

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Research artifact for the paper **_User as Code: Executable Memory for Personalized Agents_** (Bojie Li, Pine AI).

- 📄 **Paper:** [arXiv:2606.16707](https://arxiv.org/abs/2606.16707) (LaTeX sources in this repo; build with `make`)
- 🌐 **Interactive companion site:** <https://01.me/research/user-as-code> — explore the supported benchmark cases plus separately labeled exploratory alert scenarios
- ⚖️ **License:** [Apache-2.0](LICENSE)

---

## The idea in one paragraph

Personalized agents need a **user memory**: a model of the user that accumulates across
conversations. Today that memory is stored as unstructured text, knowledge graphs, or flat
fact stores and consulted by **retrieval** (similarity search). Such "bag-of-facts"
memory recalls individual records well but struggles to enumerate every relevant record
for an aggregate query. **User as Code (UaC)** adds an executable typed-Python view over
the user's records while retaining fact and raw-archive retrieval. The evaluated system
uses a **two-phase write path**—an append-only fact log followed by periodic bounded
structuring—and exposes the typed view to ordinary Python for analytical queries.

## What's in this repository

| Path | What it is |
|------|------------|
| [`paper.tex`](paper.tex), `body*.tex`, [`reference.bib`](reference.bib), [`Makefile`](Makefile) | LaTeX sources for the paper (compiles with `arxiv.sty` + `plainnat`) |
| [`figures/`](figures/) | Paper figures (PDF) and the scripts that generate them |
| [`prototype/`](prototype/) | **Reference UaC implementation** — a worked example user (`jessica_thompson`) as typed domains + executable constraints + tests |
| [`experiments/`](experiments/) | Full experiment harness, the UaC pipeline (`user_as_code_v5.py`), baseline reimplementations, generated `results/`, and strict validators. See [`experiments/README.md`](experiments/README.md) |
| [`evaluation/`](evaluation/) | Deprecated proactive-alert pilots, the reported frozen v2.1 cue-free evaluation, and the post-evaluation v3 regression. See [`evaluation/README.md`](evaluation/README.md) |
| [`benchmarks/`](benchmarks/) | Fetch script + instructions for the third-party datasets (LOCOMO, LongMemEval). Raw data is **not** redistributed. See [`benchmarks/README.md`](benchmarks/README.md) |
| [`web/`](web/) | React companion site that visualizes every graded test case. See [`web/README.md`](web/README.md) |
| [`scripts/`](scripts/) | `build_site_data.py` — turns `experiments/results/` into the site's data bundles |
| [`user-as-code/`](user-as-code/) | Slidev slide deck (talk version of the paper) |

## Quick start

### Building the paper

```bash
make            # -> paper.pdf  (pdflatex + bibtex; needs a TeX Live install)
```

### Running the reference prototype

The fastest way to see "user as code" concretely — no API keys or datasets needed:

```bash
cd prototype
python runner.py                         # run every constraint, print the alerts
python -m pytest jessica_thompson/tests/ # validate constraint behavior
```

Each user is a self-contained Python project: `manifest.py` (compact always-loaded index),
`domains/` (typed dataclass schemas + state), `constraints/` (executable invariants that
return alerts), and `tests/`.

### Reproducing the experiments

```bash
# 1. install the pinned full-LOCOMO environment
python -m venv .venv-locomo
.venv-locomo/bin/pip install -r experiments/requirements-locomo-full.txt

# 2. fetch the benchmark datasets (LOCOMO downloads directly; LongMemEval is author-distributed)
./benchmarks/fetch_benchmarks.sh

# 3. configure the Krill endpoint and API key (do not commit the key)
export KRILL_API_KEY=...
export KRILL_BASE_URL=https://api.krill-ai.net/v1

# 4. run all seven systems under both full-LOCOMO backbones
systems=(full_context uac_v5 memmachine hindsight evermemos a_mem mem0)
for system in "${systems[@]}"; do
  KRILL_MODEL=gpt-5.6-luna \
    .venv-locomo/bin/python experiments/run_locomo_full.py "$system" \
      --run-name full_locomo_gpt56_luna
done
for system in "${systems[@]}"; do
  KRILL_MODEL=gemini-3-flash-preview \
    .venv-locomo/bin/python experiments/run_locomo_full.py "$system" \
      --run-name full_locomo_gemini3_flash_preview
done

# 5. certify all 14 artifacts; partial output is not paper-ready
.venv-locomo/bin/python experiments/summarize_locomo_full.py
```

The two full suites use `gpt-5.6-luna` and `gemini-3-flash-preview` through
`https://api.krill-ai.net/v1`. The reported Gemini run sends the exact User-Agent
`GeminiCLI/0.28.0/gemini-3-flash-preview (darwin; arm64; terminal)`. Runs write
inspectable per-question outputs under [`experiments/results/`](experiments/results/),
and the strict summarizer must print 14 rows plus `FINAL VALIDATION PASSED`
before any aggregate is reported. See the
[`experiments/README.md`](experiments/README.md) for scoring denominators,
provider-fallback disclosures, parallel execution, repair procedures, and the
full script-to-result map. Some older experiments use direct Gemini or
cross-family provider keys; they are documented separately there and are not
the full-LOCOMO reproduction path.

### Running the companion website

```bash
cd web
npm install
npm run dev      # http://localhost:5173
# data bundles are regenerated with:  python3 ../scripts/build_site_data.py
```

## Headline results

| Capability | Evaluation | UaC | Interpretation |
|------------|------------|-----|----------------|
| **Factual recall and refusal** | Full LOCOMO, all 1,986 questions; validated GPT-5.6 Luna / Gemini 3 Flash Preview suites | 39.4% / 46.9% token F1 and 78.8% / 80.6% judge accuracy on 1,540 answer-bearing questions; 64.3% / 95.7% refusal accuracy on 446 adversarial questions | lexical overlap, semantic correctness, and refusal behavior are reported separately within each backbone panel |
| **Analytical inference** | 100 deterministically scored aggregate queries | 100% accuracy | UaC and the raw-record FC+REPL reference both enumerate records through Python |

All fourteen full-LOCOMO artifacts pass the strict two-suite validator. See the
paper and experiment guide for the complete two-backbone table,
reimplementation caveats, provider-fallback disclosures, and validated
analytical results.

## Reproducibility notes

- **Version-controlled source:** experiment and validation scripts, the synthetic analytical
  benchmark, exploratory proactive-alert scenarios, and the reference prototype. The alert
  pilot is retained for future protocol repair and is not a reported benchmark.
- **Generated results:** per-run JSON artifacts are written under `experiments/results/`.
  We report a full LOCOMO run only after validating its coverage, provenance,
  stored scores, judge fields, and aggregates.
- **Not committed (regenerable / third-party):** the vector-index cache
  (`experiments/chroma_db/`), the raw benchmark datasets (`benchmarks/*/data/`, fetched via
  the script), and a few large LongMemEval-derived dumps (rebuilt by the pipeline). See the
  [`.gitignore`](.gitignore) for the exact list and the per-directory READMEs for how each is
  regenerated.

## Cite this work

If you use this work, please cite the paper ([arXiv:2606.16707](https://arxiv.org/abs/2606.16707)):

```bibtex
@article{li2026userascode,
  title         = {User as Code: Executable Memory for Personalized Agents},
  author        = {Li, Bojie},
  journal       = {arXiv preprint arXiv:2606.16707},
  year          = {2026},
  eprint        = {2606.16707},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  url           = {https://arxiv.org/abs/2606.16707}
}
```

## License

Code and documentation are released under the [Apache License 2.0](LICENSE) (see also
[`NOTICE`](NOTICE)). Third-party datasets (LOCOMO, LongMemEval) and memory libraries (Mem0,
A-MEM, MemMachine, EverMemOS, Hindsight) are governed by their own licenses and are not
included here.