pino - GitLite

KO | EN

gitlite — search

#javascript #hacktoberfest #ai #go #machine-learning #rust #linux #nodejs #windows #awesome #cli #awesome-list #vue #open-source

pino

(★ 54)

Anthropic API reverse proxy with prompt-cache injection and request body transforms

File Explorer

Download Latest Version (.zip)

- CODEOWNERS
- pino-proxy.js
- pino.png
- ab-cache-harness.mjs
- usage-tally.mjs
- - default.js
- cache.js
- config.js
- logger.js
- model.js
- server.js
- cache.test.js
- fixtures.js
- transform.test.js
.gitignore
CLAUDE.md
LICENSE
package.json
README.md

# Installation Guide

1. Get the code

git clone https://github.com/alxsuv/pino

Downloads the entire project code from GitHub to your computer.

cd pino

Moves into the project folder you just downloaded.

2. Node.js

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Node.js Node.js must be installed to use npm. The LTS version is recommended.

LOG_BODIES=1 npm start

Starts the development/run server.

npm start

Starts the development/run server.

AUTO_CACHE=1 LOG_BODIES=1 npm start

Starts the development/run server.

test/ # node --test suite (run: npm test)

Type this command into your terminal and run it.

✅ After running the command, open the address shown in the terminal (usually something like http://localhost:3000) in your browser.

Pulled directly from this repo's README.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<div align="center">
  <img src="./logo/pino.png" alt="Pino proxy" width="250" />

# Pino proxy

[![License](https://img.shields.io/github/license/alxsuv/pino)](https://raw.githubusercontent.com/alxsuv/pino/main/LICENSE)
[![Node](https://img.shields.io/badge/node-%E2%89%A5%2020-43853d?logo=node.js&logoColor=white)](https://nodejs.org)
[![Zero dependencies](https://img.shields.io/badge/dependencies-0-brightgreen)](https://raw.githubusercontent.com/alxsuv/pino/main/package.json)
[![GitHub stars](https://img.shields.io/github/stars/alxsuv/pino?style=social)](https://github.com/alxsuv/pino/stargazers)

[![Cache reads at 0.1x](https://img.shields.io/badge/cache-0.1%C3%97-blueviolet?style=for-the-badge&logo=anthropic&logoColor=white)](https://raw.githubusercontent.com/alxsuv/pino/main/)

**A tiny local proxy to inspect, cache, and reshape Anthropic API requests.**

</div>

> **See exactly what your Claude client ships — then cache and trim what it leaves on the table.** Zero runtime dependencies, no build step, no SaaS. A ~500-line local reverse proxy in front of `api.anthropic.com`.

A tiny local HTTP reverse proxy in front of `api.anthropic.com`. It forwards everything to upstream untouched **except** `/v1/messages` requests, where it optionally:

- **Logs raw request/response bodies** so you can see precisely what Claude Code (or any client) sends — tool schemas, system blocks, cache breakpoints, TTLs. This observability is how every optimization below was found, and it's the proxy's most durable use.
- **Auto-injects prompt-cache breakpoints** on the chunks the client leaves uncached — the `tools` catalog (still shipped with zero breakpoints) and the static reminders block in `messages[0]`.
- **Normalizes cache TTL** to 1h on stable content and reclaims breakpoint slots the client wastes on tiny blocks.
- **Reshapes the request body** via a user-supplied `transform(body)` hook — drop unused tools, strip ANSI escapes, override the model, rewrite system text.

## Origin story

Pino started with exactly one job. Claude Code was shipping a large tool catalog (~25–29 schemas, ~15k tokens at its peak) **uncached on every turn**, so Pino injected a single prompt-cache breakpoint and turned that re-shipped block into a **0.1× cache read instead of a 1× re-bill** — up to ~90% off on that chunk. That was the whole pitch: a 500-line proxy that cached what the client forgot to.

Then Anthropic shipped **progressive tool disclosure** in **v2.1.172** (mid-June 2026). The inline catalog shrank (via `ToolSearch` lazy-loading) to ~5k tokens and system blocks gained a 1h TTL. The per-turn numbers changed; the mechanism didn't — that uncached catalog and the `messages[0]` anchor still ship at full price every turn.

The caching mission is intact — Anthropic shrank the prefix but still ships it uncached, so Pino still flips it to a 0.1× read every turn. And the proxy was always more than that one number: **it's a local seam where you can see and rewrite anything in a `/v1/messages` request.**

- Want to **override the model**? One env var (`MODEL_OVERRIDE`) — it even rewrites the model's name in the system prompt so its self-description stays consistent.
- Want to **rewrite prompts or drop tools**? A `transform(body)` hook gets the parsed body before it leaves your machine.
- Want to **investigate** what your client actually sends? `LOG_BODIES=1` dumps every request and response to disk — which is how this entire writeup was produced.
- Still want the **caching**? It's still there: the tool catalog + `messages[0]` anchor are shipped uncached on every turn, and Pino caches them.

The 90% headline was the spark; the engine is a general-purpose inspect-and-modify proxy for Anthropic API traffic — caching included.

> ### Status: per-turn overhead changed across Claude Code v2.1.158–183 — see [how the overhead evolved](#how-claude-codes-request-overhead-evolved)
>
> Over spring–summer 2026 Anthropic trimmed Claude Code's per-request overhead in **two waves** (measured from captured logs): the **system prompt** was cut ~3× around **v2.1.158**, then **progressive tool disclosure** (`ToolSearch`, **v2.1.172**) shrank the inline tool catalog from a ~15–16k-token peak to ~5k. System blocks now also ship at `ttl: "1h"`. The tool catalog and the `messages[0]` anchor are still shipped uncached, and the logging/transform features are client-version-independent. The numbers below are measured from the logs and labeled by era.

## How Claude Code's request overhead evolved

This project started because Claude Code re-shipped a large, **uncached** tool catalog on every request. Capturing raw request bodies across versions (Apr–Jun 2026) shows Anthropic trimmed the per-turn overhead in **two separate waves**, not one — and confirms the single gap that survived all of them. Token counts are calibrated against the real `usage` numbers in the paired responses (~0.23 tok/char for this payload); the catalog scales with loaded MCP servers, so treat these as a standard config.

| Version (date)        | Inline `tools`                    | `system`       | `tools` breakpoints | What changed                       |
|-----------------------|-----------------------------------|----------------|---------------------|------------------------------------|
| v2.1.119 (Apr 27)     | 28 schemas, ~15.2k tok            | ~6.5k tok      | **0**               | baseline                           |
| v2.1.146              | 29 schemas, **~16.6k tok** (peak) | ~6.7k tok      | **0**               | catalog peak                       |
| v2.1.158 (late May)   | 28 schemas, ~12.9k tok            | **~2.3k tok**  | **0**               | ⬅ **system prompt cut ~3×**        |
| v2.1.169 (Jun 9)      | 25 schemas, ~11.6k tok            | ~1.7k tok      | **0**               | last pre-`ToolSearch`              |
| **v2.1.172 (Jun 12)** | **lazy-loaded via `ToolSearch`**  | ~1.9k tok      | **0**               | ⬅ **progressive tool disclosure**¹ |
| v2.1.177              | ~15 inline, ~7.4k tok             | ~1.9k tok      | **0**               | catalog still mid-shrink           |
| v2.1.183              | ~8 inline, ~5.3k tok              | ~2.7k tok (1h) | **0**               | settled                            |

¹ gated behind the `advanced-tool-use-2025-11-20` beta flag.

Three things stand out:

1. **The catalog peaked at ~16.6k tokens (v2.1.146)** — not the "~24k" earlier back-of-envelope estimates claimed. Peak *static prefix* (tools + system + msg[0]) was ~26.6k tokens at v2.1.119, down to ~10k by v2.1.183.
2. **Two waves, weeks apart.** The **system prompt** was cut ~3× around **v2.1.158** (~6.7k → ~2.3k tok), well before any tool change. Then **`ToolSearch` (v2.1.172)** attacked the tool catalog — and even that shrank in stages (v2.1.177 still inlined ~15 tools).
3. **`tools` carried zero cache breakpoints in *every single version*, Apr → Jun.** System gained a 1h TTL; the tool catalog never gained a breakpoint. **That is the durable gap Pino fills** — alongside the still-uncached `messages[0]` reminders and a breakpoint slot the client still wastes on a ~57-char system block.

**Bottom line:** On current Claude Code (v2.1.183), Pino reclaims roughly **~7.5k uncached tokens/turn** (tool catalog ~5.3k + msg[0] anchor ~2.4k) plus the wasted tiny-system slot — the pre-`ToolSearch` peak was ~20k. Confirm it for *your* version with the [60-second check](#verify-it-yourself-60-seconds) — catalog size and TTL behavior vary by version and loaded MCP servers.

## Quickstart

### 1. Clone and install

```bash
git clone https://github.com/alxsuv/pino
cd pino
```

No `npm install` needed — zero runtime dependencies. Requires Node >= 20.

### 2. Start the proxy

**Linux / macOS (bash/zsh):**

```bash
# pure pass-through on :8787 (great for just inspecting traffic)
LOG_BODIES=1 npm start

# caching + transforms + logs
AUTO_CACHE=1 \
TRANSFORM_FILE=./src/transforms/default.js \
LOG_BODIES=1 \
npm start

# or invoke the bin directly
node bin/pino-proxy.js
```

**Windows (PowerShell):**

```powershell
$env:AUTO_CACHE=1
$env:TRANSFORM_FILE="./src/transforms/default.js"
$env:LOG_BODIES=1
npm start
```

**Windows (cmd.exe):**

```cmd
set AUTO_CACHE=1
set TRANSFORM_FILE=./src/transforms/default.js
set LOG_BODIES=1
npm start
```

### 3. Point your client at it

**Linux / macOS:**

```bash
export ANTHROPIC_BASE_URL=http://127.0.0.1:8787
```

**Windows (PowerShell):**

```powershell
$env:ANTHROPIC_BASE_URL="http://127.0.0.1:8787"
```

**Windows (cmd.exe):**

```cmd
set ANTHROPIC_BASE_URL=http://127.0.0.1:8787
```

## Verify it yourself (60 seconds)

You don't have to take any number here on faith — the proxy's whole point is that you can see what your client actually ships in 60 seconds, on your own version. (Pre-`ToolSearch` this check surfaced a ~15k-token tool catalog re-shipped uncached every turn; the current catalog is ~5k. The only way to know what *your* client and version do is to look.)

1. **Start the proxy in pass-through mode** (captures logs, mutates nothing):
   ```bash
   LOG_BODIES=1 npm start
   ```

2. **Run any command in Claude Code** (e.g., `claude "hi"`).

3. **Inspect the captured request**:
   ```bash
   # How big is the inline tool catalog, and is it cached?
   jq '{tools: (.body.tools|length), uncached: ([.body.tools[]|select(.cache_control)]|length)}' logs/*.req.json

# System blocks: are TTLs set? Any slot wasted on a tiny block?
   jq '.body.system[]? | {len: (.text|length), cache_control}' logs/*.req.json

# Is the messages[0] anchor (CLAUDE.md / reminders) cached?
   jq '[.body.messages[0].content[]?|objects|select(.cache_control)]|length' logs/*.req.json
   ```

On v2.1.172 you'll typically see: ~8 tools, **0** cached; system blocks at `ttl: "1h"` (with one breakpoint wasted on a ~57-char block); and `messages[0]` with **0** breakpoints. The tool catalog and the msg[0] anchor are the two chunks still left on the table.

> **Heads up:** the inline catalog isn't always small. On the occasional turn where `ToolSearch` actually loads a deferred tool, its full schema is shipped inline for that request — in one captured corpus, ~1 request in 40 ballooned from ~14 KB back to ~48 KB (21 tools). So don't be alarmed by an outlier; check several `logs/*.req.json`, not just one.

4. **Enable caching** and re-run the queries:
   ```bash
   AUTO_CACHE=1 LOG_BODIES=1 npm start
   ```
   You'll now see a breakpoint added to `tools`, every ephemeral block normalized to `ttl: "1h"` (except the rolling tail), the wasted tiny-block slot reclaimed, and `anthropic-beta` carrying `extended-cache-ttl-2025-04-11`.

5. **Run another command and watch the hits** in the response `usage` field — `cache_read_input_tokens` should now cover the tool catalog and msg[0] anchor instead of re-billing them.

## How the caching works

The Anthropic API allows up to **4 cache breakpoints** per request. Each tells the API "cache everything up to and including this block," so subsequent requests with the same prefix hit the cache at 0.1× base input price instead of 1×.

This proxy places them as follows (within the 4-slot ceiling):

1. **Last `tools` entry** → 1h TTL. The client still ships **zero** breakpoints on `tools` (true in every version Apr–Jun). Pre-`ToolSearch` this was a ~15k-token catalog (~16.6k at the v2.1.146 peak) and the single biggest win; on v2.1.183 it's ~5.3k tokens — uncached, and the biggest single chunk Pino still caches.
2. **Last `system` block** → 1h TTL, **and** strip wasteful breakpoints on system blocks <500 chars. On current Claude Code the system prompt already arrives at `ttl: "1h"`, so the TTL rewrite is mostly a no-op there — but the client still burns a full slot on a ~57-char block (caching ~14 tokens), which this pass reclaims for the msg[0] reminders instead.
3. **Last cacheable block of `messages[0]`** → 1h TTL. The client stuffs static reminders (CLAUDE.md, skills catalog, deferred-tools list — ~1.7k+ tokens) into the first user message and leaves them uncached. Cached once per session. **Still a real win on v2.1.172.**
4. **Rolling tail** → 5m TTL by default (`TAIL_TTL=1h` to change). The last `text`/`tool_result`/`image` block across all messages. Moves each turn, so every new turn reads the prior turn's prefix from cache and only pays base price for the delta. 5m is the right default because the tail rarely survives an hour of reuse; paying the 2.0× write multiplier on a breakpoint that moves constantly is wasteful. (Recent Claude Code versions also place a 1h breakpoint near the tail themselves.)

## Decide if caching pays off for *your* workload

The per-turn payoff depends on your client version and tool set. To check yours, Pino ships two zero-dependency analysis scripts that turn the `LOG_BODIES` dumps into hard numbers.

- **`scripts/usage-tally.mjs`** — tallies the real billed `usage` (input / cache_create / cache_read / output, with the per-TTL 5m-vs-1h split) from a directory of `*.resp.log`. Run a proxy-on session and a direct (pass-through) session over the same task, point it at both, and it diffs the actual tokens:
  ```bash
  node scripts/usage-tally.mjs logs/run-proxy-on logs/run-direct
  ```
- **`scripts/ab-cache-harness.mjs`** — replays a captured corpus through five breakpoint/TTL strategies against a simulated prompt cache, so you can compare designs without spending a token:
  ```bash
  node scripts/ab-cache-harness.mjs logs/<your-corpus>
  ```

What the numbers say on a modern (v2.1.183) client:

- **The shipped caching design holds up.** Across every alternative tried — selective 1h, dropping the rolling tail, breakpoints-only — the current strategy (1h on the stable `tools`/`system`/`messages[0]` prefix + a 5m rolling tail) was the best of the *intervention* options. The rolling tail does most of the work; removing it reprocesses all history at full price every turn.
- **Dense, back-to-back traffic is the one profile to check first.** When turns arrive faster than the client's 5m cache expires, its own breakpoints already cache well, so Pino's reshaping + 1h writes can cost marginally more than pass-through — run the harness before enabling `AUTO_CACHE` there.
- **The proxy's durable edge is idle gaps.** Its 1h TTL on the stable prefix survives pauses >5 min where the client's 5m cache expires and would re-bill the whole prefix. The more your sessions stall (reading, thinking, stepping away), the more `AUTO_CACHE` pays; the more they're a dense tool-call storm, the closer pass-through gets.

Bottom line: don't guess — run the two scripts on your own traffic and let the `weighted cost Δ` line decide whether you set `AUTO_CACHE=1`.

## Savings math

> **Read this first.** The table below is the **pre-`ToolSearch` peak** case (v2.1.119–146, ~Apr–May 2026) — the worst the overhead ever got, and the case that motivated this project. Numbers are measured from captured request bodies, calibrated to the real token `usage` in the paired responses. **On current Claude Code (v2.1.183) the reclaimable amount is ~7.5k tokens/turn (tool catalog + msg[0] anchor), so real-world input savings are typically single-digit to low-double-digit percent, not 90%.** Always measure your own version with the [60-second check](#verify-it-yourself-60-seconds).

Anthropic pricing multipliers (all relative to base input):

| Operation             | Multiplier |
|-----------------------|------------|
| Base input (uncached) | 1.0×       |
| Cache write, 5m TTL   | 1.25×      |
| Cache write, 1h TTL   | 2.0×       |
| Cache read (hit)      | 0.1×       |

### Per-round-trip savings — *peak pre-`ToolSearch`* (Claude Sonnet, $3 / M input)

A "turn" means **one HTTP round trip to `api.anthropic.com`**, not one user message. (One user message expands into many round trips.) Numbers are measured from a v2.1.119–146-era request, steady-state mid-session, standard tool set:

| Chunk                            | Tokens (measured) | Client cached it?      | Without proxy  | With Pino (cache read) | Saving/turn                 |
|----------------------------------|-------------------|------------------------|----------------|------------------------|-----------------------------|
| `tools` (28–29 schemas)          | ~15,000           | **No** (0 breakpoints) | $0.0450        | $0.0045                | **$0.0405**                 |
| `messages[0]` reminders          | ~5,000            | **No**                 | $0.0150        | $0.0015                | **$0.0135**                 |
| `system`                         | ~6,500            | Yes (client 5m)        | already cached | bump → 1h              | TTL bump + slot reclaim     |
| **Reclaimed by Pino (uncached)** | **~20,000**       |                        | **$0.060**     | **$0.0060**            | **~$0.054 (−90% on these)** |

The −90% is the cache-read multiplier (1× → 0.1×) applied to the chunks the client shipped **uncached** — the tool catalog and the `messages[0]` anchor. The `system` block was already client-cached (at 5m), so Pino only bumps its TTL and reclaims the slot wasted on a ~57-char block. Prior-turn history is cached incrementally by the rolling tail and, on modern versions, is largely the client's own job — it is *not* a fixed per-turn line item (the A/B harness in [`scripts/`](#decide-if-caching-pays-off-for-your-workload) shows reshaping it can even add churn).

On **v2.1.183**, the `tools` row shrinks to ~5.3k tokens and `system` is already 1h-cached upstream, leaving `tools` + `messages[0]` (~7.5k tok) as the reclaimable portion — roughly a third of the peak headline.

### Round-trip multiplier

The Anthropic API is **stateless**: every HTTP request carries the full conversation context (system + tools + prior messages + tool results). Claude Code makes **one round trip per tool call**, so a single prompt like "fix this bug" routinely expands into 10–50+ round trips. Any per-call input saving is multiplied by that count — which is why the per-call delta compounds over a long session, and why caching the prefix (so each round trip is a 0.1× read instead of a 1× re-ship) matters more than trimming.

> **Note on billing models.** Dollar figures apply to **pay-as-you-go API usage** (`x-api-key`). On a **Pro / Max subscription** the monthly fee doesn't change, but cached tokens count fractionally against your rate-limit window, so caching still buys you session length. (Note that the most acute version of this — the 5m TTL regression discussed in the community — was largely a server-side TTL-tier issue, not something a client breakpoint alone fixes.)

## Request-size savings (`DROP_TOOLS` + ANSI strip)

> **v2.1.172 note:** with progressive tool disclosure, most tools are no longer shipped inline — only ~8 core tools are in `body.tools`, the rest are lazy-loaded via `ToolSearch`. So `DROP_TOOLS` now matches far fewer tools than the table below assumes; check `logs/*.req.json` to see what your client actually inlines before configuring it.

Independent of caching, body mutations reduce wire size:

- `DROP_TOOLS=Name1,Name2` → removes those tools from `body.tools` **and** scrubs their names from the deferred-tools system reminder. Worth ~100–800 tokens per inlined tool dropped.
- `STRIP_ANSI=1` (default) → strips SGR escapes from `/context` output, terminal colors, etc. Roughly halves `tool_result` size on affected turns (~500–2,000 tokens).
- `TRIM_BASH_GIT=1` → drops the git-commit + PR-creation subsections of the Bash tool description. ~1,800 tokens saved if you don't use git through Claude Code.

Check `logs/*.req.json` after a turn to see which tools your client actually ships — the catalog varies by Claude Code version and which MCP servers you have loaded.

## Environment variables

| Var              | Default  | What it does                                                                                                                                                                                   |
|------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `PORT`           | `8787`   | Local port to bind (always `127.0.0.1`).                                                                                                                                                       |
| `AUTO_CACHE`     | off      | Enable cache breakpoint injection + 1h TTL rewrite + beta header.                                                                                                                              |
| `TAIL_TTL`       | `5m`     | TTL for the rolling-tail breakpoint. `5m` (recommended) or `1h`. Other slots are always 1h.                                                                                                    |
| `TRANSFORM_FILE` | —        | Path to a JS module exporting `transform(body)` for custom body edits.                                                                                                                         |
| `DROP_TOOLS`     | —        | Comma-separated tool names to remove from `body.tools` *(requires `TRANSFORM_FILE=./src/transforms/default.js`)*. Matches fewer tools on v2.1.172+ (most are deferred).                        |
| `STRIP_ANSI`     | `1`      | Strip ANSI escapes from message text + tool results. Set to `0` to disable.                                                                                                                    |
| `TRIM_BASH_GIT`  | `0`      | Truncate the Bash tool description at its "Committing changes" section.                                                                                                                        |
| `MODEL_OVERRIDE` | —        | Force a different model on every `/v1/messages` request (e.g. `claude-opus-4-8`). Also rewrites model-name references inside `system` blocks so the model's self-description stays consistent. |
| `LOG_BODIES`     | off      | Dump post-mutation request JSON + raw response bytes to `LOG_DIR`.                                                                                                                             |
| `LOG_DIR`        | `./logs` | Where to write body dumps.                                                                                                                                                                     |

## Architecture in 30 seconds

```
bin/pino-proxy.js          # CLI entry (shebang)
src/server.js              # HTTP server + request handler, exports startServer()
src/config.js              # env parsing, constants, transform loader
src/cache.js               # breakpoint inject/rewrite, beta header
src/model.js               # model-name rewrites for system-prompt self-description
src/logger.js              # timestamps, sanitizers, request/response log writers
src/transforms/default.js  # default body mutator (env-driven tools/ANSI stripping)
scripts/                   # analysis benches: usage-tally.mjs, ab-cache-harness.mjs
test/                      # node --test suite (run: npm test)
```

- `src/server.js` — HTTP server on `127.0.0.1:$PORT`. Buffers request bodies, parses JSON on matching paths, runs transform → inject → rewrite → beta-header pipeline, streams responses through.
- `src/transforms/default.js` — Default body mutator. Handles tool drops, ANSI stripping, and context-restructuring.
- Logging: `LOG_BODIES=1` writes `<reqId>.req.json` (post-mutation, auth redacted) + `<reqId>.resp.log` (raw response) per request.

See [CLAUDE.md](./CLAUDE.md) for full internals, order of operations, gotchas, and pointers for extending the transform pipeline.

## Caveats

- The proxy binds to `127.0.0.1` only — not reachable from other hosts.
- Header passthrough is verbatim: `x-api-key` / `authorization` go upstream as-is (redacted only in logs).
- Savings depend heavily on your Claude Code version. Since v2.1.172, the client caches more on its own; **measure your own traffic** with `LOG_BODIES=1` rather than trusting the historical tables.
- Pino is an unofficial tool that inspects and mutates request bodies, so it tracks the current shape of Claude Code's requests. When that shape changes (as it did at v2.1.172), adapting Pino is a transform edit, not a rewrite.