laguna-s-2.1-nvfp4-sm121-vllm - GitLite

KO | EN

gitlite — search

#self-hosted #go #developer-tools #pytorch #typescript #json #react #proxy #macos #web

laguna-s-2.1-nvfp4-sm121-vllm

(★ 9)

Native SM121 vLLM runtime and reproducible qualification for Poolside Laguna S 2.1 NVFP4

File Explorer

ZIP Download

- registry.yaml
- dependency-manifest.json
- Dockerfile.checkpoint-update
- Dockerfile.production
- runtime-manifest.production.json
- architecture.md
- methodology.md
- troubleshooting.md
- - benchmark.html
  - MANIFEST.sha256
  - summary.json
  - VERDICT.md
- admit_node.sh
- audit_runtime.py
- benchmark.py
- build_release.sh
- check_dependencies.py
- context_probe.py
- entrypoint.sh
- eval_gsm8k.py
- profile_nvfp4.py
- public_safety_scan.py
- render_report.py
- resolve_vllm_release.py
- smoke.py
- verify_release.py
- - laguna-s-2.1-nvfp4-vllm-r0b0tlab.yaml
- README.md
- test_admission_contract.py
- test_benchmark_harness_scaffold.py
- test_dependency_check.py
- test_evaluation_tools.py
- test_launch_contract.py
- test_release_contract.py
- test_release_resolver.py
- test_verify_release.py
.dockerignore
.gitignore
AGENTS.md
LICENSE
PRIVACY.md
README.md

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

A production-oriented, fail-closed vLLM implementation for the exact Poolside checkpoint [`poolside/Laguna-S-2.1-NVFP4`](https://huggingface.co/poolside/Laguna-S-2.1-NVFP4) on NVIDIA GB10 / SM121.

This repository follows r0b0tlab's proven release-contract structure: immutable dependency/model identity, source-built runtime, audit-before-exec entrypoint, native-path gates, matched AR/speculative qualification, machine-readable evidence, and SparkRun packaging.

## Status

The production contract now follows Poolside's updated `0761412` "spinquantless norot weights, 256K" checkpoint while retaining the previously qualified vLLM 0.25.1, native SM121, FP8 KV, and DFlash K=7 profile. The checksummed [VERDICT](results/v0.25.1-gb10/VERDICT.md), [machine-readable results](results/v0.25.1-gb10/summary.json), [HTML report](results/v0.25.1-gb10/benchmark.html), and [full 8,620-case scorecard](results/v0.25.1-gb10/full-battery/REPORT.md) remain historical evidence for the initial `216d1f1` checkpoint; updated revision-scoped sanity and llama-benchy evidence is published separately.

## Exact model contract

| Component | Identity |
|---|---|
| Target | `poolside/Laguna-S-2.1-NVFP4@07614121b31898586430f189d27a25a0be310843` |
| Production draft | `poolside/Laguna-S-2.1-DFlash-NVFP4@723794750422b3efbf3a7b3af76dffb4ba035943` |
| KV cache | FP8 |
| Configured context | 262,144 tokens |
| Reasoning/tool parsers | `poolside_v1` |

The target is not interchangeable with Laguna BF16, FP8, INT4, GGUF, XS, or M checkpoints. The DFlash repository is only a speculative accelerator and never replaces the target model.

## Checkpoint 0761412 quick validation

The updated Poolside `spinquantless norot weights, 256K` checkpoint passed the revision-scoped native/API sanity suite and llama-benchy 0.4.0. All six API probes passed; native `FLASHINFER_CUTLASS` was active with zero active Marlin, emulation, or fallback markers; and DFlash K=7 produced positive drafted and accepted counters.

llama-benchy used a 2,048-token prompt, exactly 128 generated tokens, concurrency 1, three runs per point, and thinking disabled. Mean decode throughput was 22.23 tok/s at depth 0, 20.67 at 4K, 20.55 at 8K, and 19.88 at 16K. See the [checkpoint update report](results/v0.25.1-gb10/checkpoint-0761412/REPORT.md) and [machine-readable evidence](results/v0.25.1-gb10/checkpoint-0761412/).

This short validation does not relabel the original 8,620-case battery, which remains bound to target revision `216d1f13878dd4e715bc7412848d0f330e95bba6`.

## Runtime selection

`scripts/resolve_vllm_release.py` resolves the newest official non-draft, non-prerelease vLLM release by requiring GitHub Releases, the GitHub tag, and non-yanked PyPI metadata to agree. The exact tag and full commit are then frozen in `docker/dependency-manifest.json`.

Current lock (2026-07-21):

- vLLM `v0.25.1` / `752a3a504485790a2e8491cacbb35c137339ad34`
- PyTorch `2.11.0+cu130`
- CUDA 13.0
- FlashInfer `0.6.15.dev20260712` python/cubin/jit-cache trio

A newer stable release available before build becomes the primary candidate. Untagged `main` is not used for production by default.

## Build

On an admitted GB10 node:

```bash
python3 scripts/resolve_vllm_release.py --write docker/dependency-manifest.json
bash scripts/admit_node.sh

tmux new-session -d -s laguna-s21-build \
  "bash scripts/build_release.sh"
```

The durable build writes its log, return code, image inspection, and PASS/FAIL stamps under ignored private evidence storage. Build and runtime JIT are capped at `MAX_JOBS=4`, `NVCC_THREADS=2`, and `FLASHINFER_NVCC_THREADS=2`.

## Run AR baseline

```bash
docker run --rm --gpus all --ipc=host --network host \
  -e DFLASH_TOKENS=0 \
  -v /path/to/Laguna-S-2.1-NVFP4:/models/Laguna-S-2.1-NVFP4:ro \
  -v /path/to/flashinfer-cache:/var/cache/flashinfer \
  ghcr.io/r0b0tlab/vllm-laguna-s-2.1-nvfp4-sm121@sha256:4f8ba8a454fefc7b81f8e6ceafe46ed92ddf1f409a160556e5b1ea4daaaee80a
```

AR remains the independent correctness baseline. The production image defaults to the qualified DFlash K=7 profile; set `DFLASH_TOKENS=0` explicitly for AR.

## Run the production DFlash profile

```bash
docker run --rm --gpus all --ipc=host --network host \
  -e DFLASH_TOKENS=7 \
  -v /path/to/Laguna-S-2.1-NVFP4:/models/Laguna-S-2.1-NVFP4:ro \
  -v /path/to/Laguna-S-2.1-DFlash-NVFP4:/models/Laguna-S-2.1-DFlash-NVFP4:ro \
  -v /path/to/flashinfer-cache:/var/cache/flashinfer \
  ghcr.io/r0b0tlab/vllm-laguna-s-2.1-nvfp4-sm121@sha256:4f8ba8a454fefc7b81f8e6ceafe46ed92ddf1f409a160556e5b1ea4daaaee80a
```

Matched K={3,5,7,11,15} qualification selected K=7 for the production profile. DFlash requests must expose positive drafted/accepted counter deltas.

## Full 8,620-case benchmark battery

The production image completed every logical case in `r0b0bench-core-v1-rc2` with 210 calibration cases excluded from official totals. Scores are reported by pillar rather than collapsed into an opaque composite.

| Pillar | Result |
|---|---:|
| BFCL v4 `multi_turn_base` | 137/200 (68.50%) |
| Generated-answer quality | 6,838/7,715 (88.63%) |
| IFEval prompt strict / loose | 82.44% / 86.69% |
| IFEval instruction strict / loose | 87.53% / 90.89% |
| HumanEval pass@1 | 154/164 (93.90%) |

See the [human-readable scorecard](results/v0.25.1-gb10/full-battery/REPORT.md), [machine-readable scorecard](results/v0.25.1-gb10/full-battery/scorecard.json), and [full methodology](results/v0.25.1-gb10/full-battery/METHODOLOGY.md). The generated-answer scorecard includes GSM8K, ARC-Challenge, PIQA, WinoGrande XL, TruthfulQA MC1, and eight MMLU subjects.

## Safety boundaries

- Native compressed-tensors NVFP4 expert path only; no active Marlin/emulation/fallback.
- FP8 KV only. NVFP4 KV is rejected by the entrypoint.
- Do not force `flashinfer_b12x` on vLLM 0.25.1.
- No `min_p` under speculative decoding.
- No model weights are included in this repository or image.
- The software performs no telemetry, callbacks, or tracking.

## Verification

```bash
bash -n scripts/*.sh
python3 -m py_compile scripts/*.py tests/*.py
python3 -m unittest discover -s tests -v
python3 scripts/public_safety_scan.py .
```

Live qualification tools:

```bash
# Matched AR or DFlash performance, latency, metrics, and telemetry
python3 scripts/benchmark.py --model poolside/Laguna-S-2.1-NVFP4 \
  --container-name laguna-s21-ar --speculation disabled --output results/raw/ar.json

# Pinned 1,319-item GSM8K 0-shot run (requires pyarrow in the client environment)
python3 scripts/eval_gsm8k.py --output-dir results/raw/gsm8k

# Calibrated 64K, 128K, and 262K retrieval probes
python3 scripts/context_probe.py --output results/raw/context.json

# Parse a captured Nsight Systems report and reject fallback kernels
python3 scripts/profile_nvfp4.py profile.nsys-rep --output results/raw/nsight-kernels.json
```

`eval_gsm8k.py` pins dataset revision `740312add88f781978c0658806c59bc2815b9866` and verifies the test parquet SHA-256 before scoring. Raw samples and host-private evidence remain ignored; only sanitized, checksummed summaries are published.

A healthy API alone is insufficient. Release gates include exact identity, native-kernel markers, semantic/reasoning/tool canaries, long generation, AR/DFlash comparison, latency/throughput/power telemetry, 256K-context qualification, GSM8K 0-shot flexible extraction, immutable image verification, non-root SparkRun, anonymous pull, and clean clone.

Detailed references:

- [`docs/architecture.md`](docs/architecture.md) — checkpoint, native SM121 path, DFlash, and container boundaries
- [`docs/methodology.md`](docs/methodology.md) — identity, native, performance, context, quality, and publication gates
- [`docs/troubleshooting.md`](docs/troubleshooting.md) — fail-closed recovery for build, JIT, model load, DFlash, and packaging issues

## License and credit

Repository code is MIT licensed. vLLM and FlashInfer are Apache-2.0 projects. Model use is governed by Poolside's OpenMDW-1.1 model license and acceptable-use terms. This repository redistributes no model weights.

## Current checkpoint image

Immutable image: `ghcr.io/r0b0tlab/vllm-laguna-s-2.1-nvfp4-sm121@sha256:4f8ba8a454fefc7b81f8e6ceafe46ed92ddf1f409a160556e5b1ea4daaaee80a`

This checkpoint-only layer inherits the previously qualified runtime at `sha256:8b0e3d07dad370853bca77441ff7c8619c41a0cb80d59a150bb0faf17ee15ef3`; vLLM, CUDA, FlashInfer, and native binaries are unchanged.

GHCR package: [vllm-laguna-s-2.1-nvfp4-sm121](https://github.com/orgs/r0b0tlab/packages/container/package/vllm-laguna-s-2.1-nvfp4-sm121)

The current contract pins target `07614121b31898586430f189d27a25a0be310843`, draft `723794750422b3efbf3a7b3af76dffb4ba035943`, and vLLM commit `752a3a504485790a2e8491cacbb35c137339ad34`. The original [VERDICT](results/v0.25.1-gb10/VERDICT.md), [machine-readable results](results/v0.25.1-gb10/summary.json), and [HTML report](results/v0.25.1-gb10/benchmark.html) remain explicitly bound to target `216d1f13878dd4e715bc7412848d0f330e95bba6`.