GLM-5.2-1M-4x-DGX-Spark

KO | EN

gitlite — search

#hacktoberfest #php #deep-learning #http #postgresql #awesome #python #vue #electron #gpt

GLM-5.2-1M-4x-DGX-Spark

(★ 9)

Unpruned GLM-5.2 (744B) at 1M context on 4x DGX Spark (GB10/sm_121a) — NVFP4 compact-KV + B12X sparse-MLA + MTP-5. Tested, stable, honest measured numbers.

파일 탐색기

ZIP 다운로드

- - validate.yml
- bench_engine.py
- v1_64k.py
- fix-indexer-mtp-overhang.py
- o14-profiles.json
- render_profile_chart.py
- validate_profiles.py
- serve-dcp2-nvfp4.sh
- test_profiles.py
AGENTS.md
FINDINGS.md
LICENSE
NOTICE
profile-index-chart.svg
README.md

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

This repository is the public profile index for the O14 GLM-5.2 runtime. Machine-readable selection is in [`profiles/o14-profiles.json`](profiles/o14-profiles.json); automation rules are in [`AGENTS.md`](AGENTS.md).

## Current profiles

| Profile | Status | Topology | Total KV | Deploy? | Runtime artifact |
|---|---|---:|---:|---:|---|
| **O14 Fast** | **READY** | TP4 / DCP1 / PP1 | **250K** | **Yes** | Build from the [pinned canonical O14 Harness source](https://github.com/0xdfi/GLM-5.2-Harness-O14-4x-DGX-Spark/tree/9ba54db8f7c852eabb90372f7dcf47f87eafbe20) |
| **O14 Balanced** | **TESTING / DO NOT DEPLOY** | TP4 / DCP2 / PP1 | **500K target** | **No** | None; separate immutable runtime required after acceptance |

### Matched-speed results

Balanced speed remains `TBD` until the exact retained DCP1 fixture set is run against the accepted DCP2 runtime. No DCP2 speed value is inferred.

| Profile | Prefill | Peak decode | Deep-context decode | Fixture status |
|---|---:|---:|---:|---|
| **O14 Fast** | **819 tok/s** | **42.3 tok/s** | **29–33 tok/s** at ~53K prompt tokens | Retained legacy exp1 DCP1 fixtures |
| **O14 Balanced** | **TBD** | **TBD** | **TBD** | Run the exact DCP1 fixtures after DCP2 acceptance |

## O14 Fast

**O14 Fast is READY and deployable at 250K total KV.**

| Machine field | Exact value |
|---|---:|
| Allocator total KV | 250,023 tokens |
| `KV_CACHE_MEMORY_BYTES` | 7,995,534,848 bytes per rank |
| `MAX_MODEL_LEN` | 249,000 |
| `MAX_NUM_SEQS` | 4 |
| `MAX_NUM_BATCHED_TOKENS` | 2,048 |
| Topology | TP4 / DCP1 / PP1 |

The selector pins canonical Harness commit [`9ba54db8f7c852eabb90372f7dcf47f87eafbe20`](https://github.com/0xdfi/GLM-5.2-Harness-O14-4x-DGX-Spark/commit/9ba54db8f7c852eabb90372f7dcf47f87eafbe20), which contains the named Fast selector and exact recipe values.

There is no published O14 Fast OCI image in phase 1. Do not invent or pull a Fast tag. Build from the pinned canonical source recipe and keep model weights outside the runtime image.

## O14 Balanced

**O14 Balanced is TESTING / DO NOT DEPLOY.** Its target is 500K total KV with an expected allocator total of 500,237 tokens, 8,000,000,000 KV bytes per rank, `MAX_MODEL_LEN=490000`, and TP4/DCP2/PP1.

Balanced speed is `TBD`. Its source recipe, image reference, and digest are intentionally `null`. Acceptance requires a separate immutable DCP2 runtime image and reproducible recipe. Agents must never synthesize a Balanced tag, reuse Fast's image, or overwrite Fast.

## Safe selection

1. Read the requested profile from [`profiles/o14-profiles.json`](profiles/o14-profiles.json).
2. Continue only when `status == "READY"` and `deployable == true`.
3. If an image reference or digest is `null`, do not pull or invent one.
4. Pin the exact source commit, verify checksums, and run the canonical validators before building.
5. Supply model paths, endpoints, ports, and network interfaces explicitly for the target environment.

See [`AGENTS.md`](AGENTS.md) for exact pull/build behavior.

## Runtime components

The current source stack combines:

- NVFP4 DS-MLA KV cache (`nvfp4_ds_mla`, 368-byte record)
- B12X sparse-MLA attention and sparse indexer for GB10 / `sm_121a`
- MTP-5 speculative decode
- TP4 plus decode-context parallelism
- full and piecewise CUDA graph execution

Model card: <https://huggingface.co/0xdfi/GLM-5.2-1M-context-NVFP4-4x-DGX-Spark>

## Legacy exp1

> **Historical evidence only — not selectable profiles.** The July 2026 exp1 work used DCP1/DCP2/DCP4 labels before the O14 profile names were adopted. Those settings and limits are not current selector values. DCP4 remains **Legacy exp1 DCP4**; it is not “O14 High.”

### Legacy exp1 topology observations

| Legacy topology | Historical max context with full graphs | Historical peak decode | Historical fastest prefill |
|---|---:|---:|---:|
| Legacy exp1 DCP1 | 375K | ~42 tok/s | ~820 tok/s |
| Legacy exp1 DCP2 | 625K | ~37 tok/s | 746 tok/s |
| Legacy exp1 DCP4 | 1M | ~38 tok/s short / ~30 tok/s deep | 614 tok/s |

These are preserved as historical observations from one rig, not deployment promises or selectable capacities. The retained DCP1 fixture values used in the current matched-speed table are 819 tok/s prefill, 42.3 tok/s peak decode, and 29–33 tok/s at ~53K actual prompt tokens.

### Legacy exp1 engineering findings

- Sparse-indexer scratch scaled with `context × batch` and was not sharded by DCP.
- One versus two active RoCE links changed measured throughput by only about 1.5–3%; this repository imposes no fixed link-count or link-rate requirement.
- CUDA graphs improved decode but consumed memory otherwise available to context.
- Decode varied with MTP acceptance at short context and converged toward the sparse-indexer ceiling at depth.
- Unified-memory load pressure required operational care around page cache and swap.

The archived matrix is in [`FINDINGS.md`](FINDINGS.md). The parameterized historical launcher and benchmark scripts remain for evidence reproduction only; they are not current profile recipes.

## Validation

```bash
python3 scripts/render_profile_chart.py --check
python3 scripts/validate_profiles.py
python3 -m unittest discover -s tests -v
```

The validator checks exact selector values/statuses, null artifact claims, chart determinism, private defaults, and stale current-profile capacities.

## License and attribution

Apache-2.0. Community attribution is preserved in [`NOTICE`](NOTICE).

GLM-5.2-1M-4x-DGX-Spark

오너의 다른 레포

파일 탐색기

# CDN으로 사용하기

Was this content helpful?

Recent Feedback

GLM-5.2-1M-4x-DGX-Spark

오너의 다른 레포

파일 탐색기

# CDN으로 사용하기

Was this content helpful?

Recent Feedback

Download README