Oz-FP4

KO | EN

gitlite — search

#mcp #tensorflow #mysql #codex #android #devops #nextjs #web #ui #ai

(★ 9)

Base-13 FP4 limbs for Ozaki-scheme DGEMM emulation on FP4 tensor cores

File Explorer

ZIP Download

- - fig_e2m1_coverage.png
  - fig_hierarchy_i.png
  - fig_hierarchy_ii.png
  - fig_speed_model.png
- build_fig_hierarchy_i_diamond2A.py
- build_fig_hierarchy_ii_v2_g48.py
- fig_breakdown_evolution_i_p15.py
- fig_breakdown_evolution_ii.py
- fig_e2m1_coverage_g48.py
- fig_g16_accuracy_g48.py
- fig_g16_speed_with_model_v2_g48.py
- fig_hierarchy_i_diamond2A.tex
- fig_hierarchy_ii_v2_sandwich_draft.tex
- karatsuba_moduli.py
- moduli_pm13n.py
- - bench_g08d.py
  - fused_decompose.py
  - fused_int8_combine.py
- - dgemm_emu.py
  - reduce_int128.py
- - dgemm_ii.py
- - dgemm_int8.py
- - crt_light.py
- - frontier.py
  - frontier_fused_tn.py
- - bench_g28.py
- - bench_g37.py
  - frontier_i_g37.py
- - g43_bridge.cu
- - bench_g48.py
  - bench_g48_acc.py
  - bench_g48_fastmode.py
- make_table1.py
.gitignore
ENVIRONMENT.md
LICENSE
README.md

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

FP4 (E2M1) values become integers when doubled, and the base-13 shift `2*FP4 + 13n`
tiles the integers.

![E2M1 integer lattice and its two-limb coverage](https://raw.githubusercontent.com/FP4-is-All-you-Need/Oz-FP4/main/docs/img/fig_e2m1_coverage.png)

Therefore the Ozaki scheme (both the position-weighted variant OzI and the
residue/CRT variant OzII) can be run on FP4 tensor cores with correct rounding.
This repository contains the kernels, the benchmark harness, and the figure/table
generators for the paper.

![OzI structure: recursive cascade of limb planes](https://raw.githubusercontent.com/FP4-is-All-you-Need/Oz-FP4/main/docs/img/fig_hierarchy_i.png)

## Result

![OzII structure: one residue plane per modulus, recombined by CRT](https://raw.githubusercontent.com/FP4-is-All-you-Need/Oz-FP4/main/docs/img/fig_hierarchy_ii.png)

![Time breakdown against the bandwidth and GEMM-count model](https://raw.githubusercontent.com/FP4-is-All-you-Need/Oz-FP4/main/docs/img/fig_speed_model.png)

Matches unmodified GEMMul8, the state-of-the-art FP8 Ozaki II, and wins at
16384^3 -- on hardware where FP4 is only twice FP8. B300 widens that ratio and
removes INT8, leaving FP4 as the only fast exact path.

## Status

Published as [arXiv:2608.06812](https://arxiv.org/abs/2608.06812). This repository is
the artifact accompanying that paper; the code is released as-is for reproduction.

## Requirements

- NVIDIA GPU with compute capability 12.0 (sm_120)
- CUDA 13.2 (nvcc) for building the GEMMul8 bridge
- PyTorch 2.11 + cu128
- Triton 3.6
- Python 3.12, plus `matplotlib`, `pillow` and `mpmath`

The exact machine, driver and library versions used for every number in the paper
are recorded in [`ENVIRONMENT.md`](ENVIRONMENT.md).

Only `matplotlib`, `pillow` and a Python 3 interpreter are needed to regenerate the
figures and Table 1 from the checked-in JSON results; the GPU and CUDA stack are
needed only to re-run the benchmarks.

<details>
<summary>Third-party baseline and the GEMMul8 bridge</summary>

The baseline is [GEMMul8](https://github.com/RIKEN-RCCS/GEMMul8) (RIKEN-RCCS), used
**unmodified**. All measurements in the paper use **v3.1.0 (commit `450c4a6`)**.
Clone and build it next to this repository, then build the C-ABI bridge:

```sh
git clone https://github.com/RIKEN-RCCS/GEMMul8
cd GEMMul8 && git checkout 450c4a6
make GPU_ARCH=120          # explicit: the autodetect path needs nvidia-smi
cd ..

nvcc <this-repo>/kernel/g43_fair/g43_bridge.cu \
 -std=c++20 -O3 -Xcompiler -fPIC -shared \
 -o <this-repo>/kernel/g43_fair/libg43bridge.so \
 -gencode arch=compute_120,code=sm_120 \
 -IGEMMul8/include -lcublas -lcublasLt -lcudart -lcuda -lnvidia-ml -ldl \
 GEMMul8/lib/libgemmul8.a
```

The bridge splits GEMMul8 into setup / run-once / teardown so that the Python harness
owns the iteration loop, which lets every method be measured with the identical
estimator (median of single CUDA-event windows) and interleaved to cancel
measurement-order bias. `bench_g48.py` loads `libg43bridge.so` at import time, so the
bridge must be built before running the benchmark.

</details>

## Layout

```
kernel/
  g10_tune/           fused limb-ization (decompose) and the group GEMM kernels
  g15_dgemm_emu/      OzI FP64 emulation shell + the int128 reduction
  g17_fp4mod2_dgemm/  OzII (residue / CRT) canonical implementation
  g20_int8_engine/    OzII residues on the INT8 engine (engine-independence check)
  g21_crt_light/      Garner and direct O(L) CRT restore
  g25_frontier/       the t knob (accuracy/speed dial) for OzII
  g28_tn_layout/      TN-layout group GEMM
  g37_i_fastest/      OzI canonical preprocessing (tiled encode, TN direct write)
  g38_perf_model/     results/  theoretical bandwidth + GEMM-count model
  g43_fair/           g43_bridge.cu -- C-ABI bridge to unmodified GEMMul8
  g48_all/            bench_g48*.py -- the benchmark harness, and results/ JSON
figs/                 figure generators (matplotlib) and TikZ sources
tools/make_table1.py  regenerates Table 1 from the benchmark JSON
```

`kernel/` keeps its directory-per-experiment structure because the modules add their
siblings to `sys.path` by relative path; do not flatten it.

<details>
<summary>Reproducing the paper</summary>

All measured numbers come from **one run of `bench_g48.py`, stored in one JSON**, so
the table and the measured figures share a single estimator and a single session.

| Paper item | Generator | Input |
|---|---|---|
| Fig. 1 (E2M1 coverage) | `figs/fig_e2m1_coverage_g48.py` | none (analytic) |
| Fig. 2 (OzI structure) | `figs/build_fig_hierarchy_i_diamond2A.py` + `figs/fig_hierarchy_i_diamond2A.tex` | none (TikZ) |
| Fig. 3 (OzII structure) | `figs/build_fig_hierarchy_ii_v2_g48.py` + `figs/fig_hierarchy_ii_v2_sandwich_draft.tex` | none (TikZ) |
| Fig. 4 (OzI pipeline evolution) | `figs/fig_breakdown_evolution_i_p15.py` | `kernel/g48_all/results/rtxpro6000ws_20260802_060356_g48_all.json` |
| Fig. 5 (OzII pipeline evolution) | `figs/fig_breakdown_evolution_ii.py` | `kernel/g48_all/results/rtxpro6000ws_20260802_060356_g48_all.json` |
| Table 1 (equivalent-DGEMM TFLOPS) | `tools/make_table1.py` | `kernel/g48_all/results/rtxpro6000ws_20260802_060356_g48_all.json` |
| Fig. 6 (accuracy vs dynamic range) | `figs/fig_g16_accuracy_g48.py` | `kernel/g48_all/results/rtxpro6000ws_20260802_140026_g48_acc.json` |
| Fig. 7 (time breakdown + model) | `figs/fig_g16_speed_with_model_v2_g48.py` | `..._g48_all.json` + `kernel/g38_perf_model/results/rtxpro6000ws_20260713_g38_perf_model.json` |

Supporting scripts: `figs/moduli_pm13n.py` (the +/-13n moduli set and its coverage)
and `figs/karatsuba_moduli.py` (the Karatsuba-style modulus reduction).

Regenerating a figure or the table needs no GPU:

```sh
py tools/make_table1.py
py figs/fig_breakdown_evolution_ii.py
py figs/fig_breakdown_evolution_i_p15.py
py figs/fig_g16_accuracy_g48.py
py figs/fig_g16_speed_with_model_v2_g48.py
py figs/fig_e2m1_coverage_g48.py
```

The two TikZ figures need a TeX Live with `pdflatex`, `pdfcrop` and `pdftoppm`
on `PATH`.

Re-running the measurement itself (GPU + built bridge required):

```sh
cd kernel/g48_all
python bench_g48.py --estimate   # wall-clock estimate only
python bench_g48.py              # speed, ablation and the gate
python bench_g48_acc.py          # accuracy sweep over the dynamic-range parameter
python bench_g48_fastmode.py     # GEMMul8 fast-mode variant
```

Each run writes a fresh JSON into `kernel/g48_all/results/`; point the figure scripts
at it with `--json <path>`.

</details>

<details>
<summary>Notes</summary>

- `bench_g48.py` also registers an `int8_fp4emu` method, and the checked-in JSON
  therefore carries an `int8_fp4emu` row. That configuration (emulating an INT8 GEMM
  with base-13 FP4 limbs) is **outside the scope of this paper**; its kernel is not
  included here, and the harness skips the configuration when the module is absent.
  Ignore that row when reading the JSON.
- The JSON files also contain a `t` sweep (`ozII_t*`) and a GEMMul8-FP8 modulus-count
  sweep (`fp8_nm*`) that are not used by the figures shipped here.
- The benchmark gate asserts that the proposed configurations are **bit-identical** to
  the canonical implementation, and bounds the relative error of the other methods; a
  run whose gate fails is reported as such in `rows[].gate`.

</details>

## Citation

```bibtex
@misc{hayashi2026fp4ozaki,
      title={{DGEMM with Ozaki Scheme I/II on FP4 Tensor Cores: A Base-13 E2M1 Limb Representation}},
      author={{Shun-ichiro} Hayashi and Daichi Mukunoki and Tetsuya Hoshino and Takahiro Katagiri},
      year={2026},
      eprint={2608.06812},
      archivePrefix={arXiv},
      primaryClass={cs.DC},
      url={https://arxiv.org/abs/2608.06812},
}
```

## License

MIT. See [`LICENSE`](LICENSE).

Oz-FP4

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README