rtx-p2p

KO | EN

gitlite — search

#macos #html #rust #sql #javascript #database #chatgpt #swift #open-source #ios

(★ 9)

PCIe P2P on consumer RTX 5090 via one GSP firmware registry key — no driver patches.

File Explorer

ZIP Download

- IS_GEFORCE.md
- REGISTRY_KEYS.md
- RMDISABLEFEATUREDISABLEMENT.md
- nvidia-p2p.conf
- disable-p2p.sh
- enable-p2p.sh
- p2p-bandwidth.py
- verify-p2p.sh
.gitignore
LICENSE
README.md

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

**Unlock PCIe peer-to-peer on GeForce RTX 5090 with a single undocumented GSP firmware registry key — stock NVIDIA driver, zero kernel patches.**

---

## What

Two `modprobe` options that enable PCIe P2P DMA between consumer GeForce cards, replacing the community approach of patching `open-gpu-kernel-modules` (aikitoria / geohot fork):

```
options nvidia NVreg_RegistryDwords="RMDisableFeatureDisablement=1;EnableResizableBar=1"
```

- No kernel module rebuild
- Works with the stock NVIDIA driver
- One `modprobe` drop-in + a driver reload

## Why it works

`RMDisableFeatureDisablement` is a registry key that exists **only inside the GSP firmware binary** — not in any public NVIDIA source tree, not in `nvrm_registry.h`, not in any driver fork. When set, it bypasses the GSP-side GeForce SKU feature gate and a cluster of firmware crash guards (`ebreak` traps) that would otherwise halt the GSP firmware when a consumer GPU touches professional-tier code paths. P2P setup is one of those paths.

`EnableResizableBar=1` resizes BAR1 from 256 MB to the full VRAM — BAR1 is the P2P data aperture.

Together, those two flags are sufficient. Full reverse‑engineering write-up:
[`docs/RMDISABLEFEATUREDISABLEMENT.md`](docs/RMDISABLEFEATUREDISABLEMENT.md).

## Quick start

```bash
sudo ./scripts/enable-p2p.sh
./scripts/verify-p2p.sh
```

`enable-p2p.sh` writes `modprobe.d/nvidia-p2p.conf` into `/etc/modprobe.d/`, kills anything holding `/dev/nvidia*`, and reloads the kernel module. `verify-p2p.sh` prints BAR1 size, `nvidia-smi topo -p2p`, and the `cudaDeviceCanAccessPeer` matrix.

To revert:

```bash
sudo ./scripts/disable-p2p.sh
```

## Prerequisites

- NVIDIA proprietary or open-kernel driver, 590.x or later (tested on 590.48.01)
- Consumer Blackwell GPU (tested on RTX 5090 / GB202)
- Re-Size BAR enabled in motherboard UEFI / BIOS
- On AMD systems, IOMMU in passthrough mode: add `amd_iommu=on iommu=pt` to the GRUB kernel command line
- Multiple GPUs on the same host for P2P to be meaningful

## Repository layout

```
modprobe.d/     NVIDIA modprobe drop-in (the two registry keys)
scripts/        enable / disable / verify helpers, cupy peer-bandwidth probe
docs/           GSP firmware reverse-engineering notes
```

## How this was found

Ghidra + Capstone RISC-V reverse engineering of the GSP firmware (`gsp_elf_9.bin`, extracted from driver 590.48.01). The GSP runs a RISC-V 64 core inside the GPU and parses `NVreg_RegistryDwords` via a giant registry parser function (`FUN_0101d664`, ~12 KB of code, ~850 keys). Cross-referencing the keys with feature-gate checks and `ebreak` trap sites identified `RMDisableFeatureDisablement` as the single flag that bypasses the SKU gate *and* the init-time crash guards.

See [`docs/RMDISABLEFEATUREDISABLEMENT.md`](docs/RMDISABLEFEATUREDISABLEMENT.md), [`docs/IS_GEFORCE.md`](docs/IS_GEFORCE.md), and [`docs/REGISTRY_KEYS.md`](docs/REGISTRY_KEYS.md).

## Related keys (not required, but relevant)

| Key | Effect |
|---|---|
| `RMForceP2PType=1` | Force PCIe P2P path (skip NVLink/C2C detection) |
| `RMPcieP2PType=1` | Force BAR1 mailbox P2P mode |
| `ForceP2P=0x11` | Force-set the P2P read/write enable bits |
| `NvLinkPeerThroughL2=1` | Route peer traffic through local L2 (undocumented, untested) |
| `RmEnableLocalizedMemory=1` | NUMA-aware peer memory (undocumented, untested) |
| `RmCeExposeAllLce=1` | Expose all copy engines to user space |

The first three are redundant once `RMDisableFeatureDisablement=1` + `EnableResizableBar=1` are set; they just skip detection paths that already succeed. The last three are speculative and documented in [`docs/REGISTRY_KEYS.md`](docs/REGISTRY_KEYS.md) for reference.

## License

MIT. See [`LICENSE`](LICENSE).

## Disclaimer

Educational reverse-engineering research. No NVIDIA source, firmware, or binary is redistributed here — only observations about its behaviour. Use on hardware you own. Flipping undocumented firmware flags can destabilise the GPU; there is no warranty of any kind.

rtx-p2p

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

rtx-p2p

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README