pegaflow
High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and SGLang.
File Explorer
- SKILL.md
- SKILL.md
- SKILL.md
- async-loading.md
- SKILL.md
- ci.yml
- release.yml
- arch.svg
- logo.png
- deployment.md
- goals.md
- metrics.md
- p2p.md
- pd-rdma-push.md
- pd.md
- rust-quality.md
- server.md
- vllm-patch.md
- vllm-request-state-machine.md
- pegaflow.json
- dashboards.yaml
- prometheus.yaml
- docker-compose.yml
- otel-collector-config.yaml
- prometheus.yml
- pegaflow.json
- dashboards.yaml
- prometheus.yaml
- docker-compose.yml
- prometheus.yml
- basic_vllm.py
- bench_kv_cache.py
- bench_layer_wise.py
- check_engine_health.py
- ipc_receiver.py
- ipc_sender.py
- run_vllm_pd_with_pega.py
- run_vllm_with_pega.py
- share_gpt_bench.py
- block.rs
- grpc.rs
- hll.rs
- lib.rs
- logging.rs
- numa.rs
- Cargo.toml
- cpu_path.rs
- pinned_copy.rs
- transfer_h2d.rs
- uds_latency.rs
- pinned_alloc.rs
- pinned_alloc_parallel.rs
- mod.rs
- rdma.rs
- rdma_fetch.rs
- ssd.rs
- ssd_cache.rs
- transfer_lock_guard.rs
- uring.rs
- tests.rs
- metaserver_client.rs
- mod.rs
- p2p_service.rs
- README.md
- mod.rs
- prefetch.rs
- read_cache.rs
- tier_attribution.rs
- transfer_lock.rs
- write_path.rs
- kernel.rs
- memcpy.rs
- mod.rs
- allocator.rs
- block.rs
- cache.rs
- gpu_worker.rs
- instance.rs
- layout.rs
- lease.rs
- lib.rs
- metrics.rs
- offload.rs
- pinned_mem.rs
- pinned_pool.rs
- seal_offload.rs
- sync_state.rs
- trace.rs
- gpu_buffer.rs
- harness.rs
- helpers.rs
- mod.rs
- engine_basic.rs
- eviction.rs
- hybrid_groups.rs
- prefetch_lease.rs
- prefix_semantics.rs
- save_validation.rs
- ssd_cache.rs
- Cargo.toml
- unregister_node.rs
- http_server.rs
- lib.rs
- main.rs
- metric.rs
- proto.rs
- service.rs
- store.rs
- Cargo.toml
- README.md
- lib.rs
- Cargo.toml
- engine.proto
- lib.rs
- build.rs
- Cargo.toml
- p2p_bench.rs
- pegaflow-router.rs
- check_cuda_version.rs
- http_server.rs
- lib.rs
- main.rs
- metric.rs
- proto.rs
- registry.rs
- service.rs
- session.rs
- trace.rs
- utils.rs
- mod.rs
- http_cleanup_hang_repro.rs
- mock_vllm_rpc_e2e.rs
- p2p_rdma.rs
- build.rs
- Cargo.toml
- README.md
- cpu_bench.rs
- cpu_bench_v2.rs
- cumem.rs
- device.rs
- driver.rs
- error.rs
- event.rs
- mem.rs
- mod.rs
- rt.rs
- mod.rs
- runtime.rs
- session.rs
- state.rs
- defer.rs
- hex.rs
- memory.rs
- mod.rs
- obj_pool.rs
- mod.rs
- verbs_address.rs
- verbs_devinfo.rs
- verbs_domain.rs
- verbs_qp.rs
- verbs_rdma_op.rs
- api.rs
- cpu_affinity.rs
- domain_group.rs
- error.rs
- fabric_engine.rs
- host_buffer.rs
- imm_count.rs
- interface.rs
- mod.rs
- mr.rs
- provider.rs
- provider_dispatch.rs
- rdma_op.rs
- topo.rs
- transfer_engine.rs
- transfer_engine_builder.rs
- worker.rs
- cuda_sys.rs
- cudart_sys.rs
- engine.rs
- error.rs
- lib.rs
- rdma_topo.rs
- Cargo.toml
- README.md
- __init__.py
- common.py
- connector_metrics.py
- scheduler.py
- state_manager.py
- tp_shards.py
- worker.py
- __init__.py
- base_scheduler.py
- base_worker.py
- connector.py
- metadata.py
- pega_rdma_v1.py
- pull_scheduler.py
- pull_worker.py
- push_scheduler.py
- push_worker.py
- README.md
- scheduler.py
- stats.py
- tp_mapping.py
- utils.py
- worker.py
- __init__.py
- async_runner.py
- base_connector.py
- chunk_tracker.py
- config.py
- decode_worker.py
- kv_params.py
- layout.py
- layout_mapping.py
- metadata.py
- metrics.py
- prefill.py
- prefill_async.py
- prefill_tasks.py
- prefill_worker.py
- proxy.py
- rdma.py
- scheduler.py
- worker.py
- __init__.py
- _bin_utils.py
- _metaserver.py
- _server.py
- ipc_wrapper.py
- logging_utils.py
- pegaflow.pyi
- vllm_plugin.py
- metaserver.rs
- server.rs
- lib.rs
- pd_rdma.rs
- rdma_v1.rs
- glm51_fp8_config.json
- __init__.py
- pd_connector_test_utils.py
- test_pd_connector_flow.py
- test_pd_connector_layout.py
- __init__.py
- conftest.py
- README.md
- session_crash_helper.py
- test_cache_group_layout.py
- test_combine_hashes.py
- test_connector_fault_tolerance.py
- test_connector_save_lifecycle.py
- test_derive_namespace.py
- test_engine_client.py
- test_hybrid_reconcile.py
- test_ipc_wrapper.py
- test_mla_replica_registration.py
- test_pd_tail_save.py
- test_session_watcher.py
- test_tp_shards.py
- test_unit_stubs.py
- test_vllm_e2e_correctness.py
- test_vllm_p2p_pp4_e2e.py
- test_vllm_save_only_e2e.py
- test_vllm_tp_shards_e2e.py
- test_vllm_warm_hit_stress.py
- test_worker_kv_layout.py
- unit_stubs.py
- vllm_helpers.py
- Cargo.toml
- pyproject.toml
- README.md
- build-wheel.sh
- check.sh
- pd_rdma_binding_probe.py
- pd_rdma_e2e.py
- run_nixl_local.sh
- run_pd_local.sh
- run_vllm_ttft_bench.sh
- main.rs
- .gitignore
- AGENTS.md
- Cargo.lock
- Cargo.toml
- CLAUDE.md
- Dockerfile
- LICENSE
- pegaflow.code-workspace
- prek.toml
- README.md
- rust-toolchain.toml
- typos.toml
# Use via CDN
jsDelivrjsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.
Link
Example
Command Glossary
Commands referenced in this DOCs, explained below.
cargo run
View Details ▼
cargo run
Run the current Cargo package.
Note: The working directory of the executed binary will be set to the current working directory.
cargo {{[r|run]}}
Run the default binary target:
cargo {{[r|run]}} --bin {{name}}
Run the specified binary:
cargo {{[r|run]}} --example {{name}}
Run the specified example:
uv pip
View Details ▼
uv pip
Provides pip-like commands for installing, uninstalling, and managing packages.
uv pip install {{package}}
Install a package:
uv pip install {{[-r|--requirements]}} {{requirements.txt}}
Install packages from a requirements file:
uv pip install {{package==1.2.3}}
Install a package with a specific version:
// repository documentation
Was this content helpful?
(0 ratings)