rwkv.cpp

(β˜… 1,580)

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

File Explorer

Download Latest Version (.zip)
  • .gitignore
  • .gitmodules
  • CMakeLists.txt
  • ggml
  • LICENSE
  • README.md
  • rwkv.cpp
  • rwkv.h
  • rwkv_error_handling.inc
  • rwkv_eval.inc
  • rwkv_file_format.inc
  • rwkv_graph.inc
  • rwkv_model_loading.inc
  • rwkv_operators.inc
  • rwkv_operators_wkv_v7.inc
  • rwkv_quantize.inc
  • rwkv_utilities.inc

# Installation Guide

1. Get the code
git clone https://github.com/RWKV/rwkv.cpp

Downloads the entire project code from GitHub to your computer.

cd rwkv.cpp

Moves into the project folder you just downloaded.

2. CMake

Medium Recommended
Prerequisites
  • Git Needed to download the project code from GitHub.
  • CMake The tool used to generate build configuration.
  • C/C++ 컴파일러 Windows needs Visual Studio (Community edition, free), macOS needs Xcode Command Line Tools, Linux needs the gcc/g++ package.
cmake .

Type this command into your terminal and run it.

cmake --build . --config Release

Type this command into your terminal and run it.

cmake . -DRWKV_CUBLAS=ON

Type this command into your terminal and run it.

βœ… Check that an executable was created inside the build folder, then run it directly (e.g. ./build/app_name).

Pulled directly from this repo's README.

3. Python

Easy
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.
pip install -r python/requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

python <μ‹€ν–‰ν•  파일λͺ…>.py # READMEμ—μ„œ μ •ν™•ν•œ μ‹€ν–‰ 파일λͺ…을 ν™•μΈν•˜μ„Έμš”

Runs the Python script (or module).

βœ… If it runs without errors and prints output in the terminal, it worked.

Today's picks

reveal.js β˜… 72,157

The HTML Presentation Framework

JavaScript
transformers β˜… 164,305

πŸ€— Transformers: the model-definition framework for state-of-the-art machine learning model

Python
awesome-llm-apps β˜… 133,464

100+ AI Agents, Agent Skills and RAG Apps - Free and Open Source.

Python
langflow β˜… 153,529

Langflow is a powerful tool for building and deploying AI-powered agents and workflows.

Python
weekly β˜… 100,534

η§‘ζŠ€ηˆ±ε₯½θ€…ε‘¨εˆŠοΌŒζ―ε‘¨δΊ”ε‘εΈƒ
graphify β˜… 109,008

Turn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowled

Python
// repository documentation