bunny-llama - GitLite

KO | EN

gitlite — search

#hacktoberfest #ai #llm #golang #machine-learning #awesome #cli #kubernetes #security #pytorch #mcp #ios #openai #open-source

bunny-llama

(★ 51)

iterate quickly with llama.cpp hot reloading. use the llama.cpp bindings with bun.sh

파일 탐색기

최종 버전 다운로드 (.zip)

- - global_llama.ts
- bunny.ts
- bunny_hmr.ts
- README.md
- - cuda_hmr.gif
  - js_hmr.gif
- clone_llama.ts
- make_cuda_llama.ts
- - .gitignore
  - clone_repo.ts
.env
.gitignore
dev.ts
index.ts
LICENSE
package.json
README.md
ride.ts
tsconfig.json

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/spirobel/bunny-llama

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd bunny-llama

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Node.js

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Node.js npm 명령어를 쓰려면 Node.js가 설치되어 있어야 합니다. LTS(장기지원) 버전을 추천합니다.

npm install

package.json에 명시된 라이브러리들을 내려받아 설치합니다.

npm start

개발/실행 서버를 켭니다.

✅ 명령어 실행 후 터미널에 나타나는 주소(보통 http://localhost:3000 형태)를 브라우저에서 열어보세요.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# bunny-llama
![js_hmr](https://raw.githubusercontent.com/spirobel/bunny-llama/master/buntils/img/js_hmr.gif)
![cuda_hmr](https://raw.githubusercontent.com/spirobel/bunny-llama/master/buntils/img/cuda_hmr.gif)
[demo video](https://www.youtube.com/watch?v=H9f7Np_zUzc)
## What is this?

A bunny that sits on top of a llama (and controls it).

### On a more serious note:

The bunny-llama project is a tool that uses [bun](https://bun.sh/) to interact with [llama.cpp](https://github.com/ggerganov/llama.cpp). It provides a way to **quickly develop and test code** using llama.cpp bindings. The project supports hot module reloading, which allows for changes in the code to be reflected in the running application **without needing to manually restart it.**

This also means **the model does not have to be reloaded** every time you make a change and **recompile your custom prompting functions**.
## Hot module reloading

under the hood in the llama.cpp dir:
``` bash
 make api-llama.so && curl localhost:1337
```

```bun make``` and ```bun make-cuda``` will take care of the curl already.

## To run:

``` bash
bun clone
bun make
bun ride
```

### To clean:

``` bash
bun clean
```

### To install bun:

``` bash
curl -fsSL https://bun.sh/install | bash
```

## Nvidia llama

For people with nvidia gpus:

install [conda.](https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html)
``` bash
conda create -n bunny
conda activate bunny
conda install cuda -c nvidia
```

then make the llama with cuda, like so:

``` bash
bun clone
bun make-cuda
bun ride.ts
```

now you have a special cuda enabled llama.

if you closed your shell and you want to build the cuda llama again,
you need to activate the conda environment first:
``` bash
conda activate bunny
bun make-cuda
```