LlamaKit - GitLite

KO | EN

gitlite — search

#swift #web #animation #markdown #cross-platform #react-native #cli #kubernetes #agent #graphql

LlamaKit

(★ 17)

Swift wrapper for llama.cpp

파일 탐색기

ZIP 다운로드

- - Package.swift
- - ChatMessage.swift
  - LlamaBackend.swift
  - LlamaModel+Hub.swift
  - LlamaModel.swift
  - LlamaSession+Generation.swift
  - LlamaSession.swift
  - Sampler.swift
  - StreamingDetokenizer.swift
- - LlamaModelTests.swift
  - LlamaSessionTests.swift
  - SamplerTests.swift
  - StreamingDetokenizerTests.swift
.gitignore
LICENSE
Package.swift
README.md

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

명령어 용어집

이 문서에서 사용된 명령어를 모아봤습니다. 낯선 명령어가 있다면 펼쳐서 확인해보세요.

🔍

`swift`

설명 보기 ▼

Swift 프로젝트를 생성, 실행 및 빌드.
더 많은 정보: <https://manned.org/swiftc>.

swift repl

REPL(대화형 셸) 시작:

swift {{file.swift}}

프로그램 실행:

swift package init

패키지 관리자로 새 프로젝트 시작:

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Swift wrapper around [llama.cpp](https://github.com/ggml-org/llama.cpp), built on the [llama.swift](https://github.com/mattt/llama.swift) XCFramework, and with Hub integration via [swift-huggingface](https://github.com/huggingface/swift-huggingface).

Provides idiomatic Swift types for loading GGUF models, tokenizing, applying
chat templates, sampling, and streaming generation.

## Load a model from the Hugging Face Hub

```swift
import LlamaKit

let model = try await LlamaModel.from(
    repo: "Qwen/Qwen2.5-0.5B-Instruct-GGUF",
    filename: "*Q8_0.gguf"
)
```

The first matching `.gguf` file in the snapshot is loaded.

To load a local file:

```swift
let model = try LlamaModel(contentsOf: URL(fileURLWithPath: "model.gguf"))
```

## Apply a chat template

```swift
let prompt = try model.tokenizer.applyChatTemplate([
    .system("You are a helpful assistant."),
    .user("Explain entanglement in one sentence."),
])
```

Renders the conversation using the chat template embedded in the GGUF.
Pass `addGenerationPrompt: false` if you don't want the assistant-turn marker appended.

## Generate text

```swift
let session = try LlamaSession(model: model)

let response = try await session.complete(
    prompt: prompt,
    sampler: .temperature(0.7),
    maxTokens: 256
)
```

Or stream tokens as they're sampled:

```swift
for try await chunk in session.generate(prompt: prompt) {
    print(chunk, terminator: "")
}
```

Use `Sampler.greedy` for deterministic output.
`Sampler.minP(0.05)` and custom `Sampler(stages: [...])` chains are also
supported.

## Example: `kitchat`

`Examples/Chat/` contains a [small interactive chat CLI](https://github.com/pcuenca/LlamaKit/blob/main/Examples/Chat/Sources/kitchat/ChatCommand.swift) built on LlamaKit:

```bash
swift run kitchat --hf Qwen/Qwen2.5-0.5B-Instruct-GGUF
```

## Installation

```swift
.package(url: "https://github.com/pcuenca/LlamaKit", from: "0.1.0")
```

## Requirements

- Swift 6.1+
- macOS 13+ / iOS 16+ / visionOS 1+