llm-book

KO | EN

gitlite — search

#python #hacktoberfest #ai #typescript #golang #machine-learning #deep-learning #nodejs #java #awesome #cli #kubernetes #vue #ios

(★ 474)

「大規模言語モデル入門」（2023）と「大規模言語モデル入門Ⅱ〜生成型LLMの実装と評価」（2024）のGitHubリポジトリ

llm-book 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# 大規模言語モデル入門

「[大規模言語モデル入門](https://gihyo.jp/book/2023/978-4-297-13633-8)」（2023）と「[大規模言語モデル入門Ⅱ〜生成型LLMの実装と評価](https://gihyo.jp/book/2024/978-4-297-14393-0)」（2024）のリポジトリです。

## コード

コードはすべて Google Colaboratory で動作確認を行なっています。
コードの中で利用したデータセットや作成したモデルは[Hugging Face Hub](https://huggingface.co/llm-book)にて公開しています。

**⚠️ 2023/7/28 現在、MARC-ja のデータセットの配布元のリンクが切れており、書籍上の 5.2, 5.3, 5.5.4 に掲載されているコードにおいて、データセット読み込みの箇所でエラーが出る状態です。
現在問い合わせのメールを送り、復旧待機中です。**

**これに応じて、日本語感情分析データセットである [WRIME](https://github.com/ids-cv/wrime) を使用したノートブックを追加致しましたので、コードを動作させたい方はご活用ください。**

| 章 | 節／項 | Colab | Link |
| --- | --- | --- | --- |
| 第 1 章 はじめに | 1.1 transformers を使って自然言語処理を解いてみよう 1.2 transformers の基本的な使い方 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter01/1-introduction.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter01/1-introduction.ipynb) |
| 第 2 章 Transformer | 2.2 エンコーダ | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter02/2-2-transformer-position-encoding.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter02/2-2-transformer-position-encoding.ipynb) |
| 第 3 章 大規模言語モデルの基礎 | 3.2 GPT（デコーダ） 3.3 BERT・RoBERTa（エンコーダ） 3.4 T5（エンコーダ・デコーダ）| [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter03/3-zero-shot-inference.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter03/3-zero-shot-inference.ipynb) |
| | 3.6 トークナイゼーション | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter03/3-6-tokenization.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter03/3-6-tokenization.ipynb) |
| 第 5 章 大規模言語モデルのファインチューニング | 5.2 感情分析モデルの実装 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-2-sentiment-analysis-finetuning.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-2-sentiment-analysis-finetuning-wrime.ipynb) | [Link (MARC-ja)](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-2-sentiment-analysis-finetuning.ipynb) [Link (WRIME)](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-2-sentiment-analysis-finetuning-wrime.ipynb) |
| | 5.3 感情分析モデルのエラー分析 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-3-sentiment-analysis-analysis.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-3-sentiment-analysis-analysis-wrime.ipynb) | [Link (MARC-ja)](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-3-sentiment-analysis-analysis.ipynb) [Link (WRIME)](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-3-sentiment-analysis-analysis-wrime.ipynb) |
| | 5.4.1 自然言語推論の実装（訓練）| [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-4-nli-finetuning.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-4-nli-finetuning.ipynb) |
| | 5.4.1 自然言語推論の実装（分析）| [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-4-nli-analysis.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-4-nli-analysis.ipynb) |
| | 5.4.2 意味的類似度計算の実装（訓練）| [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-4-sts-finetuning.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-4-sts-finetuning.ipynb) |
| | 5.4.2 意味的類似度計算の実装（分析）| [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-4-sts-analysis.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-4-sts-analysis.ipynb) |
| | 5.4.3 多肢選択式質問応答モデルの実装（訓練）| [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-4-multiple-choice-qa-finetuning.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-4-multiple-choice-qa-finetuning.ipynb) |
| | 5.4.3 多肢選択式質問応答モデルの実装（分析）| [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-4-multiple-choice-qa-analysis.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-4-multiple-choice-qa-analysis.ipynb) |
| | 5.5.4 LoRA チューニング（感情分析）| [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-5-sentiment-analysis-finetuning-LoRA.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter05/5-5-sentiment-analysis-finetuning-LoRA-wrime.ipynb) | [Link (MARC-ja)](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-5-sentiment-analysis-finetuning-LoRA.ipynb) [Link (WRIME)](https://github.com/ghmagazine/llm-book/blob/main/chapter05/5-5-sentiment-analysis-finetuning-LoRA-wrime.ipynb) |
| 第 6 章 固有表現認識 | 6.2 データセット・前処理・評価指標 6.3 固有表現認識モデルの実装 6.4 アノテーションツールを用いたデータセット構築 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter06/6-named-entity-recognition.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter06/6-named-entity-recognition.ipynb) |
| 第 7 章 要約生成 | 7.2 データセット 7.3 評価指標 7.4 見出し生成モデルの実装 7.5 多様な生成方法による見出し生成 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter07/7-summarization-generation.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter07/7-summarization-generation.ipynb) |
| 第 8 章 文埋め込み | 8.3 文埋め込みモデルの実装 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter08/8-3-simcse-training.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter08/8-3-simcse-training.ipynb) |
| | 8.4 最近傍探索ライブラリ `Faiss` を使った検索 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter08/8-4-simcse-faiss.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter08/8-4-simcse-faiss.ipynb) |
| 第 9 章 質問応答 | 9.3 ChatGPT にクイズを答えさせる | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter09/9-3-quiz-chatgpt.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter09/9-3-quiz-chatgpt.ipynb) |
| | 9.4.3 BPR の実装 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter09/9-4-3-bpr-training.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter09/9-4-3-bpr-training.ipynb) |
| | 9.4.4 BPR によるパッセージの埋め込みの計算 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter09/9-4-4-bpr-embedding.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter09/9-4-4-bpr-embedding.ipynb) |
| | 9.5 文書検索モデルと ChatGPT を組み合わせる | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter09/9-5-quiz-chatgpt-plus-bpr.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter09/9-5-quiz-chatgpt-plus-bpr.ipynb) |
| 第 10 章 性能評価 | 10.2.2 llm-jp-evalで扱うタスク | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter10/10-2a-llm-jp-eval-dataset.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter10/10-2a-llm-jp-eval-dataset.ipynb) |
| | 10.2.3 llm-jp-evalで使用される評価指標 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter10/10-2b-llm-jp-eval-metrics.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter10/10-2b-llm-jp-eval-metrics.ipynb) |
| | 10.2.4 多肢選択式質問応答タスクによる自動評価 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter10/10-2c-llm-jp-eval-jcommonsenseqa.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter10/10-2c-llm-jp-eval-jcommonsenseqa.ipynb) |
| | 10.2.4 多肢選択式質問応答タスクによる自動評価（ツールを使用した評価） | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter10/10-2d-llm-jp-eval-jcommonsenseqa-flexeval.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter10/10-2d-llm-jp-eval-jcommonsenseqa-flexeval.ipynb) |
| | 10.3.2 Japanese Vicuna QA Benchmarkによる自動評価 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter10/10-3a-ja-vicuna-qa.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter10/10-3a-ja-vicuna-qa.ipynb) |
| | 10.3.2 Japanese Vicuna QA Benchmarkによる自動評価（ツールを使用した評価） | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter10/10-3b-ja-vicuna-qa-flexeval.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter10/10-3b-ja-vicuna-qa-flexeval.ipynb) |
| 第 11 章 指示チューニング | 11.2 指示チューニングの実装 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter11/11-2-instruction_tuning-train.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter11/11-2-instruction_tuning-train.ipynb) |
| | 11.3 指示チューニングしたモデルの評価 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter11/11-3-instruction-tuning-eval.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter11/11-3-instruction-tuning-eval.ipynb) |
| 第 12 章 選好チューニング | 12.2 選好チューニングの実装 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter12/12-2-dpo-training.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter12/12-2-dpo-training.ipynb) |
| | 12.3 選好チューニングの評価 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter12/12-3-dpo-evaluation.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter12/12-3-dpo-evaluation.ipynb) |
| 第 13 章 RAG | 13.1 RAG とは | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter13/13-1-rag-overview.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter13/13-1-rag-overview.ipynb) |
| | 13.2 基本的な RAG のシステムの実装 | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter13/13-2-rag-langchain.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter13/13-2-rag-langchain.ipynb) |
| | 13.3.1 AI 王データセットを用いた指示チューニング | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter13/13-3-1-rag-instruct.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter13/13-3-1-rag-instruct.ipynb) |
| | 13.3.2 指示チューニングしたモデルを LangChain で使う | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ghmagazine/llm-book/blob/main/chapter13/13-3-2-rag-instruct-langchain.ipynb) | [Link](https://github.com/ghmagazine/llm-book/blob/main/chapter13/13-3-2-rag-instruct-langchain.ipynb) |
| 第 14 章 分散並列学習 | 14.3 LLMの分散並列学習 | | [Link](https://github.com/ghmagazine/llm-book-Megatron-LM) |

## 正誤表

本書の正誤表は以下のページで公開しています。

- [「大規模言語モデル入門」正誤表](https://github.com/ghmagazine/llm-book/wiki/errata)
- [「大規模言語モデル入門Ⅱ〜生成型LLMの実装と評価」正誤表](https://github.com/ghmagazine/llm-book/wiki/errata2)

## リンク

- [Hugging Face Hub](https://huggingface.co/llm-book)
- [「大規模言語モデル入門」の技術評論社のページ](https://gihyo.jp/book/2023/978-4-297-13633-8)
- [「大規模言語モデル入門」のAmazon.co.jpのページ](https://www.amazon.co.jp/o/ASIN/4297136333/)
- [「大規模言語モデル入門Ⅱ〜生成型LLMの実装と評価」の技術評論社のページ](https://gihyo.jp/book/2024/978-4-297-14393-0)
- [「大規模言語モデル入門Ⅱ〜生成型LLMの実装と評価」のAmazon.co.jpのページ](https://www.amazon.co.jp/o/ASIN/4297143933)

llm-book

llm-book 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

llm-book

llm-book 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README