NLPNew

KO | EN

gitlite — search

#react #java #cloud-native #command-line #openai #api #machine-learning #data-science #linux #security

(★ 9)

No description available.

파일 탐색기

ZIP 다운로드

- - ci.yml
- README.md
- rag_experiments.json
- rag_sources.json
- history_instruct.jsonl
- rag_eval.jsonl
- rag_eval_results.json
- DATA_CARD.md
- EVALUATION.md
- MODEL_CARD.md
- PROJECT_DETAILS.md
- RUN_GUIDE.md
- - __init__.py
  - embeddings.py
  - text.py
  - vector_store.py
- __init__.py
- chat.py
- model.py
- prompting.py
- _bootstrap.py
- build_rag_database.py
- chat.py
- collect_history_sources.py
- download_dataset.py
- evaluate_rag.py
- generate_stage1.py
- prepare_context_sft.py
- search_rag.py
- train_sft.py
- __init__.py
- test_rag_pipeline.py
- vi_tokenizer.json
.editorconfig
.gitattributes
.gitignore
LICENSE
pyproject.toml
README.md
requirements.in
requirements.txt

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Vietnamese history question answering system built from a small custom GPT-style
language model, supervised fine-tuning data, and a SQLite-backed retrieval
pipeline. The project is designed as a reproducible NLP portfolio project: model
code, data preparation, retrieval, evaluation, tests, and runnable chat
entrypoints live in one repository.

## What This Project Demonstrates

- Custom causal Transformer in PyTorch: token/position embeddings, multi-head
  self-attention, residual blocks, layer norm, language-model loss, and
  autoregressive decoding.
- Supervised fine-tuning for Vietnamese historical QA with prompt masking so the
  model learns only from answer tokens.
- Retrieval-augmented generation using SQLite, FTS5 lexical search, dense
  embeddings, and reciprocal-rank-fusion hybrid search.
- Hallucination control through retrieval support checks, future/current-year
  abstention, context compression, and source reporting.
- Reproducible CLI scripts for dataset preparation, corpus collection, database
  building, retrieval evaluation, SFT, and chat.
- Unit tests for chunking, vector search, prompt formatting, data splitting, and
  abstention behavior.

## Architecture

Core package:

- `history_gpt/model.py`: MiniGPT Transformer implementation.
- `history_gpt/chat.py`: unified chat service for baseline and RAG modes.
- `history_gpt/rag/`: text normalization, embeddings, and SQLite vector store.
- `scripts/`: data, training, retrieval, evaluation, and chat entrypoints.
- `tests/`: unit coverage for the retrieval and SFT pipeline.

## Repository Layout

```text
configs/              RAG source manifest and experiment definitions
data/                 Small tracked datasets and evaluation reports
docs/                 Architecture, data, model, evaluation, and run guides
history_gpt/          Python package with model, prompting, chat, and RAG code
scripts/              Reproducible CLI workflows
tests/                Unit tests
tokenizer/            Vietnamese tokenizer JSON
checkpoints/          Local-only model weights, ignored by Git
```

Large `.pt`, `.sqlite`, generated corpus, and training artifacts are ignored by
Git. Keep trained checkpoints under `checkpoints/` locally or publish them as
release/model artifacts.

## Quick Start

```powershell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
python -B -m unittest discover -s tests -v
```

Run chat after placing a compatible checkpoint in `checkpoints/`:

```powershell
python scripts/chat.py --mode baseline --cli
python scripts/chat.py --mode dense --cli
python scripts/chat.py --query "Nhà Trần được thành lập năm nào?"
```

Rebuild the RAG database:

```powershell
python scripts/collect_history_sources.py --delay 0.5
python scripts/build_rag_database.py --device cpu --batch-size 16
python scripts/evaluate_rag.py --top-k 5
```

## Current Retrieval Results

The tracked evaluation report in `data/rag_eval_results.json` contains eight
Vietnamese history fact queries:

| Mode | Hit@5 | MRR |
| --- | ---: | ---: |
| Lexical FTS5 | 0.625 | 0.625 |
| Dense E5 | 1.000 | 0.875 |
| Hybrid RRF | 1.000 | 0.817 |

These are retrieval metrics, not end-to-end answer quality. See
[`docs/EVALUATION.md`](docs/EVALUATION.md) for details and limitations.

## Data And Licensing

- `data/history_instruct.jsonl` is normalized from
  [`minhxthanh/Vietnam-History-1M-Vi`](https://huggingface.co/datasets/minhxthanh/Vietnam-History-1M-Vi),
  which is listed on Hugging Face with MIT license metadata.
- `configs/rag_sources.json` includes both redistributable and local-index-only
  sources. Generated RAG corpora and SQLite databases are not committed.
- See [`docs/DATA_CARD.md`](docs/DATA_CARD.md) before redistributing generated
  corpora or trained weights.

## Portfolio Summary

Built an end-to-end Vietnamese history QA system with a custom PyTorch
Transformer, SFT pipeline, SQLite/FTS5/dense RAG retrieval, source-aware
abstention logic, evaluation scripts, and unit tests. The project emphasizes
reproducibility, artifact hygiene, and retrieval-based hallucination reduction.

## Limitations

- The model is small and educational; it is not a production-grade historical
  authority.
- Checkpoints are not committed because they are large binary artifacts.
- RAG quality depends on the generated local corpus and embedding model.
- Some configured source pages are for local indexing only and should not be
  redistributed as raw text without checking their licenses.

## Documentation

- [Run Guide](docs/RUN_GUIDE.md)
- [Project Details](docs/PROJECT_DETAILS.md)
- [Data Card](docs/DATA_CARD.md)
- [Model Card](docs/MODEL_CARD.md)
- [Evaluation](docs/EVALUATION.md)

NLPNew

파일 탐색기

# CDN으로 사용하기

Was this content helpful?

Recent Feedback

Download README