MCP-powered-Agentic-RAG - GitLite

KO | EN

gitlite — search

#cli #chatbot #nodejs #awesome #hacktoberfest #nlp #terminal #agent #svg #nextjs

MCP-powered-Agentic-RAG

(★ 87)

An MCP-powered agentic RAG system that transcribes, indexes, and intelligently queries audio content using advanced retrieval-augmented generation.

File Explorer

ZIP Download

- - .gitkeep
  - ai_fundamentals.txt
- .gitkeep
- .gitkeep
- mcp-agentic-rag.gif
- .gitkeep
- __init__.py
- chromadb_tool.py
- .gitkeep
- chroma.sqlite3
.env.example
00_START_HERE.md
API.md
CONTRIBUTING.md
EXECUTIVE_SUMMARY.md
FILE_INDEX.md
GETTING_STARTED.md
IMPLEMENTATION_COMPLETE.md
INSTALLATION_CHECKLIST.md
LICENSE
main.py
mcp_config.yaml
PROJECT_SUMMARY.md
quickstart.py
rag_agent.py
README.md
requirements.txt
streamlit_app.py
test_utils.py

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

Command Glossary

Commands referenced in this DOCs, explained below.

🔍

`ollama`

View Details ▼

A large language model runner.
For a list of available models, see <https://ollama.com/library>.

ollama serve

Start the daemon required to run other commands:

ollama run {{model}}

Run a model and chat with it (will automatically download the model if it's not downloaded):

ollama run {{model}} --think=false "{{prompt}}"

Run a model with a single prompt and thinking turned off:

🔍

`pip install`

View Details ▼

Install Python packages.

pip install {{package1 package2 ...}}

Install one or more packages:

pip install {{package1 package2 ...}} {{[-U|--upgrade]}}

Upgrade all specified packages to the latest version, installing any that are not already present:

pip install {{package}}=={{version}}

Install a specific version of a package:

🔍

`python`

View Details ▼

Python language interpreter.

python

Start a REPL (interactive shell):

python {{path/to/file.py}}

Execute a specific Python file:

python -i {{path/to/file.py}}

Execute a specific Python file and start a REPL:

🔍

`streamlit`

View Details ▼

Framework for creating interactive, data-driven web apps in Python.

streamlit hello

Check for the Streamlit installation:

streamlit run {{project_name}}

Run a Streamlit application:

streamlit --help

Display help:

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

A local, modular Retrieval-Augmented Generation (RAG) system using the Model Context Protocol (MCP) to connect an LLM to external tools like vector databases and document loaders.

<img src="/images/mcp-agentic-rag.gif" alt="description" style="width:100%; height:auto;" />

## Overview

This project implements an agentic RAG system that:
- **Retrieves** relevant documents from a local vector database (ChromaDB)
- **Augments** prompts with retrieved context
- **Generates** informed responses using a local LLM (Ollama)
- **Exposes** functionality via REST API with FastAPI

## Tech Stack

| Component | Tool/Library | Details |
|-----------|---|---|
| **Language Model** | Ollama | Local LLM inference (mistral, llama3, etc.) |
| **Agent Framework** | mcp + FastAPI | API server with tool registration |
| **RAG Pipeline** | LangChain + Custom | Context retrieval and prompt engineering |
| **Vector Store** | ChromaDB | Local, persistent vector database |
| **Embeddings** | SentenceTransformers | all-MiniLM-L6-v2 model |
| **File Handling** | pypdf, python-docx | PDF and document loading |
| **Frontend (Optional)** | Streamlit | Interactive web UI |
| **Environment** | Python 3.10+ | virtualenv or Conda |

## Project Structure

```
agentic-rag-mcp/
├── main.py                    # FastAPI MCP server
├── rag_agent.py              # Agent query logic and RAG orchestration
├── mcp_config.yaml           # Configuration file
├── requirements.txt          # Python dependencies
├── vector_store/             # Persisted ChromaDB vector store
├── data/
│   └── sample_docs/          # Sample documents for ingestion
└── tools/
    └── chromadb_tool.py      # Vector search tool implementation
```

## Installation & Setup

### 1. Clone and Create Virtual Environment

```bash
cd agentic-rag-mcp
python -m venv .venv

# On Windows
.venv\Scripts\activate

# On macOS/Linux
source .venv/bin/activate
```

### 2. Install Dependencies

```bash
pip install -U pip
pip install -r requirements.txt
```

### 3. Set Up Ollama

Download and install [Ollama](https://ollama.ai) from the official website.

Start the Ollama server:

```bash
# On the system terminal (not in virtual environment)
ollama serve
```

In another terminal, pull a model:

```bash
ollama pull mistral    # Recommended for RAG
# or
ollama pull llama3
```

Verify the server is running:

```bash
curl http://localhost:11434/api/tags
```

## Running the System

### Option 1: Chat Interface (Interactive)

Run the interactive chat loop:

```bash
python rag_agent.py
```

This will:
1. Load sample documents into the vector store
2. Start an interactive chat where you can ask questions
3. The agent will retrieve relevant documents and generate answers

Example interaction:
```
You: What is MCP?
Agent: The Model Context Protocol (MCP) enables modular tool use for AI agents by providing a standardized way to connect language models to external services...

[Used 2 retrieved documents as context]
```

### Option 2: API Server

Start the FastAPI MCP server:

```bash
python main.py
```

The server will be available at: `http://localhost:8000`

### API Endpoints

#### Health Check
```bash
GET /health
```

#### Query Agent
```bash
POST /query
Content-Type: application/json

{
  "query": "What is artificial intelligence?",
  "use_context": true,
  "n_results": 3
}
```

#### Search Documents
```bash
POST /search
Content-Type: application/json

{
  "query": "MCP protocol",
  "n_results": 5
}
```

#### Add Documents
```bash
POST /documents
Content-Type: application/json

{
  "documents": [
    "Document text 1",
    "Document text 2"
  ],
  "ids": ["doc1", "doc2"],
  "metadata": [
    {"source": "file1.txt"},
    {"source": "file2.txt"}
  ]
}
```

#### Get Statistics
```bash
GET /stats
```

### Python Usage Examples

```python
from rag_agent import RAGAgent

# Initialize agent
agent = RAGAgent(
    ollama_url="http://localhost:11434",
    model="mistral"
)

# Get a response
result = agent.get_response("What is RAG?")
print(result["response"])
print(f"Retrieved {len(result['retrieved_documents'])} documents")
```

## Configuration

Edit `mcp_config.yaml` to customize:

- **LLM Settings**: Model, temperature, max tokens
- **Vector Store**: Embedding model, collection name
- **RAG**: Number of retrieved documents, similarity metric
- **Server**: Host, port, log level
- **Security**: API rate limits, authentication

## Adding Custom Documents

### Programmatically

```python
from tools.chromadb_tool import ChromaTool

tool = ChromaTool()
documents = [
    "Your document text 1",
    "Your document text 2"
]
tool.add_documents(documents, ids=["id1", "id2"])
```

### Via API

```bash
curl -X POST http://localhost:8000/documents \
  -H "Content-Type: application/json" \
  -d '{
    "documents": ["Document 1", "Document 2"],
    "ids": ["doc1", "doc2"]
  }'
```

## Optional Streamlit Frontend

Create `streamlit_app.py`:

```python
import streamlit as st
import requests

st.set_page_config(page_title="RAG Agent", layout="wide")
st.title("MCP-Powered Agentic RAG")

query = st.text_input("Ask a question:")

if query:
    response = requests.post(
        "http://localhost:8000/query",
        json={"query": query}
    )
    result = response.json()
    
    st.subheader("Response")
    st.write(result["response"])
    
    st.subheader("Retrieved Context")
    for i, doc in enumerate(result["retrieved_documents"], 1):
        st.write(f"**Doc {i}**: {doc[:200]}...")
```

Run Streamlit:
```bash
streamlit run streamlit_app.py
```

## Extensions & Future Work

- ✅ Basic RAG with ChromaDB
- ⬜ Web search tool integration
- ⬜ PDF document ingestion UI
- ⬜ Agent memory (conversation history)
- ⬜ Multi-modal support (images, tables)
- ⬜ Fine-tuning on domain-specific data
- ⬜ Structured output (JSON schemas)
- ⬜ Real-time streaming responses

## Troubleshooting

### "Connection refused" for Ollama
- Make sure Ollama server is running: `ollama serve`
- Check it's accessible: `curl http://localhost:11434/api/tags`

### ChromaDB embedding errors
- Ensure sentence-transformers is installed: `pip install sentence-transformers`
- First run downloads embeddings model (~30MB)

### Vector store not persisting
- Check `./vector_store/` directory exists and is writable
- Verify `persist_dir` in configuration matches actual path

## License

MIT License - See LICENSE file for details

## Contributing

Contributions are welcome! Please:
1. Fork the repository
2. Create a feature branch
3. Commit changes
4. Push and open a pull request

## References

- [Model Context Protocol](https://modelcontextprotocol.io/)
- [Ollama Documentation](https://ollama.ai)
- [ChromaDB Documentation](https://docs.trychroma.com/)
- [LangChain RAG Guide](https://python.langchain.com/docs/use_cases/question_answering/)
- [FastAPI Documentation](https://fastapi.tiangolo.com/)