SmarterRouter
SmarterRouter: An intelligent LLM gateway and VRAM-aware router for Ollama, llama.cpp, and OpenAI. Features semantic caching, model profiling, and automatic failover for local AI labs.
File Explorer
Download Latest Version (.zip)- security.yml
- artificial_analysis_models.example.yaml
- client-integration.md
- openwebui-integration.md
- production-deployment.md
- README.md
- api.md
- architecture.md
- backends.md
- benchmarks.md
- configuration.md
- contributing.md
- docker-compose.amd.yml
- docker-compose.apple.md
- docker-compose.intel.yml
- docker-compose.multi-gpu.yml
- docker-compose.nvidia.yml
- DOCKER.md
- external-providers.md
- installation.md
- kubernetes.md
- performance.md
- troubleshooting.md
- clusters.json
- predictor.npz
- weights_down.bin
- weights_down.npy
- weights_gate.bin
- weights_gate.npy
- weights_up.bin
- weights_up.npy
- clusters.json
- predictor.npz
- weights_down.bin
- weights_down.npy
- weights_gate.bin
- weights_gate.npy
- weights_up.bin
- weights_up.npy
- layer_0_down.npy
- layer_0_gate.npy
- layer_0_up.npy
- layer_1_down.npy
- layer_1_gate.npy
- layer_1_up.npy
- metadata.json
- __init__.py
- admin.py
- chat.py
- health.py
- models.py
- __init__.py
- base.py
- external.py
- llama_cpp.py
- ollama.py
- openai.py
- registry.py
- resilience.py
- retry.py
- __init__.py
- amdgpu.py
- apple.py
- base.py
- intel.py
- nvidia.py
- __init__.py
- artificial_analysis.py
- base.py
- huggingface.py
- lmsys.py
- __init__.py
- audit.py
- benchmark_db.py
- benchmark_sync.py
- cache.py
- cache_redis.py
- cache_stats.py
- circuit_breaker.py
- cli.py
- config.py
- database.py
- dlq.py
- encryption.py
- entrypoint.py
- exceptions.py
- judge.py
- lifecycle.py
- logging_config.py
- metrics.py
- middleware.py
- modality.py
- model_filter.py
- model_metadata.py
- models.py
- persistent_cache.py
- profiler.py
- prompts.py
- provider_db.py
- router.py
- schemas.py
- security.py
- skills.py
- state.py
- vram_manager.py
- vram_monitor.py
- apply_optimizations.py
- apply_router_optimizations.py
- download_provider_db.sh
- fix_schema.py
- optimize_performance.py
- test_ollama_real.py
- k6-script.js
- locustfile.py
- __init__.py
- conftest.py
- test_admin_ip_whitelist.py
- test_amd_backend.py
- test_audit.py
- test_backend_contract.py
- test_backend_failover.py
- test_backend_registry.py
- test_backend_streaming_kwargs.py
- test_benchmark_sync.py
- test_cache_persistence.py
- test_cache_redis.py
- test_cache_stats.py
- test_caching.py
- test_concurrency_stress.py
- test_config.py
- test_database.py
- test_dlq.py
- test_edge_cases.py
- test_embeddings_feedback.py
- test_entrypoint.py
- test_external_backend.py
- test_integration.py
- test_integration_extended.py
- test_judge.py
- test_judge_markdown.py
- test_llama_cpp_backend.py
- test_modality.py
- test_model_filter.py
- test_model_metadata.py
- test_ollama_backend.py
- test_ollama_multimodal_fix.py
- test_openai_backend.py
- test_profiler.py
- test_property_based.py
- test_provider_db.py
- test_provider_fixtures.py
- test_providers.py
- test_quota_detection.py
- test_router.py
- test_schemas.py
- test_security.py
- test_security_edge_cases.py
- test_snapshot_routing.py
- test_vram_manager.py
- test_vram_monitor.py
- .dockerignore
- .gitignore
- artificial_analysis_models.example.yaml
- CHANGELOG.md
- CONTRIBUTING.md
- debug_explain.py
- DEEPDIVE.md
- docker-compose.external.yml
- docker-compose.yml
- docker-run.sh
- Dockerfile
- ENV_DEFAULT
- LICENSE
- main.py
- pyproject.toml
- README.md
- requirements.txt
- SECURITY.md
- test_model.gguf
# Installation Guide
1. Get the code
git clone https://github.com/peva3/SmarterRouter
Downloads the entire project code from GitHub to your computer.
cd SmarterRouter
Moves into the project folder you just downloaded.
2. Docker
Easy RecommendedPrerequisites
- Git Needed to download the project code from GitHub.
- Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
docker-compose up -d
Runs the command against the services defined in the compose file.
chmod +x docker-run.sh
Type this command into your terminal and run it.
./docker-run.sh
Type this command into your terminal and run it.
./docker-run.sh --port 11436 --data-dir ./smarterrouter-data --env-file .env
Type this command into your terminal and run it.
Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.
Pulled directly from this repo's README.
3. Python
EasyPrerequisites
python -m smarterrouter setup
Runs the Python script (or module).
python -m smarterrouter check
Runs the Python script (or module).
python -m smarterrouter generate-env
Runs the Python script (or module).
If it runs without errors and prints output in the terminal, it worked.
Pulled directly from this repo's README.
// repository documentation
Was this content helpful?
(0 ratings)