ai-gateway

KO | EN

gitlite — search

#gpt #devops #database #android #api #cloud-native #ai-agent #svg #data-science #ai-agents

(★ 9)

No description available.

File Explorer

ZIP Download

- - ci.yml
  - release.yml
- next-ai-gateway.js
- - multi-machine-edge-agent.config.json
  - multi-machine-toolhub.config.json
  - README.md
  - single-machine-edge-agent.config.json
  - single-machine-toolhub.config.json
- external-protocols.md
- plugins.md
- publishing.md
- scheduling-architecture.md
- usage.md
- benchmark-gateway-server.mjs
- release.mjs
- test-openai-responses-sdk.mjs
- - listener-readiness.ts
- - registry.test.ts
  - registry.ts
- - concurrency-limiter.ts
  - event-bus.test.ts
  - event-bus.ts
  - file-store.ts
  - guards.ts
  - index.ts
  - persistence.ts
  - prompt-builder.test.ts
  - prompt-builder.ts
  - provider-adapter.test.ts
  - provider-adapter.ts
  - provider-router.ts
  - publisher.test.ts
  - publisher.ts
  - README-TEST.md
  - request-identity.test.ts
  - request-identity.ts
  - resolve-tool-definitions.ts
  - retry.test.ts
  - retry.ts
  - routes.test.ts
  - routes.ts
  - runtime.test.ts
  - runtime.ts
  - session-lock.test.ts
  - session-lock.ts
  - store.ts
  - task-state.ts
  - tools.test.ts
  - tools.ts
  - transcript-window.ts
  - types.ts
- - calculate.test.ts
  - calculate.ts
  - index.ts
  - publisher.test.ts
  - publisher.ts
- - auth.test.ts
  - auth.ts
  - client-disconnect.test.ts
  - client-disconnect.ts
  - client-ip.test.ts
  - client-ip.ts
  - codex-websocket-conversion.test.ts
  - codex-websocket-conversion.ts
  - cors.test.ts
  - cors.ts
  - deepseek-live.test.ts
  - embeddings.test.ts
  - embeddings.ts
  - handler.test.ts
  - handler.ts
  - health-routing.test.ts
  - health-routing.ts
  - idempotency.test.ts
  - idempotency.ts
  - lenient-json-parser.test.ts
  - lenient-json-parser.ts
  - metrics.test.ts
  - metrics.ts
  - multipart.ts
  - openai-json.ts
  - openai-sdk-responses.integration.test.ts
  - policy.test.ts
  - policy.ts
  - precheck.test.ts
  - precheck.ts
  - protocol-conversion.bench.ts
  - provider-health-check.ts
  - provider-health-scheduler.test.ts
  - provider-health-scheduler.ts
  - provider-health.test.ts
  - provider-health.ts
  - readiness.test.ts
  - readiness.ts
  - routes.test.ts
  - routes.ts
  - runtime.ts
  - scheduler.test.ts
  - scheduler.ts
  - streaming-conversion.ts
  - strict-billing.test.ts
  - strict-billing.ts
  - upstream-circuit-breaker.test.ts
  - upstream-circuit-breaker.ts
  - upstream-concurrency.test.ts
  - upstream-concurrency.ts
  - video-compat.test.ts
  - video-compat.ts
  - websocket.test.ts
  - websocket.ts
- - index.ts
  - routes.test.ts
  - routes.ts
- - index.ts
  - jsonrpc.ts
  - policy.test.ts
  - policy.ts
  - routes.ts
  - runtime.oauth.test.ts
  - runtime.ts
  - websocket.ts
- - events.test.ts
  - events.ts
  - execution.ts
  - hooks.test.ts
  - hooks.ts
  - loader.test.ts
  - loader.ts
  - sdk.ts
- - deepseek-thinking.ts
  - external.test.ts
  - external.ts
  - plugins.test.ts
  - plugins.ts
  - webhook.test.ts
  - webhook.ts
- - ip.ts
  - pattern.ts
  - usage.test.ts
  - usage.ts
- - client.test.ts
  - client.ts
- config.test.ts
- config.ts
- external-config.test.ts
- external-config.ts
- external-event-sink.test.ts
- external-event-sink.ts
- external-json-source.ts
- grpc-json.ts
- index.ts
- raw-trace.ts
- redis-client.test.ts
- redis-client.ts
- sse.test.ts
- sse.ts
- test-setup.ts
- types.ts
- utils.test.ts
- utils.ts
.dockerignore
.env.production.example
.gitignore
.npmignore
build.js
dev.js
docker-compose.prod.yml
docker-compose.yml
Dockerfile
gateway.config.compose.json
gateway.config.production.json
LICENSE
package-lock.json
package.json
pnpm-lock.yaml
README.md
README.zh-CN.md
toolhub.config.compose.json
tsconfig.build.json
tsconfig.json
vitest.config.ts

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

[简体中文](README.zh-CN.md)

A TypeScript + Fastify AI protocol gateway for OpenAI, Anthropic, Gemini, MCP Gateway, and event-driven agent workflows.

## Quick Start

```bash
npm install
npm run dev
```

Production:

```bash
npm run build
npm start
```

Docker Compose:

```bash
export AUTH_STATIC_API_KEYS='replace-with-gateway-client-key'
export MANAGER_API_KEY='replace-with-manager-admin-key'
export OPENAI_API_KEY='sk-...'
export MCP_REMOTE_KEY='replace-with-strong-mcp-key'
export TOOLHUB_MANAGEMENT_TOKEN='replace-with-toolhub-admin-key'
export MINIMAX_API_KEY='replace-with-minimax-api-key'
export ANTHROPIC_API_KEY='replace-with-tool-search-provider-key'
docker compose up --build
```

Production Docker Compose:

```bash
cp .env.production.example .env.production
# edit .env.production
docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build
```

For multi-instance deployments, enable Redis-backed shared state by changing the
shared state backends in `.env.production` from `memory` to `redis`:

```dotenv
GATEWAY_IDEMPOTENCY_STORAGE_TYPE=redis
GATEWAY_UPSTREAM_CONCURRENCY_STORAGE_TYPE=redis
GATEWAY_UPSTREAM_CIRCUIT_BREAKER_STORAGE_TYPE=redis
GATEWAY_SCHEDULING_STORAGE_TYPE=redis
PROVIDER_HEALTH_CHECK_STORAGE_TYPE=redis
PRECHECK_STORAGE_TYPE=redis
```

Then start the Redis profile:

```bash
docker compose --env-file .env.production -f docker-compose.prod.yml --profile redis up -d --build
```

## Features

- OpenAI `chat/completions`, `responses`, `embeddings`, `moderations`, `images/generations`, `images/edits`, `videos`, `videos/:id`, and `videos/:id/content`
- xAI `videos/generations` and `videos/:id`, with OpenAI ↔ xAI video request/response conversion
- Authenticated, encrypted, expiring video IDs preserve routing/model/owner context without exposing it; configure a shared `media.videoIdSigningSecret` for multi-instance deployments
- The gateway does not hard-code an OpenAI video model. OpenAI has announced removal of the Videos API, `sora-2`, and `sora-2-pro` on September 24, 2026, with no replacement; pin provider models explicitly and plan migrations accordingly ([official deprecation notice](https://developers.openai.com/api/docs/deprecations#2026-03-24-sora-2-video-generation-models-and-videos-api))
- Anthropic `messages`
- Gemini `generateContent` / `streamGenerateContent`
- Cross-protocol conversion, provider fallback, provider plugins, health checks, metrics, idempotency, concurrency isolation, circuit breaking, and retries
- MCP Gateway, MCP WebSocket RPC, and event-driven agents
- HTTP / WebSocket / gRPC / stdio external config sources and event delivery

## Documentation

- [Full usage guide](docs/usage.md)
- [External protocol integration](docs/external-protocols.md)
- [Publishing and CI/CD](docs/publishing.md)
- [MCP WebSocket deployment templates](deploy/mcp-ws/README.md)

## npm Publishing

This repository includes `.npmignore`, `prepack` builds, GitHub Actions CI/CD, and a release command that publishes a specific npm version:

```bash
npm run release -- 1.2.3
```

See [Publishing and CI/CD](docs/publishing.md) for more options.

ai-gateway

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README