n8n-migration-toolkit

KO | EN

gitlite — search

#javascript #hacktoberfest #react #typescript #llm #golang #android #machine-learning #rust #kubernetes #security #pytorch #mcp #open-source

n8n-migration-toolkit

(★ 9)

No description available.

n8n-migration-toolkit Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# n8n to Pure Code: The Zero-Downtime Migration Toolkit

A guided, interactive toolkit for migrating production [n8n](https://n8n.io)
workflows to pure Python/FastAPI code — without downtime, without losing
behavior, without flying blind.

You run it inside [Claude Code](https://claude.com/claude-code). It walks you
through 7 phases, asks before doing anything risky, and produces a tested,
benchmarked, deployment-ready Python application that mirrors your n8n
behavior byte-for-byte.

This toolkit was built while migrating a real production bot off n8n. The
hard-won lessons in this README all come from things that actually went wrong
the first time.

---

## Why Migrate Off n8n?

n8n is excellent for prototyping. It's a different tool once you're running
it in production at scale. After migrating a real n8n production workflow
serving **600+ active users** with multiple scheduled jobs, AI agent
integration, and a PostgreSQL backend, here's what changed:

### Real resource savings

| Metric | n8n stack | Pure code | Change |
|---|---|---|---|
| RSS memory (idle) | **877 MB** | **52 MB** | **94% reduction** (17× smaller) |
| Docker image size | 1.95 GB | 284 MB | **6.9× smaller** |
| Webhook ack latency | 430 ms median | **<100 ms** | async via FastAPI BackgroundTasks |
| Lifetime CPU (5 weeks) | 1.79% of 1 vCPU | comparable | structurally identical for LLM-bound work |
| Smallest fitting droplet | $18/mo (2 GB / 2 vCPUs) | **$6/mo (1 GB)** | $144/year savings |

The memory number is the headline win. n8n carries the visual editor, the
workflow engine, execution history storage, credential manager, queue runners,
and the UI server in every container — none of which a pure code deployment
needs. That's structural, not configurable.

### What you get back beyond hardware

- **Version control.** Every change is a diff in git, reviewable, blameable,
  reverteable.
- **Real testability.** `pytest` against your business logic. Mock the
  expensive integrations. Run the entire suite in 30 seconds locally.
- **Standard debugging.** `pdb`, `print()`, your IDE breakpoints, full
  stack traces. No more clicking through node UIs to figure out what an
  expression evaluated to at runtime.
- **No platform upgrade risk.** n8n's monthly releases occasionally break
  workflows. Your Python code does what your tests say it does.
- **Faster feature development.** Adding a new endpoint takes minutes, not
  a click-and-drag session through the editor.

### What the trade-off honestly costs

- Non-developers can no longer edit workflows. If your team includes
  ops/PMs who tweak workflows directly, that goes away.
- You own the deployment. n8n's "click to deploy" disappears.
- Initial migration takes real effort. The toolkit reduces it dramatically,
  but it's not zero.

If those trade-offs are acceptable, the rest of this guide gets you there.

---

## What This Toolkit Does

It runs as a set of Claude Code skills + slash commands. You type `/start`
inside Claude Code, and an AI assistant guides you through 7 phases in
strict order, asking before every risky action.

### The 7-phase approach

| # | Phase | What happens |
|---|---|---|
| 1 | **Discovery** | Connects to your n8n REST API, catalogs every workflow, credential, integration, and (if SSH provided) baseline server resources. |
| 2 | **Analysis** | Parses workflow JSON into dependency graphs, identifies every node type, decomposes large workflows into subgraphs, extracts test fixtures from real execution history. |
| 3 | **Credentials** | For every external integration, you decide: real test credentials, mock, or production-with-acknowledged-risk. Generates an isolated `.env`. |
| 4 | **Translation** | Converts each subgraph to Python/FastAPI code, one node at a time, with byte-for-byte fidelity to Code-node content (no paraphrasing). Generates routers, services, models, and mocks. |
| 5 | **Testing** | Generates `pytest` cases from real execution history. Includes a smoke suite with admin trigger endpoints, a test-capture buffer for outgoing messages, and (opt-in) real-LLM tests for prompt-quality coverage. |
| 6 | **Benchmarking** | 3-layer comparison: idle baseline, per-interaction wall time, extrapolation to production load. Honest fairness caveats included. |
| 7 | **Migration Plan** | Generates a custom zero-downtime cutover plan tailored to your infrastructure: shadow deployment, traffic mirroring, gradual cutover, rollback. |

The assistant **NEVER** skips a phase. **NEVER** starts the next phase
without your confirmation.

### Folder structure

```
n8n-migration-toolkit/
├── CLAUDE.md                           # The assistant's operating instructions
├── verify.sh                           # Sanity-check the toolkit is intact
├── .claude/
│   ├── settings.json                   # Permissions: what tools auto-run vs prompt
│   ├── hooks/pre-bash-safety.sh        # Pre-execution guard against destructive ops
│   ├── commands/                       # Slash commands (/start, /status, /resume, /take-notes)
│   └── skills/                         # 7 phase-specific skill instructions
│       ├── discovering-n8n-instance/
│       ├── analyzing-workflows/
│       ├── collecting-test-credentials/
│       ├── translating-workflows/      # + references/translation-patterns.md
│       ├── testing-translated-code/
│       ├── benchmarking-resources/
│       └── planning-zero-downtime-migration/
├── migration-state/                    # Generated as you go (gitignored)
│   ├── discovery/   analysis/   benchmarks/   notes/
└── workspace/                          # Generated application (gitignored)
    ├── src/   tests/   config/   scripts/   docs/
```

### How the skill system works

Claude Code skills are markdown files that the AI loads into context only
when relevant to the current task. That keeps each phase focused: when
you're in Phase 4 (translation), the assistant isn't carrying the discovery
or benchmarking instructions in its head — it loads `translating-workflows/SKILL.md`
and works from there.

Each `SKILL.md` has a frontmatter `description` that tells the assistant
when to activate the skill. You don't need to invoke them manually — the
slash commands and phase progression handle activation automatically.

---

## Prerequisites

- **[Claude Code](https://claude.com/claude-code)** installed and signed in
- **n8n instance** with REST API access (Settings → n8n API → Create API Key)
- **PostgreSQL knowledge** — the toolkit assumes Postgres as the database
- **Test credentials** for every external integration your workflows touch
  (the toolkit walks you through setting these up)
- **Basic Python familiarity** — you'll be reading the generated code, even
  if you don't write it from scratch
- **(Optional) SSH access** to the n8n host for accurate resource baselines.
  Read-only is fine; the toolkit only runs `top`, `df`, `free`, `docker stats`.

The generated code targets:

| Layer | Choice |
|---|---|
| Language | Python 3.12+ |
| Framework | FastAPI (async) |
| Validation | Pydantic v2 |
| HTTP Client | httpx (async) |
| Database | PostgreSQL via asyncpg |
| Scheduling | APScheduler |
| Deployment | Docker Compose |
| Proxy | Nginx |
| Testing | pytest + pytest-asyncio |

These are locked. The assistant won't propose alternatives without your
explicit approval.

---

## Quick Start

```bash
git clone <repo-url> n8n-migration-toolkit
cd n8n-migration-toolkit
chmod +x .claude/hooks/pre-bash-safety.sh
chmod +x verify.sh
./verify.sh                # confirm toolkit structure is intact
claude                     # launch Claude Code in this directory
```

Then inside Claude Code:

```
/start
```

The assistant will introduce itself, list the 7 phases, and ask for your
n8n URL + API key. From there, follow the conversation. You're in control —
the assistant explains every action before taking it.

---

## The 7 Phases (In Detail)

### Phase 1 — Discovery

**What happens:** The assistant connects to your n8n REST API and pulls
every workflow definition, credential metadata, recent execution history
(with full per-node I/O via `?includeData=true`), and — if you provided SSH —
container resource baselines, cgroup CPU lifetime, container TZ, and host
shape (vCPU/RAM/droplet size).

**Sample conversation:**

```
You: /start
Assistant: Welcome to the n8n Migration Toolkit. I'll guide you through
  migrating your production n8n workflows to pure Python/FastAPI code with
  zero downtime. ... To begin, I need: your n8n URL, an API key, and
  optionally SSH access for server metrics.

You: https://n8n.example.com / API key abc123 / ssh user@host
Assistant: Verifying connectivity... ✓
  I found 2 workflows (2 active), 7 credentials, 14 unique node types,
  and 250 executions over the last 7 days. The container is on
  Africa/Cairo timezone, lifetime CPU averages 1.8% of one vCPU, and
  the host is a Basic 2 GB / 1 vCPU droplet.

Top 3 largest workflows:
    1. Main Bot — 87 nodes
    2. Utilities — 23 nodes
    3. Weekly Report — 11 nodes

Ready to proceed to Phase 2 (Analysis)?
```

**How long it takes:** 5–15 minutes depending on workflow count and SSH availability.

**Top pitfall:** Pulling executions WITHOUT `?includeData=true` from the
start. Without the flag you only get IDs and timing — and you'll re-fetch
every execution later for both fixture extraction and benchmarking. The
toolkit pulls with data on first contact.

### Phase 2 — Analysis

**What happens:** Parses each workflow JSON into a dependency graph.
Identifies entry points, branch points, merge points, error paths,
disabled nodes (which are flagged as "deliberate divergences" and skipped
by default in translation). For workflows with 50+ nodes, decomposes them
into subgraphs that fit in context. Extracts test fixtures from real
execution history.

**Sample conversation:**

```
Assistant: Workflow 'Main Bot' has 87 nodes — too large for a single
  translation pass. I'm proposing 4 subgraphs:
    sg_a — Entry routing (12 nodes)
    sg_b — Onboarding flow (24 nodes)
    sg_c — Prayer logic (38 nodes)
    sg_d — Error handling (13 nodes)

I also found 2 disabled nodes you should know about:
    - "Daily 4am Check" is OFF in production. By default I will NOT
      translate it. Re-enable in dev? (yes/no, default: no)

You: no
Assistant: Got it — flagged as deliberate divergence. ...
```

**How long it takes:** 5–20 minutes. Mostly fast unless you have
unknown community nodes — in which case the assistant asks you to either
paste the docs URL or describe what the node does.

**Top pitfall:** Letting the assistant translate disabled nodes. They're
disabled in production for a reason. The toolkit defaults to skipping
them and surfacing them as a divergence report.

### Phase 3 — Credentials

**What happens:** For each external integration the workflows touch, the
assistant offers 3 strategies: real test credentials (recommended), mock,
or production-with-acknowledged-risk. For Telegram bots specifically, the
assistant walks you through @BotFather to create a test bot if you don't
have one.

**Sample conversation:**

```
Assistant: Integration 1 of 5: Telegram Bot
  Used by: Main Bot, Utilities
  Side effects: Yes — sends messages to users
  How would you like to handle this for testing?

[a] Walk me through @BotFather (recommended)
  [b] I already have a test bot — paste the token
  [c] Use production credentials (NOT RECOMMENDED)

You: a
Assistant: Open Telegram and message @BotFather. Send /newbot. Give it
  a name like "MyBot Dev". Send a username ending in `bot`. Paste the
  token here when BotFather replies.
```

**How long it takes:** 15–45 minutes depending on how many integrations
you have and whether you already have test instances ready.

**Top pitfall:** Skipping this phase or using production credentials in
dev. The assistant requires you to type "I understand the risks" verbatim
before allowing production credentials, and even then warns aggressively.

### Phase 4 — Translation

**What happens:** Converts each subgraph to Python/FastAPI code, one node
at a time, in topological order. Generates routers, services, Pydantic
models, and mock modules for any integration set to "mock" strategy.
Every translated expression gets an inline comment showing the original
n8n syntax for traceability.

**Sample conversation:**

```
Assistant: Translating subgraph sg_b (Onboarding, 24 nodes).
  I will create:
    workspace/src/routers/onboarding_router.py
    workspace/src/services/onboarding_service.py
    workspace/src/models/onboarding_models.py

This subgraph contains a Code node 'Build Welcome Message' (180 lines
  of JS with Arabic message pools). I will port it byte-for-byte — no
  paraphrasing. Approve?

You: yes
```

**How long it takes:** 30 minutes – 4 hours depending on workflow
complexity. Most of the time is the assistant working through nodes;
your involvement is approving file plans and answering occasional
clarifying questions.

**Top pitfall:** Letting the assistant paraphrase Code-node content.
The first migration that built this toolkit paraphrased an 800-line
"Calculate Stats" Code node down to 35 lines, dropping entire scenario
branches and message pools. The toolkit now has a hard rule (§0.2 in
`translating-workflows/SKILL.md`): for Code nodes >50 lines, manually
re-port byte-for-byte. Watch this one carefully.

### Phase 5 — Testing

**What happens:** Generates a `pytest` suite with three layers — unit
tests from execution-history fixtures, integration tests against your
test credentials, and parity tests that compare code output to n8n
output for the same input. Also generates a smoke suite with admin
trigger endpoints (`/admin/trigger/<job>`), seed endpoints
(`/admin/db/seed-test-user`), and a test-capture buffer that intercepts
outgoing Telegram messages by chat_id range.

**Sample conversation:**

```
Assistant: Generated 74 tests across 17 sections. Running...

Unit Tests:        62/62 passed
  Integration Tests: 8/8 passed
  Parity Tests:      4/4 passed

Real-LLM tests are opt-in. Run them now with --with-llm? (recommended
  for prompt-quality coverage)

You: yes
Assistant: Running 6 real-LLM tests... 6/6 passed.
  All 80 tests pass. Ready to proceed to Phase 6 (Benchmarking)?
```

**How long it takes:** 20 minutes – 2 hours including fix cycles.

**Top pitfall:** Picking a `TEST_USER_PREFIX` below 10¹³. Telegram user
IDs are now in the **8–9 billion** range (8 × 10⁹) and growing. Anything
below 10¹⁰ is unsafe today; the toolkit defaults to **10¹³ = 10_000_000_000_000**
(ten trillion). The first version of the toolkit used 999_000_000 — the
test cleanup deleted a real user's row and the test-capture buffer
swallowed real production messages. Don't repeat that.

### Phase 6 — Benchmarking

**What happens:** A 3-layer fair comparison of n8n vs the new code:
(1) idle baseline using cgroup `cpu.stat` (NOT `docker stats` — see below),
(2) per-interaction wall time across 13 representative scenarios,
(3) extrapolation to production load using real row counts and execution
history. Generates `migration-state/benchmarks/report.md` with honest
fairness caveats.

**Sample conversation:**

```
Assistant: Before I design the benchmark harness, I want to confirm
  fairness: production has many users; dev runs with 1 test user. I'll
  normalize by:
    - Idle baseline (RAM + steady-state CPU) — not normalized
    - Per-interaction (wall time per scenario) — not normalized
    - Extrapolation (lifetime CPU at production load) — normalized
  Sound fair?

You: yes
Assistant: [...] Benchmark complete. Headline:
  - 94% memory reduction (877 MB → 52 MB)
  - 17× smaller image (1.95 GB → 284 MB)
  - $12/mo savings ($144/year)
  See migration-state/benchmarks/report.md for full breakdown with caveats.
```

**How long it takes:** 30 minutes – 2 hours. The 10-minute idle window
is mandatory; everything else is fast.

**Top pitfall:** Using `docker stats --no-stream` for CPU. It's a
1-second snapshot and has nasty artifacts (Postgres routinely shows 0%
then 100% in successive samples). The toolkit reads cumulative cgroup
CPU time as the authoritative number.

### Phase 7 — Migration Plan

**What happens:** Generates a custom zero-downtime cutover plan based on
your specific infrastructure — DNS provider, reverse proxy, maintenance
window, risk tolerance, rollback requirements. Includes shadow
deployment via nginx path prefix, traffic mirroring config, gradual
cutover steps, integration-specific webhook switchover instructions
(Telegram, Stripe, etc.), and rollback procedures at three levels
(instant, fast, full).

**How long it takes:** 30 minutes for the plan; the actual cutover is
on your schedule (typically 1-2 weeks of shadow + 1-3 days of gradual).

**Top pitfall:** Cutting over without the shadow phase. The toolkit
recommends side-by-side deployment via nginx path prefix
(`/devbot/...` on your existing TLS) so production never gets touched
until you're confident. Skipping the shadow phase makes rollback
expensive.

---

## Real-World Results

**Project context:** A real n8n production workflow serving **600+ active
users**, with multiple scheduled jobs (heartbeat every 15 minutes, daily
feedback, weekly reports), an AI agent integration with tool use, a
PostgreSQL backend, and a webapp dashboard. 100K+ executions per month
across 2 workflows totaling ~120 nodes.

### Headline numbers (real measurements)

| Metric | n8n | Pure code | Change |
|---|---|---|---|
| **RSS memory (idle)** | 877 MB | **52 MB** | **94% reduction** |
| **Docker image size** | 1.95 GB | 284 MB | 6.9× smaller |
| **Webhook ack latency** | 430 ms median | **<100 ms** | async via BackgroundTasks |
| **End-to-end chat latency** | 430 ms median | ~430 ms | identical (LLM round-trip dominates) |
| **Lifetime CPU avg (5 weeks)** | 1.79% of 1 vCPU | comparable | structurally similar |
| **Smallest fitting droplet** | $18/mo (Basic 2 GB / 2 vCPUs) | **$6/mo (Basic 1 GB)** | $144/year savings |

The webhook ack number is interesting beyond raw cost: n8n is
synchronous — it holds the TCP connection open until the entire workflow
finishes. The new code uses FastAPI BackgroundTasks to ack in <100 ms
and process in the background. End-to-end latency is the same (the LLM
round-trip dominates), but the platform-side cost of holding connections
disappears, and traffic spikes can be absorbed without backpressure.

### Honest assessment

- The first translation pass achieved roughly **90% behavioral parity**.
  The remaining 10% required **2-3 parity fix rounds**, mostly catching
  message-pool drift in long Code nodes and `asyncpg` type-binding
  issues. The toolkit's `translating-workflows/SKILL.md` §0 codifies
  every gotcha we hit so you don't repeat them.
- Total wall time end-to-end (Discovery → tested + benchmarked + cutover plan):
  approximately **2-3 long working days** for a 4-workflow / 120-node
  project. Most of that was the assistant working; my involvement was
  approving plans and answering clarifying questions.
- The dev application has been running side-by-side with n8n via an
  nginx path prefix (`/devbot/...`) for the entire validation period.
  Production was never touched.

---

## What You Will Miss From n8n (And How to Replace It)

n8n's UI provides several conveniences that disappear in pure code. Each
has a working substitute — but they're substitutes, not equivalents.

| n8n feature | Pure-code replacement |
|---|---|
| **Execution history UI** | Structured logs (`logger.info` / JSON logs to stdout) + log aggregator of choice. Less visual, more searchable. |
| **Visual debugging** | `pytest` + inline comments showing original n8n expressions (`# TRANSLATED: {{ $json.field }}`). Set a breakpoint, step through. |
| **One-click "Execute Workflow"** | Admin trigger endpoints (`/admin/trigger/<job_name>`, bearer-token guarded). Fire any scheduled job on demand from curl or a script. |
| **Credential UI** | `.env` files via `pydantic-settings`. Source-controlled `.env.example`, gitignored `.env`. |
| **Workflow activation toggle** | Feature flags via env vars or a config table. |
| **Built-in retry/error workflow** | `@backoff.on_exception(httpx.HTTPError, max_tries=3)` decorators + a global `@app.exception_handler` for unhandled errors. |
| **Non-developer editing** | This is the genuine trade-off. Workflow changes now require a code edit + PR + redeploy. If non-developers tweak workflows in your org, evaluate this honestly before committing. |

---

## Troubleshooting

### "Claude Code lost context mid-translation"

Use `/status` to see where you are in the manifest, then `/resume` to
pick up from the last incomplete subgraph. The toolkit tracks progress
in `migration-state/manifest.json` so context loss is recoverable.

### "Parity tests are failing"

Expected on first pass — usually 1-3 fix rounds before everything is
green. The assistant's parity-test loop is:

1. Identify the exact node where divergence occurs
2. Show you the n8n node config vs the translated code
3. Propose a fix
4. WAIT for your approval
5. Re-run after fix

Don't proceed to Phase 6 until all parity tests pass.

### "Unknown node type"

The analyzing-workflows skill tries 3 layers: official n8n docs URL,
trigger-node URL, core-node URL. If all 404, it asks you to either
paste the docs URL or describe what the node does. Community nodes
almost always need the user-description path.

### "Workflow too large to translate in one pass"

Automatic. Workflows with 50+ nodes are decomposed into subgraphs by
trigger, branch, and chain length. Each subgraph fits in context and
gets its own service file. The interface contracts between subgraphs
are recorded in `migration-state/analysis/<workflow_id>_subgraphs.json`.

### "The benchmark numbers look unfair"

Read `migration-state/benchmarks/report.md` §7 — the toolkit always
generates a fairness caveats section. n8n's CPU is averaged over its
real lifetime serving real users; dev's CPU is measured at near-idle
because it has 1 test user. The 3-layer methodology (idle / per-
interaction / extrapolation) is designed to surface the apples-to-
apples numbers, not hide them.

---

## Available Slash Commands

| Command | What it does |
|---|---|
| `/start` | Begin the migration. Activates Phase 1 (Discovery). |
| `/status` | Show current migration progress from `migration-state/manifest.json`. |
| `/resume` | Continue from the last incomplete checkpoint. |
| `/take-notes` | Reflect on the current session and write improvement notes to `migration-state/notes/`. Run this after a tough debugging session — your future self (or the next person migrating) will thank you. |

---

## Contributing

The toolkit gets better with every migration. There are three high-value
ways to contribute:

1. **Add node mappings.** Found a node type the toolkit didn't know? Add
   it to `.claude/skills/translating-workflows/references/translation-patterns.md`
   with a short example. The format is a markdown table — no codegen
   required.

2. **Submit edge cases as GitHub issues.** Anything weird that bit you
   during translation, testing, or benchmarking — open an issue with the
   reproduction steps. Edge cases that affect more than one user become
   sections in the relevant `SKILL.md`.

3. **Run `/take-notes` after your migration and PR the improvements.**
   The notes file follows a standard template (what worked, what should
   change, edge cases, anticipated questions). Apply them locally and
   PR the diff against the affected `SKILL.md` files.

---

## License

MIT. Use it, fork it, ship it. If it helps you ship a migration, that's
the only metric that matters.

n8n-migration-toolkit

n8n-migration-toolkit Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

n8n-migration-toolkit

n8n-migration-toolkit Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README