ai-video-generation

KO | EN

gitlite — search

#javascript #ai #typescript #llm #android #deep-learning #macos #java #awesome #cli #awesome-list #security #ios #openai

ai-video-generation

(★ 28)

AI Video Storyboard Platform｜基于无限画布的Seedance仿即梦的AI视频自动生成平台｜文生视频｜图生视频｜React-flow|首尾针｜九宫格｜创意对话，AI分镜对话，精细调控，批量视频生成，｜自动生成八要素｜主体，构图，光线，背景，动作，文字叠加，专场细节，音频

ai-video-generation 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# AI Video Storyboard Platform

From Inspiration to Final Cut, AI Assists the Whole Way — Everyone Can Be a Director

🇺🇸 English | <a href="README.zh.md">🇨🇳 中文</a>

Follow our WeChat account for more AI news and updates

📧 Contact: <a href="mailto:gongfan1213@163.com">gongfan1213@163.com</a>

---

## 😫 Do These Pain Points Sound Familiar?

- **You have a vision but can't write it down** — You want to shoot a short video, but writing the storyboard takes an entire day, and bringing your idea to life feels endlessly far away.
- **AI-generated videos look like Frankensteins** — You generate clips with various tools, but the styles and actions don't connect, creating a jarring mess when edited together.
- **Clients/bosses suddenly change requirements** — Scenes get scrapped and rebuilt from scratch, forcing you to rework storyboards, scripts, and reference images, making all-nighters the norm.
- **You don't know camera terminology, so you can't control shots precisely** — You want a "push-in + slow motion + cinematic look," but have no idea how to express that in prompts.
- **Multiple video clips can't maintain character/scene consistency** — The protagonist looks one way in this second, and has a completely different face in the next.

**If any of the above resonates with you, this platform was built for you.**

---

## 💡 In a Nutshell

**AI Video Storyboard Platform** is a one-stop workstation from **Creative Dialogue → AI Storyboard Planning → Fine Control → Batch Video Generation**. Just describe your ideas in natural language, and AI automatically breaks down scenes, writes storyboards, assigns camera techniques, and generates **stylistically consistent** multi-segment videos.

---

## 🎯 Who Is This For?

| Audience | Typical Use Case |
|----------|------------------|
| **Short Video Creators / MCN Directors** | Daily posting pressure demands rapid storyboarding and batch material generation |
| **Indie Animation / Ad Producers** | No full team — one person handles everything from planning to final cut |
| **AI Video Enthusiasts** | Want to create coherent story shorts with AI, but struggle with inconsistent clips |
| **Brand Marketing / Creative Agencies** | Client pitches need visual storyboards and demos, with tight deadlines and frequent changes |
| **Film School Students** | Learn the language of storyboarding and quickly validate creative ideas through AI |

---

## ⚡ Traditional Workflow vs. AI Workbench

| Stage | Traditional Way | Using This Platform |
|-------|----------------|---------------------|
| Ideation | Brainstorm alone, easy to get stuck | **Chat with AI**, one sentence sparks a complete idea |
| Storyboard Script | Handwrite or draw storyboards, hours of work | **AI three-stage generation**, complete storyboard in 5 minutes |
| Shot Description | Write by feel, unstable results | **Eight-element structured description**, every shot is professionally controllable |
| Video Generation | Generate clips one by one, chaotic styles | **Chain generation with first/last frame linking**, visually unified multi-segment videos |
| Requirement Changes | Start over, overtime rework | **Canvas nodes adjust anytime**, one change auto-syncs everywhere |
| Camera Control | Requires professional knowledge | **33+ preset techniques with one click**, beginners can create cinematic shots |

**Result: Storyboard work that used to take 1 day is compressed to 30 minutes.**

---

---

## 🎯 Features

### 1. AI Storyboard Generation (Core Feature)

Create video storyboards through natural language dialogue with AI:

```
User: "I want to make an animated short about a turtle competing in a sports meet, in an anime style"
```

AI automatically completes a three-stage generation:

#### Stage 1: Concept Phase

```
┌─────────────────────────────────────────────────────────────┐
│  Creative Analysis                                           │
│  - Core Concept: Turtle's inspirational story                │
│  - Style Preset: Anime (Japanese animation)                  │
│  - Target Audience: All ages                                 │
│  - Target Duration: 30 seconds                               │
├─────────────────────────────────────────────────────────────┤
│  Scene Breakdown                                             │
│  ┌────────┬────────┬────────┬────────┐                     │
│  │ Scene 1│ Scene 2│ Scene 3│ Scene 4│  (4 scenes total)   │
│  │ Setup  │Conflict│ Climax │Ending  │                     │
│  └────────┴────────┴────────┴────────┘                     │
└─────────────────────────────────────────────────────────────┘
```

#### Stage 2: Planning Phase

```
┌─────────────────────────────────────────────────────────────┐
│  Storyboard Planning                                         │
│                                                              │
│  Total Shots: 6  │  Total Duration: 28s                      │
│                                                              │
│  ┌────┬────┬────┬────┬────┬────┐                           │
│  │Shot1│Shot2│Shot3│Shot4│Shot5│Shot6│                      │
│  │ 5s │ 4s │ 6s │ 5s │ 4s │ 4s │                           │
│  │Push│CU  │MS  │WS  │Slow│Slow│                           │
│  │In  │    │    │    │Mo  │Mo  │                           │
│  └────┴────┴────┴────┴────┴────┘                           │
└─────────────────────────────────────────────────────────────┘
```

#### Stage 3: Storyboard Phase

Each shot contains a complete **Eight-Element** description:

| Element | Example Output |
|---------|---------------|
| **Subject** | Chibi cute turtle holding a small trophy |
| **Composition** | Turtle crossing the finish line centered, eye-level centered composition, deep depth of field |
| **Lighting** | Top diffused soft light, warm and bright atmosphere |
| **Background** | Finish line with colorful small flags, forest animals cheering |
| **Action/Movement** | Turtle slowly crosses the finish line, camera stays static |
| **Text Overlay** | None |
| **Transition Detail** | Fade out, slowly fading as turtle raises trophy in celebration |
| **Audio** | Background music reaches cheerful climax, paired with cheering sounds |

### 2. Visual Canvas Editor

A node-based editor powered by React Flow for real-time storyboard preview and editing:

```
┌──────────────────────────────────────────────────────────────────┐
│                        Canvas Panel                               │
│  ┌────────────────┐    ┌────────────────┐    ┌────────────────┐  │
│  │   ConfigNode   │───▶│   ConfigNode   │───▶│   ConfigNode   │  │
│  │   (Shot 1)     │    │   (Shot 2)     │    │   (Shot 3)     │  │
│  │                │    │                │    │                │  │
│  │  Prompt: ...   │    │  Prompt: ...   │    │  Prompt: ...   │  │
│  │  Techniques:   │    │  Techniques:   │    │  Techniques:   │  │
│  │  - Low Angle   │    │  - Push In     │    │  - Steadicam   │  │
│  │  - Cinematic   │    │  - Slow Mo     │    │  - Orbit       │  │
│  └────────────────┘    └────────────────┘    └────────────────┘  │
│          │                   │                   │              │
│          ▼                   ▼                   ▼              │
│  ┌────────────────┐    ┌────────────────┐    ┌────────────────┐  │
│  │   VideoNode    │    │   VideoNode    │    │   VideoNode    │  │
│  │   [▶ Video 1]  │    │   [▶ Video 2]  │    │   [▶ Video 3]  │  │
│  └────────────────┘    └────────────────┘    └────────────────┘  │
│                                                                  │
│  ═══════════════════════════════════════════════════════════════│
│  [+ Add Node]  [⤴️ Chain Generate All]  [📥 Export JSON] [💾 Save]│
└──────────────────────────────────────────────────────────────────┘
```

### 3. Camera Technique Library

33+ preset camera techniques, apply to shots with one click:

#### Angle — 6 Types

```
Low Angle     Eye Level     High Angle    Dutch Angle   Bird's Eye    Overhead
   ↑            →            ↓            ↗            ⊕            ∠
```

#### View — 6 Types

```
Extreme CU    Close-up      Medium Shot   Wide Shot     Long Shot     Over Shoulder
   ■■          ■■■          ■■■■■        ■■■■■■■      ■■■■■■■■■    ■■■■■
```

#### Movement — 12 Types

| Technique | Description |
|-----------|-------------|
| Push In | Camera moves toward the subject |
| Pull Out | Camera moves away from the subject |
| Pan Left | Camera sweeps to the left |
| Pan Right | Camera sweeps to the right |
| Tilt Up | Camera tilts upward |
| Tilt Down | Camera tilts downward |
| Tracking | Follows the subject's movement |
| Orbiting | Circles around the subject |
| Zoom In | Camera zooms in |
| Zoom Out | Camera zooms out |
| Crane | Camera moves up/down on a crane |
| Steadicam | Handheld stabilizer shot |

#### Style — 9 Types

```
Cinematic     Documentary   Music Video   Slow Motion   Time-lapse
🎬            📽            🎵            ⏱            ⏰
Montage       Handheld      Aerial        Film Noir     Vaporwave
🎞            📷            🚁            🎥            🌃
```

### 4. Three Video Generation Modes

#### T2V (Text-to-Video)

```
Input: "A turtle running in the forest"
   ↓
Output: [Generated video]
```

#### I2V (Image-to-Video)

```
First frame image + Text description
   ↓
Output: [Video generated from the first frame]
```

#### R2V (Reference-to-Video) (Requires Seedance 1.0 Pro)

```
First frame image + Last frame image + Text description
   ↓
Output: [Video transitioning from first to last frame]
```

### 5. Chain Video Generation

Ensures visual consistency across multiple video segments:

```
┌────────────────────────────────────────────────────────────────────┐
│                       Chain Generation                              │
│                                                                     │
│  Shot 1:  [First Frame]──┐                                         │
│         AI Generate      ▼                                          │
│                    [Video1] → [Last Frame1] ─┐                      │
│                                             ▼                       │
│  Shot 2:  [First Frame2] = [Last Frame1] ──┐ │                      │
│              Inherited from prev shot      ▼ │                      │
│                    [Video2] → [Last Frame2] ─┤ │                     │
│                                             ▼ │                     │
│  Shot 3:  [First Frame3] = [Last Frame2] ──┐ │ │                    │
│              Inherited from prev shot      ▼ │ │                    │
│                    [Video3] → [Last Frame3] ─┴─┘                     │
│                                                                     │
│  Result: Three visually coherent videos with natural transitions     │
└────────────────────────────────────────────────────────────────────┘
```

### 6. Nine-Grid Multi-Angle Generation

```
┌─────────┬─────────┬─────────┐
│ Front   │ Side    │ Back    │
│ Close-up│ Close-up│ Close-up│
├─────────┼─────────┼─────────┤
│ Front   │ Side    │ Back    │
│ Medium  │ Medium  │ Medium  │
├─────────┼─────────┼─────────┤
│ Front   │ Side    │ Back    │
│ Wide    │ Wide    │ Wide    │
└─────────┴─────────┴─────────┘
     Select one as the reference image
```

---

## 🏗 Tech Architecture

### Tech Stack

| Layer | Technology | Description |
|-------|------------|-------------|
| **Frontend** | Next.js 16.2.1 + React 19.2.4 | App Router architecture |
| **Language** | TypeScript | Type safety |
| **Visual Editor** | React Flow (@xyflow/react) | Node graph editing |
| **State Management** | Zustand | Lightweight state management |
| **Styling** | Tailwind CSS v4 + shadcn/ui | Modern UI |
| **Animation** | Framer Motion | Interactive animations |
| **AI SDK** | Vercel AI SDK + @ai-sdk/react | AI model integration |
| **AI Models** | Volcano Engine ARK API | Doubao/Seedance/Seedream |
| **Database** | Supabase | PostgreSQL + realtime subscriptions |
| **Storage** | Supabase Storage | Project media storage |

### System Architecture

```
┌─────────────────────────────────────────────────────────────────────────┐
│                           Client (Browser)                              │
│  ┌─────────────────────────────────────────────────────────────────┐   │
│  │                     Next.js Application                          │   │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐  │   │
│  │  │  CanvasPanel │  │  LeftPanel    │  │    RightPanel        │  │   │
│  │  │  - ConfigNode│  │  - Technique │  │    - ChatMessages    │  │   │
│  │  │  - VideoNode │  │    Library   │  │    - StoryboardTabs  │  │   │
│  │  └──────────────┘  └──────────────┘  └──────────────────────┘  │   │
│  │         │                                        │               │   │
│  │         ▼                                        ▼               │   │
│  │  ┌──────────────────────────────────────────────────────────┐    │   │
│  │  │                    Zustand Store                          │    │   │
│  │  │  ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐│    │   │
│  │  │  │ canvas-store│ │ technique-  │ │   app-shell-store  ││    │   │
│  │  │  │  (nodes/    │ │   store     │ │                     ││    │   │
│  │  │  │   edges)    │ │             │ │                     ││    │   │
│  │  │  └─────────────┘ └─────────────┘ └─────────────────────┘│    │   │
│  │  └──────────────────────────────────────────────────────────┘    │   │
│  └─────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                           API Routes (Next.js)                           │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐   │
│  │ /api/chat   │ │ /api/       │ │ /api/       │ │ /api/config     │   │
│  │             │ │ generate    │ │ generate-   │ │                 │   │
│  │ Doubao AI   │ │ image       │ │ image/[id]  │ │ API Key Check   │   │
│  └─────────────┘ └─────────────┘ └─────────────┘ └─────────────────┘   │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐   │
│  │ /api/       │ │ /api/       │ │ /api/       │ │ /api/project-   │   │
│  │ generate    │ │ generate-   │ │ generate-   │ │ media/          │   │
│  │             │ │ frame       │ │ multi-frame│ │                 │   │
│  └─────────────┘ └─────────────┘ └─────────────┘ └─────────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
┌─────────────────────────┐ ┌─────────────────┐ ┌─────────────────────┐
│   Volcano Engine ARK    │ │   Supabase      │ │   Supabase Storage   │
│                         │ │                 │ │                     │
│  ┌─────────────────┐   │ │  PostgreSQL     │ │  project-media       │
│  │  Doubao         │   │ │  - profiles     │ │  bucket              │
│  │  - Chat Model   │   │ │  - projects     │ │                      │
│  │  - Storyboard   │   │ │  - assets       │ │                      │
│  └─────────────────┘   │ │  - generation_  │ │                      │
│  ┌─────────────────┐   │ │    tasks       │ │                      │
│  │ Seedance (Video)│   │ │                 │ │                      │
│  │  - T2V/I2V/R2V  │   │ └─────────────────┘ │                      │
│  └─────────────────┘   │                       └─────────────────────┘
│  ┌─────────────────┐   │
│  │ Seedream (Image)│   │
│  │  - Character/   │   │
│  │    Nine-Grid    │   │
│  └─────────────────┘   │
└─────────────────────────┘
```

### Data Model

```
┌─────────────────────────────────────────────────────────────────────────┐
│                              Supabase Schema                             │
│  ┌────────────────┐  ┌────────────────┐  ┌────────────────────────┐    │
│  │   profiles     │  │   projects     │  │   generation_tasks     │    │
│  ├────────────────┤  ├────────────────┤  ├────────────────────────┤    │
│  │ id (uuid)      │  │ id (uuid)      │  │ id (uuid)              │    │
│  │ user_id (fk)   │  │ user_id (fk)   │  │ project_id (fk)       │    │
│  │ nickname       │  │ name           │  │ task_type              │    │
│  │ avatar_url     │  │ state (json)   │  │ task_id                │    │
│  │ created_at     │  │ created_at     │  │ status                 │    │
│  └────────────────┘  │ updated_at     │  │ created_at            │    │
│                      └────────────────┘  └────────────────────────┘    │
│  ┌────────────────┐                                                       │
│  │    assets      │                                                       │
│  ├────────────────┤                                                       │
│  │ id (uuid)      │                                                       │
│  │ project_id(fk) │                                                       │
│  │ file_url       │                                                       │
│  │ file_type      │                                                       │
│  │ created_at     │                                                       │
│  └────────────────┘                                                       │
└─────────────────────────────────────────────────────────────────────────┘
```

---

## 🚀 Quick Start

### Requirements

- Node.js 18+
- npm / yarn / pnpm
- Supabase account
- Volcano Engine ARK API Key

### Installation

```bash
# 1. Clone the project
git clone https://github.com/your-username/ai-video.git
cd ai-video

# 2. Install dependencies
npm install

# 3. Configure environment variables
cp .env.example .env.local
```

### Environment Variables

```env
# Supabase Config
NEXT_PUBLIC_SUPABASE_URL=https://your-project-ref.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=your-supabase-anon-key

# Volcano Engine ARK API Config
ARK_API_KEY=your-volcengine-api-key
ARK_ENDPOINT_ID=your-doubao-endpoint-id

# Optional: Model overrides (usually not needed)
SEEDANCE_MODEL=doubao-seedance-1-5-pro-251215
SEEDANCE_LITE_MODEL=doubao-seedance-1-0-pro-250528
IMAGE_MODEL=doubao-seedream-5-0-260128
```

### Start Development Server

```bash
npm run dev
```

Visit http://localhost:3000 to use.

---

## 🚢 Deployment Guide

### Deploy to Vercel (Recommended)

This project is optimized for Vercel and can be deployed with one click:

[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https://github.com/your-username/ai-video)

#### Manual Deployment

```bash
# 1. Install Vercel CLI
npm install -g vercel

# 2. Login to Vercel
vercel login

# 3. Deploy
vercel
```

#### Vercel Environment Variables

Add the following environment variables in your Vercel project settings:

| Variable | Description | Required |
|----------|-------------|----------|
| `NEXT_PUBLIC_SUPABASE_URL` | Supabase project URL | ✅ |
| `NEXT_PUBLIC_SUPABASE_ANON_KEY` | Supabase Anon Key | ✅ |
| `ARK_API_KEY` | Volcano Engine ARK API Key | ✅ |
| `ARK_ENDPOINT_ID` | Doubao model Endpoint ID | ✅ |
| `SEEDANCE_MODEL` | Seedance model ID (optional) | ❌ |
| `SEEDANCE_LITE_MODEL` | Seedance Lite model ID (optional) | ❌ |
| `IMAGE_MODEL` | Seedream model ID (optional) | ❌ |

### Supabase Deployment Config

#### 1. Create Supabase Project

1. Visit [supabase.com](https://supabase.com) to create a new project
2. Wait for project creation to complete, then get the project URL and Anon Key

#### 2. Execute Database Schema

Run [`docs/supabase/schema.sql`](docs/supabase/schema.sql) in the Supabase SQL Editor:

```sql
-- Or import the SQL file directly in the Supabase Dashboard
```

#### 3. Configure Row Level Security (RLS)

All tables have RLS policies configured to ensure users can only access their own data:

```sql
-- Enable RLS (usually enabled by default)
ALTER TABLE profiles ENABLE ROW LEVEL SECURITY;
ALTER TABLE projects ENABLE ROW LEVEL SECURITY;
ALTER TABLE assets ENABLE ROW LEVEL SECURITY;
ALTER TABLE generation_tasks ENABLE ROW LEVEL SECURITY;
```

#### 4. Configure Storage Bucket

1. Create a `project-media` storage bucket in the Supabase Dashboard
2. Set it to **Public** bucket
3. Configure storage policies to allow uploads and access

#### 5. Configure Auth Providers (Optional)

In Supabase Dashboard → Authentication → Providers, configure:

- Email/Password login (enabled by default)
- Google OAuth (optional)
- GitHub OAuth (optional)

### Volcano Engine ARK API Configuration

#### 1. Get API Key

1. Visit [Volcano Engine ARK Console](https://www.volcengine.com/product/ark)
2. Create an API Key
3. Get the Endpoint ID (Doubao model)

#### 2. Model Reference

| Model | Purpose | Description |
|-------|---------|-------------|
| `doubao-seedance-1-5-pro-251215` | Video generation (T2V/I2V) | Seedance 1.5 Pro |
| `doubao-seedance-1-0-pro-250528` | Video generation (R2V first/last frame) | Seedance 1.0 Pro |
| `doubao-seedream-5-0-260128` | Image generation | Seedream 5.0 |

### Other Deployment Platforms

#### Docker Deployment

```dockerfile
# Dockerfile
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:18-alpine AS runner
WORKDIR /app
ENV NODE_ENV production
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./
COPY --from=builder /app/public ./
EXPOSE 3000
CMD ["node", "server.js"]
```

#### Traditional VPS Deployment

```bash
# 1. Install Node.js 18+
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
sudo apt-get install -y nodejs

# 2. Install PM2
npm install -g pm2

# 3. Clone and build
git clone https://github.com/your-username/ai-video.git
cd ai-video
npm install
npm run build

# 4. Start with PM2
pm2 start npm --name "ai-video" -- start
```

#### PM2 Process Management

```bash
# View status
pm2 status

# View logs
pm2 logs ai-video

# Restart
pm2 restart ai-video

# Stop
pm2 stop ai-video
```

### Production Checklist

- [ ] Supabase project created and configured
- [ ] Database schema executed
- [ ] RLS policies enabled
- [ ] `project-media` storage bucket created and set to Public
- [ ] Volcano Engine ARK API Key obtained
- [ ] All environment variables configured on deployment platform
- [ ] Custom domain configured (optional)
- [ ] HTTPS enabled (enabled by default on Vercel)

---

## 📁 Project Structure

```
ai-video/
├── src/
│   ├── app/                          # Next.js App Router
│   │   ├── (auth)/                    # Auth pages
│   │   │   ├── login/
│   │   │   ├── register/
│   │   │   ├── forgot-password/
│   │   │   └── update-password/
│   │   ├── (landing)/                 # Landing page
│   │   │   └── page.tsx
│   │   ├── api/                       # API routes
│   │   │   ├── chat/                  # AI chat
│   │   │   ├── config/                # Config check
│   │   │   ├── enhance-prompt/        # Prompt enhancement
│   │   │   ├── generate/              # Video generation
│   │   │   ├── generate-image/        # Image generation
│   │   │   ├── generate-frame/        # First/last frame generation
│   │   │   ├── generate-multi-frame/ # Nine-grid generation
│   │   │   └── project-media/         # Media storage
│   │   ├── app/                      # Main workspace
│   │   │   └── app-client.tsx
│   │   ├── auth/                     # Auth callbacks
│   │   ├── profile/                  # User profile
│   │   └── projects/                 # Project list
│   │
│   ├── components/                   # React components
│   │   ├── ui/                      # Base UI components
│   │   │   ├── button.tsx
│   │   │   ├── dialog.tsx
│   │   │   ├── badge.tsx
│   │   │   └── ...
│   │   ├── panels/                  # Panel components
│   │   │   ├── canvas-panel.tsx     # Canvas panel
│   │   │   ├── left-panel.tsx       # Left technique library
│   │   │   ├── right-panel.tsx      # Right AI chat
│   │   │   └── workspace-actions-rail.tsx
│   │   ├── nodes/                   # React Flow nodes
│   │   │   ├── config-node.tsx       # Config node
│   │   │   └── video-node.tsx        # Video node
│   │   ├── landing/                 # Landing components
│   │   ├── providers/               # Context Providers
│   │   └── auth/                    # Auth components
│   │
│   ├── hooks/                       # Custom Hooks
│   │   ├── use-video-generation.ts  # Video generation
│   │   ├── use-image-generation.ts  # Image generation
│   │   ├── use-chain-generation.ts   # Chain generation
│   │   └── use-batch-video-generation.ts
│   │
│   ├── stores/                      # Zustand state management
│   │   ├── canvas-store.ts         # Canvas state
│   │   ├── technique-store.ts      # Technique library state
│   │   ├── user-store.ts           # User state
│   │   └── app-shell-store.ts      # UI state
│   │
│   ├── lib/                        # Utilities
│   │   ├── seedance.ts             # Seedance API
│   │   ├── volcengine.ts           # Volcengine config
│   │   ├── constants.ts            # Constants
│   │   ├── supabase/               # Supabase clients
│   │   ├── projects.ts             # Project management
│   │   └── utils.ts                # Utility functions
│   │
│   └── types/                      # TypeScript types
│       ├── canvas.ts               # Canvas types
│       ├── video.ts                # Video types
│       ├── chat.ts                 # Chat types
│       ├── concept.ts              # Concept phase types
│       ├── planning.ts             # Planning phase types
│       ├── technique.ts            # Technique types
│       └── index.ts                # Exports
│
├── public/                         # Static assets
├── docs/                           # Documentation
│   └── supabase/                   # Database schema
├── .env.example                    # Environment variables example
├── package.json
├── tsconfig.json
└── README.md
```

---

## 🔌 API Reference

### Video Generation

#### POST `/api/generate`
Create a video generation task

```typescript
// Request
{
  prompt: string;              // Prompt
  negativePrompt?: string;      // Negative prompt
  techniques?: Technique[];    // Camera techniques
  ratio: VideoRatio;           // Aspect ratio
  duration: VideoDuration;     // Duration
  firstFrameUrl?: string;      // First frame image
  lastFrameUrl?: string;       // Last frame image
  // ... eight-element fields
}

// Response
{ taskId: string; }
```

#### GET `/api/generate/[taskId]`
Query video task status

```typescript
// Response
{
  status: 'queued' | 'running' | 'succeeded' | 'failed';
  videoUrl?: string;
  lastFrameUrl?: string;
}
```

### Image Generation

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/generate-image` | POST | Create image generation task |
| `/api/generate-image/[taskId]` | GET | Query image task status |
| `/api/generate-frame` | POST | Generate first/last frame image |
| `/api/generate-multi-frame` | POST | Generate nine-grid images |

### AI Chat

#### POST `/api/chat`
AI storyboard dialogue, supports three tools:

- `generateConcept` - Generate concept
- `generatePlanning` - Generate planning
- `generateStoryboard` - Generate storyboard

---

## 🎨 The Eight-Element System

The Eight-Element system is the core output specification for AI storyboarding, ensuring every shot has a complete, professional visual description:

| Element | English | Keyword | Description | Example |
|---------|---------|---------|-------------|---------|
| **subject** | Subject | Subject | Character/object features | "Chibi cute turtle holding a small trophy" |
| **composition** | Composition | Composition | Subject position, perspective, depth of field | "Eye-level centered composition, deep depth of field" |
| **lighting** | Lighting | Lighting | Light source direction, quality | "Top diffused soft light, warm and bright" |
| **background** | Background | Background | Environment, space, color | "Finish line with colorful small flags" |
| **actionMovement** | Action/Movement | Action/Movement | Subject action, camera movement | "Turtle slowly crosses the finish line" |
| **textOverlay** | Text Overlay | Text Overlay | Subtitle content | "None" or "At 1s: Go for it!" |
| **transitionDetail** | Transition | Transition | Type, timing, effect | "Fade out, slowly fading" |
| **audio** | Audio | Audio | Music, sound effects, narration | "Cheerful background music + cheering" |

### Eight-Element Prompt Template

```
[Visual Elements]
Subject: {subject}
Composition: {composition}
Lighting: {lighting}
Background: {background}
Action: {actionMovement}
Transition Detail: {transitionDetail}
Audio: {audio}

[Consistency Requirements]
{consistency_notes}
```

---

## 📊 Style Templates

5 preset styles, each with complete visual specifications:

### 1. Anime
```
Color: High saturation, dreamy tones
Composition: Dynamic compositions, extreme close-ups
Visual: Big eyes, sharp outlines, soft light, crisp rendering
```

### 2. Film
```
Color: Low saturation, film grain texture
Composition: Deep depth of field, dramatic lighting
Visual: 4K UHD, cinematic color grading
```

### 3. Cyberpunk
```
Color: Neon colors, high contrast
Composition: Low angles, dive shots
Visual: Blue lighting, smoke, city nightscapes
```

### 4. Realistic
```
Color: Natural tones, medium saturation
Composition: Realistic, narrative-driven
Visual: 8K UHD, skin texture detail
```

### 5. Google
```
Color: Bright, clean
Composition: Minimal, flat design
Visual: Minimalist style, CMYK color scheme
```

---

## 🔄 Workflow

### Complete Creation Process

```
┌─────────────────────────────────────────────────────────────────────────┐
│                         Complete Workflow                                │
│                                                                         │
│  ┌─────────┐     ┌─────────┐     ┌─────────┐     ┌─────────┐         │
│  │Ideation │────▶│Storyboard│────▶│Fine-    │────▶│Generate │         │
│  │ Input   │     │Generate │     │tune     │     │ Video   │         │
│  └─────────┘     └─────────┘     └─────────┘     └─────────┘         │
│       │              │              │              │                   │
│       ▼              ▼              ▼              ▼                   │
│  ┌─────────┐    ┌─────────┐    ┌─────────┐    ┌─────────┐             │
│  │Natural  │    │AI Three-│    │Apply    │    │Chain    │             │
│  │Language │    │Stage    │    │Technique│    │Generate │             │
│  │Describe │    │Process  │    │         │    │         │             │
│  │Idea     │    │         │    │         │    │         │             │
│  └─────────┘    └─────────┘    └─────────┘    └─────────┘             │
│                           │              │                             │
│                           ▼              ▼                             │
│                    ┌─────────────┐  ┌─────────────┐                   │
│                    │  Eight      │  │  First/Last │                    │
│                    │  Elements   │  │  Frame Link │                    │
│                    │  Detailed   │  │             │                    │
│                    └─────────────┘  └─────────────┘                   │
└─────────────────────────────────────────────────────────────────────────┘
```

### Three-Stage AI Process

```
Step 1: Concept Phase
──────────────────────────────────────────────────────────
User inputs core idea
        │
        ▼
┌───────────────────┐
│   AI Analyzes     │
│   - Core Theme    │
│   - Style Type    │
│   - Target Audience│
└────────┬──────────┘
         │
         ▼
┌───────────────────┐
│   Scene Breakdown │
│   - Setup/Conflict│
│     /Climax/Ending│
└────────┬──────────┘
         │
         ▼
    [Concept Result JSON]
         │
         ▼
Step 2: Planning Phase
──────────────────────────────────────────────────────────
         │
         ▼
┌───────────────────┐
│   Content Analysis│
│   - Narrative     │
│     Structure     │
│   - Rhythm Control│
└────────┬──────────┘
         │
         ▼
┌───────────────────┐
│   Shot Planning   │
│   - Shot Count    │
│   - Duration per  │
│     Shot          │
│   - Transition    │
│     Method        │
└────────┬──────────┘
         │
         ▼
    [Planning Result JSON]
         │
         ▼
Step 3: Storyboard Phase
──────────────────────────────────────────────────────────
         │
         ▼
┌───────────────────┐
│   Detailed        │
│   Storyboard      │
│   - Description   │
│     per Shot      │
│   - Eight Elements│
│     Filled        │
│   - First/Last    │
│     Frame Prompts │
└────────┬──────────┘
         │
         ▼
┌───────────────────┐
│   Alternative     │
│   Prompts         │
│   - 3-5 Variant   │
│     Versions      │
└────────┬──────────┘
         │
         ▼
    [Storyboard Result JSON]
         │
         ▼
──────────────────────────────────────────────────────────
    Add storyboard to canvas → Generate video shot by shot
```

---

## 🎮 User Guide

### Basic Operations

1. **Create New Project**: Click "Create Project" on the homepage
2. **AI Storyboard**: Enter your creative description in the right panel, chat with AI to generate storyboard
3. **Add to Canvas**: Click "Add to Canvas" on the storyboard card
4. **Adjust Techniques**: Select a node, click technique tags in the left library to apply
5. **Generate Video**: Click the "Generate" button on the node
6. **Chain Generation**: Click "Chain Generate All" at the bottom

### Node Operations

| Operation | Method |
|-----------|--------|
| Add Node | Double-click empty canvas area |
| Delete Node | Select node → Delete/Backspace |
| Select Node | Click on node |
| Edit Prompt | Double-click node or click edit icon |
| Apply Technique | Click technique tags in left library |

### Shortcuts

| Shortcut | Function |
|----------|----------|
| `Delete` / `Backspace` | Delete selected node |
| `Double-click canvas` | Add empty config node |
| `Ctrl/Cmd + S` | Save project |

---

## 🧩 Component Reference

### ConfigNode

Each shot on the canvas corresponds to a config node, containing:

- **Prompt Editor**: Editable prompt text box
- **Technique Tags**: Applied camera techniques, clickable to remove
- **Eight-Element Editor**: Collapsible detailed visual description
- **Video Specs**: ratio, duration, resolution selectors
- **First/Last Frame Panel**: Reference frame mode toggle and image management
- **Generate Button**: Trigger video generation

### VideoNode

One-to-one mapping with ConfigNode, displaying the generated video:

- **Video Player**: HTML5 video with control bar
- **Last Frame Preview**: Thumbnail of the video's final frame
- **Status Display**: idle / generating / done / error
- **Action Bar**: Regenerate, download video
- **Version History**: Support switching to historical versions

### LeftPanel (Left Technique Library)

- **Category Accordions**: Angle / View / Movement / Style
- **Technique Chips**: Click to apply to selected node
- **Hover Tooltip**: Display technique description and keywords

### RightPanel (Right AI Chat)

- **GlobalConfigBar**: Global model and specs settings
- **ChatMessages**: Chat history
- **StoryboardTabs**: Storyboard result display
  - **OverviewTab**: Overview table
  - **DetailsTab**: Detailed cards

---

## 📝 Database Schema

See [`docs/supabase/schema.sql`](docs/supabase/schema.sql)

### Main Tables

| Table | Description |
|-------|-------------|
| `profiles` | User profiles (nickname, avatar) |
| `projects` | Projects (contains full canvas state JSON) |
| `project_snapshots` | Project snapshots (version history) |
| `generation_tasks` | Generation task records |
| `assets` | Generated media assets |

---

## 🔐 Security Features

- **Row Level Security (RLS)**: All tables have row-level security policies enabled
- **User Data Isolation**: Users can only access their own data
- **API Key Hidden**: Sensitive configs exist only on the server side
- **HTTPS Enforced**: All external resource URLs must be HTTPS

---

## 🤝 Contributing

Issues and Pull Requests are welcome!

1. Fork this repository
2. Create a feature branch (`git checkout -b feature/AmazingFeature`)
3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the branch (`git push origin feature/AmazingFeature`)
5. Create a Pull Request

---

## 📄 License

This project is licensed under the MIT License — see the [LICENSE](LICENSE) file for details.

---

## 🙏 Acknowledgments

- [Next.js](https://nextjs.org/) - React framework
- [React Flow](https://reactflow.dev/) - Visual editor
- [Supabase](https://supabase.com/) - Database and auth
- [Volcano Engine](https://www.volcengine.com/) - AI model services
- [shadcn/ui](https://ui.shadcn.com/) - UI component library
- [Tailwind CSS](https://tailwindcss.com/) - Styling framework

---

If this project helps you, please give it a ⭐️

ai-video-generation

ai-video-generation 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

ai-video-generation

ai-video-generation 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README