--- name: ai-animation description: Use when creating AI-powered animations and videos — nano-banana keyframes → Kling/Luma/Veo video generation via fal.ai → web-ready output. Full pipeline from concept to scroll-driven animation or video embed. Triggers on "AI animation", "animate image", "AI video", "Kling", "image to video", "анимируй", "сделай видео из картинки". arguments: - name: pipeline description: "Pipeline: keyframe-to-video, image-to-video, text-to-video, video-to-web (default: keyframe-to-video)" required: false - name: model description: "Model: kling, luma, veo, ltx, pixverse (default: auto-select by use case)" required: false - name: style description: "Visual style or reference for generation" required: false --- # AI Animation Pipeline > nano-banana (keyframes) → fal.ai (video gen) → FFmpeg (web-ready) → scroll/embed > Prices verified 2026-03-13 [E1-fal.ai API docs] ## Pipeline Overview ``` Concept / Prompt │ ├─→ [1. Keyframes] nano-banana → reference images (poses, angles, scenes) │ ├─→ [2. Animate] fal.ai → Kling / Luma / Veo → MP4 video │ ├─→ [3. Process] FFmpeg → frame sequence / optimized video / GIF │ └─→ [4. Integrate] scroll-animation / video embed / motion-design ``` --- ## Model Selection Matrix [E1-fal.ai docs] | Use Case | Model | Endpoint | Cost/sec | Duration | Why | |----------|-------|----------|----------|----------|-----| | **Product reveal** | Kling v3 Pro | `fal-ai/kling-video/v3/pro/image-to-video` | $0.112 (no audio) / $0.168 (audio) | 3-15s | Best motion, multi-prompt, elements | | **Hero loop** | Luma Ray 2 | `fal-ai/luma-dream-machine/ray-2/image-to-video` | $0.10/sec @540p | 5s, 9s | `loop: true`, start+end keyframes | | **Cinematic** | Veo 3 | `fal-ai/veo3` | $0.20 (no audio) / $0.40 (audio) | 4-8s | Best quality + audio, 1080p | | **Cinematic (fast)** | Veo 3 Fast | `fal-ai/veo3/fast/image-to-video` | $0.10 (no audio) / $0.15 (audio) | 4-8s | 50% cheaper I2V | | **Bulk/cheap** | LTX 2.0 | `fal-ai/ltx-2/text-to-video/fast` | $0.04 @1080p | 6-20s | Cheapest, up to 4K, extend/retake | | **Stylized** | PixVerse v4.5 | `fal-ai/pixverse/v4.5/image-to-video` | $0.03-0.08/sec | 5-8s | anime/3d/clay/comic/cyberpunk + camera presets | ### Quick Decision - Loop for hero section? → **Luma Ray 2** (loop: true) - Animate a product/design comp? → **Kling v3 Pro** - Maximum quality, budget OK? → **Veo 3** - Need 10+ videos cheap? → **LTX 2.0** - Specific art style? → **PixVerse v4.5** --- ## Step 1: Generate Keyframes (nano-banana) Generate reference images that will be animated into video. ### Best Practices for Animation Keyframes ```bash # Product floating in space — good for Kling rotation nano-banana "premium wireless headphones floating on dark background, studio lighting, centered composition, clean edges" -s 2K -a 16:9 -o keyframe-product # Scene for cinematic animation — good for Veo/Kling nano-banana "cyberpunk street at golden hour, rain puddles, neon reflections, cinematic composition" -s 2K -a 16:9 -o keyframe-scene # Character pose — good for Kling motion nano-banana "3D character mascot robot waving, isometric view, white background, Pixar style" -s 1K -a 1:1 -o keyframe-character # Hero background — good for Luma loop nano-banana "abstract flowing liquid metal, iridescent purple and gold, macro photography" -s 2K -a 16:9 -o keyframe-hero # Transparent asset for animation nano-banana "floating crystal sphere with inner glow" -t -s 1K -o keyframe-crystal ``` ### nano-banana → fal.ai Upload Workflow nano-banana outputs local PNG files. To use as input for fal.ai video models: ```python import fal_client # Option A: Upload to fal CDN (recommended for large files) image_url = fal_client.upload_file("keyframe-product.png") # → https://fal.media/files/... # Option B: Encode as data URI (inline, faster for small files, no CDN hop) image_data = fal_client.encode_file("keyframe-product.png") # Then use in any video model result = fal_client.run("fal-ai/kling-video/v3/pro/image-to-video", arguments={ "start_image_url": image_url, # or image_data "prompt": "the headphones slowly rotate", "duration": "5" }) ``` **Optimal nano-banana settings per video model:** | Video Model | nano-banana Size | Aspect | Notes | |------------|-----------------|--------|-------| | Kling v3 Pro | `-s 2K` | `-a 16:9` or `-a 9:16` or `-a 1:1` | Match `aspect_ratio` param | | Luma Ray 2 | `-s 2K` | `-a 16:9` or any of 6 ratios | 540p output, 2K input still better | | Veo 3 | `-s 2K` | `-a 16:9` or `-a 9:16` only | **Must be 720p+ and <8MB** | | LTX 2.0 | `-s 2K` | `-a 16:9` | Flexible, handles most inputs | | PixVerse v4.5 | `-s 2K` | match target | Resolution matches input | ### Keyframe Prompting Rules 1. **Clean edges** — AI video models struggle with busy/cluttered compositions 2. **Centered subject** — off-center subjects may drift during animation 3. **Consistent lighting** — dramatic lighting gives video models more to work with 4. **Simple backgrounds** — solid/gradient backgrounds → cleaner motion 5. **Describe potential motion** — "floating", "flowing", "spinning" primes the composition 6. **16:9 for video** — match the target video aspect ratio in the keyframe 7. **High resolution** — use `-s 2K` minimum, video models downsample better than upscale --- ## Step 2: Animate via fal.ai ### Prerequisites ```bash pip install fal-client export FAL_KEY="your-key" ``` ### fal_client SDK Patterns (Python) ```python import fal_client # SYNC — blocks until done (simplest, for single generations) result = fal_client.run("fal-ai/veo3", arguments={...}) video_url = result["video"]["url"] # ASYNC — non-blocking import asyncio async def generate(): result = await fal_client.run_async("fal-ai/veo3", arguments={...}) return result["video"]["url"] # QUEUE with progress tracking — best for long jobs async def generate_with_progress(): handler = await fal_client.submit_async("fal-ai/veo3", arguments={...}) logs_index = 0 async for event in handler.iter_events(with_logs=True): if isinstance(event, fal_client.Queued): print(f"Queue position: {event.position}") elif isinstance(event, (fal_client.InProgress, fal_client.Completed)): for log in event.logs[logs_index:]: print(log["message"]) logs_index = len(event.logs) result = await handler.get() return result["video"]["url"] # FILE UPLOAD — local file → fal CDN URL url = fal_client.upload_file("local-image.png") # returns https://fal.media/files/... data_uri = fal_client.encode_file("local-image.png") # inline data URI, no upload ``` ### REST API Pattern (curl) ```bash # Sync (blocks until done, timeout risk for long generations) curl -X POST "https://fal.run/{endpoint}" \ -H "Authorization: Key $FAL_KEY" \ -H "Content-Type: application/json" \ -d '{"prompt": "...", ...}' # Queue submit (non-blocking, recommended) REQUEST_ID=$(curl -s -X POST "https://queue.fal.run/{endpoint}" \ -H "Authorization: Key $FAL_KEY" \ -H "Content-Type: application/json" \ -d '{"prompt": "...", ...}' | jq -r '.request_id') # Check status curl -s "https://queue.fal.run/{endpoint}/requests/$REQUEST_ID/status" \ -H "Authorization: Key $FAL_KEY" # Get result curl -s "https://queue.fal.run/{endpoint}/requests/$REQUEST_ID" \ -H "Authorization: Key $FAL_KEY" # Webhook (fire and forget) curl -X POST "https://queue.fal.run/{endpoint}" \ -H "Authorization: Key $FAL_KEY" \ -H "Content-Type: application/json" \ -H "X-Fal-Webhook-Url: https://your-server.com/webhook" \ -d '{"prompt": "..."}' ``` ### Kling v3 Pro — Image to Video [E1-verified] Best for: product reveals, design comp animation, character motion, multi-shot sequences. ```python import fal_client result = fal_client.run( "fal-ai/kling-video/v3/pro/image-to-video", arguments={ "prompt": "the headphones slowly rotate 360 degrees, studio lighting, smooth motion", "start_image_url": "https://...keyframe-product.png", "duration": "5", # "3" to "15" seconds (string) "aspect_ratio": "16:9", # 16:9, 9:16, 1:1 "generate_audio": False, # True adds $0.056/sec } ) video_url = result["video"]["url"] print(f"Video: {video_url}") ``` ```javascript // Node.js import { fal } from "@fal-ai/client"; const result = await fal.subscribe("fal-ai/kling-video/v3/pro/image-to-video", { input: { prompt: "the headphones slowly rotate 360 degrees", start_image_url: "https://...keyframe.png", duration: "5", aspect_ratio: "16:9", generate_audio: false, }, logs: true, onQueueUpdate: (update) => { if (update.status === "IN_PROGRESS") { update.logs.map((log) => log.message).forEach(console.log); } }, }); ``` **Parameters:** | Param | Values | Default | Notes | |-------|--------|---------|-------| | `prompt` | string | — | Describe MOTION, not scene. Mutually exclusive with `multi_prompt` | | `start_image_url` | URL | required | Source keyframe (**not** `image_url`) | | `end_image_url` | URL | — | End frame for interpolation | | `duration` | `"3"`-`"15"` | `"5"` | String, per-second increments | | `aspect_ratio` | `16:9`, `9:16`, `1:1` | `16:9` | | | `generate_audio` | bool | `true` | Native audio (Chinese/English) | | `negative_prompt` | string | `"blur, distort, and low quality"` | | | `cfg_scale` | float | `0.5` | Prompt adherence strength | | `multi_prompt` | array | — | Multi-shot: `[{"prompt": "...", "duration": "5"}, ...]` | | `elements` | array | — | Character/object consistency via reference images | **Cost [E1]:** - No audio: $0.112/sec → 5s = $0.56, 10s = $1.12 - With audio: $0.168/sec → 5s = $0.84, 10s = $1.68 - With voice control: $0.196/sec **Multi-shot example:** ```python result = fal_client.run("fal-ai/kling-video/v3/pro/image-to-video", arguments={ "start_image_url": "https://...keyframe.png", "multi_prompt": [ {"prompt": "camera orbits around the product slowly", "duration": "5"}, {"prompt": "zoom in to the detail panel", "duration": "5"} ], "generate_audio": False, }) ``` **curl:** ```bash curl -X POST "https://fal.run/fal-ai/kling-video/v3/pro/image-to-video" \ -H "Authorization: Key $FAL_KEY" \ -H "Content-Type: application/json" \ -d '{"start_image_url": "https://...keyframe.png", "prompt": "rotate slowly", "duration": "5", "generate_audio": false}' ``` **Prompting for motion:** - Describe WHAT MOVES: "the camera slowly orbits around the product" - NOT the scene: ~~"beautiful product on dark background"~~ - Be specific: "rotate 90 degrees clockwise" > "rotate" - Keep it simple: one primary motion per clip ### Luma Ray 2 — Looping Hero Videos [E1-verified] Best for: hero section backgrounds, ambient loops, living textures, start→end interpolation. ```python result = fal_client.run( "fal-ai/luma-dream-machine/ray-2/image-to-video", arguments={ "prompt": "gentle flowing motion, seamless loop", "image_url": "https://...keyframe-hero.png", "loop": True, # seamless loop! "resolution": "540p", # 540p, 720p (2x), 1080p (4x) "duration": "5s", # "5s" or "9s" "aspect_ratio": "16:9", } ) video_url = result["video"]["url"] ``` **Key features:** - `loop: True` → seamless loops for `