Video Short Creator

Two-phase workflow for creating AI-narrated short videos from source clips and a script, with human review at every critical step.

Prerequisites

Phase 1: Material Preparation & Review

This phase generates the narration audio and exports a review sheet. Stop and wait for user approval before proceeding.

Step 1.1: Analyze Materials

1. Ask the user for:
   - Source video clips (paths or a folder)
   - Script / narration text (or generate one from provided reference material)
   - Target resolution (default: 1920x1080 landscape)
   - Voice preference (default: zh-CN-YunxiNeural, young male Mandarin)
   - Publishing platform (optional, affects aspect ratio recommendations)
2. Scan all source clips with ffprobe to catalog:
   - Resolution, duration, file size
   - Generate a clip inventory table
3. If no script is provided but reference material (paper URL, markdown, etc.) is given:
   - Research the content and draft a narration script
   - Structure as 3-5 segments with natural flow

Step 1.2: Build SCRIPT Data Structure

Create a Python list of segment dicts that maps each narration segment to source clips:

SCRIPT = [
    {
        "id": "seg1",          # unique segment ID
        "text": "...",         # narration text for this segment
        "videos": [            # source clips to use
            {"file": "clip1.mp4", "start": 0, "max_dur": 14},
            {"file": "clip2.mp4", "start": 5, "max_dur": 30},
        ],
    },
    # ... more segments
]

Key design decisions:
- Each segment's total available video duration should exceed narration duration by ~0.5s
- Use multiple clips per segment for visual variety
- max_dur prevents over-long clips from a single source

Step 1.3: Generate TTS & Export Review Sheet

Run scripts/step1_generate_review.py with the SCRIPT config:

python scripts/step1_generate_review.py

This script:
1. Generates TTS audio for each segment using edge-tts
2. Extracts clean subtitle timing from SentenceBoundary events (NOT WordBoundary)
3. Exports subtitle_review.md with all subtitle entries + timing info

Step 1.4: Present Review Materials to User

Display the following for user review:

1. Clip Inventory - All source clips with metadata
2. Script Review - Full narration text organized by segment
3. Subtitle Review Sheet (subtitle_review.md) - All subtitle entries with:
   - Sequence number, segment ID, start/end time, duration, text content
4. Segment-to-Clip Mapping - Which clips are used for each narration segment

Ask the user to review and confirm or provide modifications:
- Text corrections in narration or subtitles
- Timing adjustments
- Clip substitutions or reorderings
- Adding/removing subtitle entries

> CRITICAL: Do NOT proceed to Phase 2 until the user explicitly approves.

Phase 2: Video Assembly

Only start after user approval of Phase 1 materials.

Step 2.1: Apply User Edits (if any)

If the user provided subtitle edits:
1. Parse their feedback and update the subtitle entries accordingly
2. If they created a subtitle_edited.txt, load it (format: start_sec|end_sec|text per line)
3. Re-verify that all source clip paths exist

Step 2.2: Execute Video Assembly

Run scripts/step2_edit_video.py:

python scripts/step2_edit_video.py

The assembly pipeline:
1. Clip & Scale - Extract clips at specified start/duration, scale to target resolution (1920x1080)
2. Concatenate - Join multiple clips per segment using concat demuxer
3. Burn Subtitles - Overlay subtitles using .srt file + FFmpeg subtitles filter
4. xfade Transitions - Chain segments with alternating fade/fadeblack transitions (0.8s)
5. Build Audio - Concatenate narration segments with short silences between them
6. Final Merge - Combine video + audio into final output

Step 2.3: Output

• Final video at output/{PROJECT_NAME}_FINAL.mp4
• Resolution: 1920x1080, 30 FPS
• Subtitle style: SimHei font, size 14, white text, thin outline, bottom-aligned (movie-style)
• Audio: AAC 128kbps
• Video: H.264 CRF 20

Step 2.4: Present Final Video

Use open_result_view to present the final video. Ask user for feedback on:
- Subtitle readability and positioning
- Transition smoothness
- Audio-video sync
- Overall pacing

Configuration

All configuration lives in the SCRIPT data structure at the top of each script. Key constants:

Known Pitfalls (Windows)

• Never use drawtext filter — shell escaping for \n and special characters is unreliable on Windows. Always use .srt file + subtitles filter instead.
• SRT file paths need escaping for FFmpeg subtitles filter: replace \ with / and : with \:.
• edge-tts SentenceBoundary gives clean text; WordBoundary may contain artifacts like stray n characters.
• Always re-fetch subtitle timing even when audio files are cached — edge-tts does not store boundary events.
• xfade filter labels must use tmp0, tmp1 format (not v01, v02) to avoid FFmpeg parsing errors.
• For detailed pitfalls and debugging guidance, see references/ffmpeg-pitfalls.md.