Voice-Cloned Lecture Audio Generator
Overview
Generate a warm, natural MP3 of a Chinese lesson explanation or lecture draft,
read in the user's own cloned voice. The skill accepts three input modes
and uses Coqui XTTS v2 zero-shot voice cloning driven by a reference voice
sample (assets/ref_voice.wav). Output preserves the proven style used in prior
samples: cloned timbre + natural pace + a 0.35s breathing pause between
paragraphs + 24kHz/128k MP3.
When to Use
- A user supplies a 课文解读/讲稿/讲解稿 and asks to hear it in their own voice.
- A user sends a Word
.docxlesson file and wants a voice-cloned audio version. - A user sends a photo of text (e.g. a textbook page) and wants it read aloud in their voice.
- Any task that joins voice cloning + TTS for teaching, reading-aloud, or lecture audio.
Do not use this skill for generic TTS with a fixed celebrity/synthetic voice
(use a standard TTS service instead); this skill is specifically about cloning
the user's own reference voice.
Workflow
1. Prepare the environment (first time only)
Follow references/install_and_troubleshoot.md. In short:
pip install torch==2.8.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cpu
pip install -r scripts/requirements.txt
Pin torch==2.8.0 to avoid the torchcodec requirement (torch 2.9 or newer) and keep
transformers below 5.0 to avoid the removed isin_mps_friendly import. Both pitfalls
and their fixes are documented in the same reference file.
2. Ensure a reference voice sample exists
Confirm assets/ref_voice.wav is present. For a real run, replace it with a
6–30s, clean, mono WAV of the user's own voice (16k or 22.05k). The bundled
sample lets the pipeline be tested out of the box.
3. Run from the scripts/ directory
cd scripts
# Text input
python run.py --text "孩子们，上课啦！今天老师带你们去一所大青树下的小学……"
# Word file input
python run.py --file "课文解读.docx"
# Photo input (auto OCR)
python run.py --image "课文照片.jpg"
# Optional: custom output name / skip leading meta lines
python run.py --file "讲稿.docx" --output 三年级上册第一课 --skip 2
Outputs land in outputs/ at the skill root: outputs/NAME.wav and
outputs/NAME.mp3 (NAME defaults to lecture_output unless --output is given).
4. Tune via scripts/config.yaml
Adjust voice.pause_between_paragraphs (breathing pause), voice.device
(cpu/cuda), preprocess.skip_first_paragraphs (skip title/subtitle lines),
and output.format. See references/usage.md for the full parameter table.
Resources
scripts/
Executable code, run directly.
scripts/run.py— unified CLI entry; parses--text/--file/--image,
loads config, runs cloning, writes WAV + MP3.scripts/config.yaml— all tunable parameters (voice sample, model, pause, output).scripts/requirements.txt— pinned dependency set.scripts/src/inputs/— three input parsers:text_input.py,docx_input.py,
image_input.py(EasyOCR).scripts/src/tts/clone_tts.py— XTTS v2 zero-shot cloning core.scripts/src/utils/—preprocess.py(clean/split/chunk text),
audio.py(WAV concat + WAV→MP3).
references/
Load as needed for detailed guidance.
references/usage.md— full feature list, run examples, config table, FAQ.references/install_and_troubleshoot.md— verified install combo and fixes for
theisin_mps_friendly/torchcodec/ EasyOCR / ffmpeg errors.
assets/
Files consumed by the produced output, not loaded into context.
assets/ref_voice.wav— the reference voice sample for cloning (replace with
the user's own voice for real use).assets/sample_lecture.txt— a sample lecture draft for quick testing.
Notes / Limitations
- XTTS v2 model (~2GB) downloads on first run; keep the network stable.
- CPU inference is slow but works; on an NVIDIA GPU set
voice.device: cuda
for a large speed-up. - Voice-cloning quality depends on the reference sample: cleaner, ~10–20s,
single-speaker audio yields the best result.
