{"slug":"jianshuo-wjs-dubbing-video-59629f","title":"\u89c6\u9891\u914d\u97f3TTS\u65f6\u95f4\u8f74\u5bf9\u9f50","description":"\u628a\u5df2\u6709\u76ee\u6807\u8bed\u8a00SRT\u7684\u89c6\u9891\u505a\u6210\u65f6\u95f4\u5bf9\u9f50\u7684TTS\u914d\u97f3\u8f68\uff1a\u6309\u97f3\u8272ID\u8def\u7531\u706b\u5c71\u8c46\u5305\u6216edge-tts\uff0c\u9010\u6761\u5408\u6210\u540e\u6309SRT\u69fd\u4f4d\u62c9\u4f38\u6216\u8865\u9759\u97f3\uff0c\u8f93\u51fa\u66ff\u6362\u97f3\u8f68\u7684mp4\u3002\u9ed8\u8ba4\u5355\u8bf4\u8bdd\u4eba\uff0c\u591a\u8bf4\u8bdd\u4eba\u4e3a\u663e\u5f0f\u53ef\u9009\u3002\u6df7\u97f3\u4e0e\u5b57\u5e55\u70e7\u5f55\u4ea4\u7ed9\u4e0b\u6e38\u6280\u80fd\u3002","category":"\u53e3\u64ad\u8868\u8fbe","tags":["jianshuo","\u5efa\u7855","\u53e3\u64ad\u8868\u8fbe"],"author":"\u5efa\u7855 \u00b7 jianshuo","source_url":"https://github.com/jianshuo/claude-skills/blob/d36bc423286a34eb32adc604121e5c7714b08740/wjs-dubbing-video/SKILL.md","license":"MIT","status":"published","verification":"source_reviewed","verification_notes":"\u5df2\u6838\u5bf9\u539f\u59cb\u4ed3\u5e93\u3001\u56fa\u5b9a\u63d0\u4ea4\u3001\u8bb8\u53ef\u3001\u6280\u80fd\u6b63\u6587\u4e0e\u5305\u7ed3\u6784\uff0c\u5e76\u8865\u5145\u4e2d\u6587\u6267\u884c\u9a8c\u6536\u8bf4\u660e\uff1b\u672a\u8fd0\u884c\u5305\u5185\u811a\u672c\u3002","revision":"e55616fac71f4f59904713dbde069867","created_at":"2026-11-02T14:47:08+00:00","updated_at":"2026-11-02T14:47:08+00:00","source_repo":"jianshuo/claude-skills","source_path":"wjs-dubbing-video/SKILL.md","source_commit":"d36bc423286a34eb32adc604121e5c7714b08740","source_updated_at":"2026-11-02T14:46:04+00:00","stars":0,"proxy_score":null,"score":null,"grade":null,"dimensions":{},"evaluation":{},"installs":0,"platforms":[],"formats":[],"category_slug":"delivery","category_name":"\u53e3\u64ad\u8868\u8fbe","quality":{"review_type":"text_and_package_review","readiness":"requires_setup","reviewed_at":"2026-11-02T14:46:04+00:00","review_summary":"\u6d41\u7a0b\u4e0e\u8e29\u5751\u8bf4\u660e\u5177\u4f53\uff0c\u4ea7\u7269\u8def\u5f84\u660e\u786e\uff1b\u4f46\u9700\u7528\u6237\u81ea\u5907ffmpeg\u3001venv\u4e0e\u706b\u5c71\u8d26\u53f7\uff0c\u4e14\u5f15\u7528\u7684 sample_voices.py \u7f3a\u5931\uff0c\u811a\u672c\u672a\u8fd0\u884c\u9a8c\u8bc1\u3002","cost_notes":"\u706b\u5c71\u8c46\u5305TTS\u6309\u8c03\u7528\u8ba1\u8d39\uff0c\u4ef7\u683c\u4e0e\u989d\u5ea6\u4ee5\u7528\u6237\u8d26\u53f7\u63a7\u5236\u53f0\u4e3a\u51c6\uff0c\u672c\u5305\u4e0d\u62a5\u4ef7\uff1bedge-tts\u514d\u8d39\u4f46\u53d7\u7aef\u70b9\u9650\u6d41\u5f71\u54cd\u3002","source_sha256":"8f5b50b58398b94cb5803d9d8f23bdfca2c91858e5984b3ce408a4aca9beafd9","prerequisites":["Python\u73af\u5883\u4e0evenv\uff0c\u5b89\u88c5 edge-tts\u3001requests\uff1b\u591a\u8bf4\u8bdd\u4eba\u53e6\u9700 mediapipe\u3001opencv-python","ffmpeg \u4e0e ffprobe \u53ef\u7528","\u706b\u5c71TTS\u9700\u7528\u6237\u81ea\u6709\u8d26\u53f7\u5e76\u5df2\u5f00\u901a\u5bf9\u5e94\u8d44\u6e90\uff0c\u51ed\u636e\u7531\u7528\u6237\u81ea\u884c\u653e\u5165\u73af\u5883\u53d8\u91cf","\u9996\u6b21\u8fd0\u884c visual_diarize.py \u4f1a\u4e0b\u8f7d\u7ea63.6MB\u6a21\u578b\u5230 /tmp"],"checks":["\u8f93\u51famp4\u65f6\u957f\u4e0e\u6e90\u89c6\u9891\u4e00\u81f4\uff0c\u97f3\u8f68\u5df2\u66ff\u6362\u4e14\u89c6\u9891\u6d41\u4e3acopy\u672a\u91cd\u7f16\u7801","\u9010\u6761\u7247\u6bb5\u65f6\u957f\u4e0eSRT\u69fd\u4f4d\u5bf9\u9f50\uff0c\u62c9\u4f38\u672a\u4f4e\u4e8e0.82\u00d7\uff0c\u8d85\u51fa\u90e8\u5206\u4ee5\u9759\u97f3\u8865\u9f50","\u5168\u7247\u6e32\u67d3\u524d\u5df2\u63d0\u4f9b\u8bd5\u542c\u6837\u672c\u5e76\u83b7\u7528\u6237\u660e\u786e\u9009\u5b9a\u97f3\u8272","\u706b\u5c71\u97f3\u8272\u5728\u7528\u6237\u5b9e\u4f8b\u4e0a\u5148\u505a5\u8bcd\u5192\u70df\u6d4b\u8bd5\uff0c\u5931\u8d25\u65f6\u5982\u5b9e\u62a5\u544a\u5f00\u901a\u95ee\u9898\u800c\u975e\u91cd\u8bd5\u627f\u8bfa","\u591a\u8bf4\u8bdd\u4eba\u65f6\u4f4e\u7f6e\u4fe1\u5ea6\uff08confidence_ratio<1.5\uff09\u6761\u76ee\u5df2\u5217\u51fa\u5f85\u4eba\u5de5\u6821\u6b63"],"limitations":["\u672c\u6b21\u4ec5\u5ba1\u9605\u6587\u672c\u4e0e\u811a\u672c\u6e05\u5355\uff0c\u672a\u8fd0\u884c dub.py \u6216 visual_diarize.py\uff0c\u65e0\u6cd5\u786e\u8ba4\u5b9e\u9645\u53ef\u7528\u6027","\u706b\u5c71\u97f3\u8272\u53ef\u7528\u6027\u4f9d\u8d56\u7528\u6237\u5b9e\u4f8b\u5957\u9910\uff0c\u6587\u6863\u6240\u5217\u97f3\u8272\u53ef\u80fd\u8fd4\u56de55000000","\u89c6\u89c9\u8bf4\u8bdd\u4eba\u5206\u79bb\u5728\u8bf4\u8bdd\u4eba\u79bb\u955c\u3001\u955c\u5934\u5207\u6362\u6216\u4e09\u4eba\u4ee5\u4e0a\u540c\u4f4d\u7f6e\u65f6\u5931\u6548\uff0c\u5305\u5185\u4e0d\u542b\u97f3\u9891\u5206\u79bb\u65b9\u6848","edge-tts \u7aef\u70b9\u9650\u6d41\u4e0e\u7a33\u5b9a\u6027\u4e0d\u53d7\u672c\u5305\u63a7\u5236"],"issues":["SKILL.md \u63d0\u5230 scripts/sample_voices.py \u4f5c\u4e3a\u8bd5\u542c\u5305\u88c5\u5668\uff0c\u4f46\u6587\u4ef6\u6e05\u5355\u4e2d\u4e0d\u5b58\u5728\u8be5\u811a\u672c\uff0c\u9700\u6309\u6587\u4e2d\u8bf4\u660e\u81ea\u884c\u7528Python\u5faa\u73af\u5b9e\u73b0","\u591a\u5904\u201c\u5df2\u9a8c\u8bc1/validated baseline\u201d\u4e3a\u4f5c\u8005\u81ea\u8ff0\uff0c\u5305\u5185\u65e0\u6d4b\u8bd5\u8bb0\u5f55\uff0c\u5c5e\u672a\u6838\u5b9e","\u51ed\u636e\u8bfb\u53d6\u793a\u4f8b\u4e3a source ~/code/.env\uff0c\u5c5e\u4f5c\u8005\u672c\u5730\u4e60\u60ef\uff0c\u672a\u8bf4\u660e\u5176\u4ed6\u73af\u5883\u7684\u914d\u7f6e\u65b9\u5f0f"],"missing_files":[]},"quality_url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/files/SHILIU-QUALITY.md","usage_guide":{"task":"\u7528\u76ee\u6807\u8bed\u8a00SRT\u4e3a\u89c6\u9891\u751f\u6210\u65f6\u95f4\u5bf9\u9f50\u7684\u914d\u97f3\u97f3\u8f68\uff0c\u5e76\u8f93\u51fa\u66ff\u6362\u97f3\u8f68\u7684\u89c6\u9891\u6587\u4ef6\u3002","inputs":"\u6e90\u89c6\u9891\u6587\u4ef6\uff1b\u76ee\u6807\u8bed\u8a00SRT\uff08\u5982 *.zh-CN.srt\uff09\uff1b\u9009\u5b9a\u97f3\u8272ID\u4e0e\u8bed\u901f/\u97f3\u9ad8\uff1b\u706b\u5c71TTS\u9700 VOLC_TTS_APPID \u4e0e VOLC_TTS_ACCESS_TOKEN\uff08\u7528\u6237\u81ea\u884c\u914d\u7f6e\uff0c\u4e0d\u5f97\u4ee3\u586b\uff09\uff1bedge-tts\u9700\u672c\u5730venv\u5b89\u88c5\uff1bffmpeg/ffprobe\u3002","outputs":"<\u6e90\u6587\u4ef6\u540d>_<\u8bed\u8a00>_dub.mp4\uff08\u89c6\u9891\u6d41copy\u3001\u97f3\u8f68\u66ff\u6362\uff09\uff1bdub_work/seg_NN.mp3 \u9010\u6761\u7247\u6bb5\uff1b\u591a\u8bf4\u8bdd\u4eba\u65f6\u53e6\u6709 diarized SRT \u4e0e JSON \u62a5\u544a\u3002","example_input":"\u6211\u6709 input.mp4 \u548c input.zh-CN.srt\uff0c\u8bf7\u7528 zh-CN-XiaoxiaoNeural \u8bed\u901f-8%\u3001\u97f3\u9ad8-10Hz \u505a\u4e2d\u6587\u914d\u97f3\uff0c\u5148\u7ed9\u62113\u4e2a\u5019\u9009\u97f3\u8272\u7684\u77ed\u6837\u786e\u8ba4\uff0c\u518d\u8dd1\u5168\u7247\u3002","example_output":"\u793a\u610f\uff1a\u5148\u8fd4\u56de3\u20134\u6bb53\u20138\u79d2\u8bd5\u542c\u6837\u672c\u4e0e\u97f3\u8272\u8bf4\u660e\uff1b\u786e\u8ba4\u540e\u8f93\u51fa input_zh_dub.mp4\u3001dub_work/ \u4e0b\u9010\u6761mp3\u3001\u4ee5\u53ca\u6bcf\u6761\u69fd\u4f4d\u662f\u5426\u62c9\u4f38/\u8865\u9759\u97f3\u7684\u65e5\u5fd7\u3002\u4ee5\u4e0a\u4e3a\u7ed3\u6784\u793a\u610f\uff0c\u975e\u5b9e\u9645\u751f\u6210\u7ed3\u679c\u3002"},"read_url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f","skill_url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/files/SKILL.md","download_url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/download","install_url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/install","files":[{"path":"LICENSE","bytes":1082,"url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/files/LICENSE"},{"path":"SHILIU-QUALITY.md","bytes":2138,"url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/files/SHILIU-QUALITY.md"},{"path":"SKILL.md","bytes":21522,"url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/files/SKILL.md"},{"path":"scripts/dub.py","bytes":13318,"url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/files/scripts/dub.py"},{"path":"scripts/visual_diarize.py","bytes":8585,"url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/files/scripts/visual_diarize.py"},{"path":"shiliu-quality.json","bytes":2105,"url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/files/shiliu-quality.json"},{"path":"shiliu-source.json","bytes":269,"url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/files/shiliu-source.json"},{"path":"usage.json","bytes":965,"url":"https://shiliuai.cn/api/v2/skills/jianshuo-wjs-dubbing-video-59629f/files/usage.json"}],"content":"---\nname: wjs-dubbing-video\ndescription: Use when the user has a video + a target-language SRT and wants the video to actually speak that language \u2014 generates a time-aligned TTS voice dub. Routes by voice ID \u2014 Volcano (\u8c46\u5305) TTS for Chinese, edge-tts neural for any language. Defaults to one voice (single-speaker); opt-in multi-speaker via visual diarization. Outputs `*_<lang>_dub.mp4` with the dub audio in place of the original. Final mixing (audio bed + burn-in) is handed off to `/wjs-burning-subtitles`. Triggers \u2014 \"\u914d\u97f3\", \"\u4e2d\u6587\u914d\u97f3\", \"Chinese dub\", \"voice over this\", \"dub the video\", \"TTS this SRT\", \"different voice for each speaker\".\n---\n\n# wjs-dubbing-video\n\nVideo + target-language SRT \u2192 `*_<lang>_dub.mp4` with a time-aligned TTS voice. **This skill stops at the dub track.** Burn-in + audio bed mixing is the next skill (`/wjs-burning-subtitles/render.py` composites everything in one final encode).\n\n## When to use\n\n- User has a target-language SRT (e.g., `entrevista.zh-CN.srt`) and wants the video to speak that language.\n- User says \"\u4e2d\u6587\u914d\u97f3 / \u914d\u97f3 / \u5e2e\u6211\u505a\u914d\u97f3 / dub it / voice over\".\n- User has multiple speakers on camera and wants different voices per speaker.\n\n## When NOT to use\n\n- No SRT yet \u2192 run `/wjs-transcribing-audio` then `/wjs-translating-subtitles` first.\n- Source-language only TTS (rare; usually you translate first) \u2192 still use this skill, but pass the source SRT.\n- Burn-in only, no audio change \u2192 skip to `/wjs-burning-subtitles`.\n\n## Number of speakers \u2014 default to one\n\n**Default: assume one speaker.** Use a single voice for the entire dub. This is the right answer for monologues, vlogs, recorded talks, narrator-only clips, and the overwhelming majority of videos people ask about. Don't run diarization, don't tag the SRT with `[A]`/`[B]`, don't bring up multi-speaker complexity.\n\n**Switch to multi-speaker only when the user explicitly says so** \u2014 phrasings like \"two people\", \"interview\", \"dialogue\", \"conversation between\", \"separate the speakers\", \"different voice for each\", or a direct request to do diarization. When triggered, follow the \"Multi-speaker dubbing\" section below.\n\nIf you're unsure whether a video is one speaker or many, ship the single-voice version first. Adding speaker separation later is cheap (just regenerate the dub); shipping confused multi-speaker output by default wastes the user's time.\n\n## Engine routing \u2014 by voice ID\n\n`scripts/dub.py` auto-routes by voice-ID prefix:\n\n| Voice ID pattern | Engine | Auth |\n|---|---|---|\n| `zh_..._bigtts` | **Volcano (\u5b57\u8282\u8df3\u52a8\u8c46\u5305) TTS** | `VOLC_TTS_APPID` + `VOLC_TTS_ACCESS_TOKEN` |\n| `zh-CN-...Neural` / `en-US-...Neural` / etc. | **edge-tts** (Microsoft Edge neural) | none (free) |\n\nFor Mandarin, Volcano is markedly more natural than edge-tts, especially for emotional/contemplative content. Use edge-tts when Volcano credentials aren't available or as a debugging fallback.\n\n## Volcano TTS (Chinese only)\n\nEndpoint: `https://openspeech.bytedance.com/api/v3/tts/unidirectional` (used for both TTS 1.0 and 2.0; the Resource-Id header picks the backend).\n\nHeaders:\n\n```\nX-Api-App-Id:       (env: VOLC_TTS_APPID)         # 10-digit speech App ID\nX-Api-Access-Key:   (env: VOLC_TTS_ACCESS_TOKEN)  # 32-char token from speech console\nX-Api-Resource-Id:  volc.service_type.10029       # see resource ID note below\nContent-Type:       application/json\n```\n\nLoading credentials: most users keep them in `~/code/.env`. Read them at the top of any session via:\n\n```bash\nset -a; source ~/code/.env; set +a\n```\n\n### Resource ID \u2014 important quirk\n\nThe doc lists `seed-tts-2.0` as the \"TTS 2.0 (recommended)\" resource, but a typical TTS-SeedTTS2.0 console instance does **not** include the popular `*_bigtts` speaker catalog (\u723d\u5feb\u65af\u65af, \u9ad8\u51b7\u5fa1\u59d0, \u5f00\u6717\u59d0\u59d0, etc.). Trying those speakers against `seed-tts-2.0` returns `200 code=55000000 \"resource ID is mismatched with speaker related resource\"`. The fix is to use `volc.service_type.10029` (the TTS 1.0 V3 endpoint) \u2014 the audio quality of the bigtts speakers is identical, and they all work against this resource. The bundled `dub.py` defaults to `volc.service_type.10029`; override with `VOLC_TTS_RESOURCE` env if you have a different instance.\n\nOther 401/403 errors:\n\n- `401 code=45000010 \"load grant: requested grant not found in SaaS storage\"` \u2014 the App ID + key combo is valid against the gateway, but the user has not activated this resource. They must go to \u706b\u5c71\u5f15\u64ce \u2192 \u8bed\u97f3\u6280\u672f \u2192 \u8bed\u97f3\u5408\u6210\u5927\u6a21\u578b \u2192 \u5b9e\u4f8b\u7ba1\u7406 and \u5f00\u901a the service. No workaround.\n- `403 code=45000030` \u2014 the speaker isn't included in the user's instance bundle.\n\n### Response format\n\nDespite the doc's casual language, the response is **streaming NDJSON**, not a single JSON object and not raw audio bytes. Each line is a separate JSON event with a base64-encoded MP3 chunk in `data`. The terminal event has `code: 20000000` (which means OK in this API's success codes \u2014 different from `code: 0`). Concatenate the decoded chunks for the full MP3.\n\n```python\nimport base64, json, requests\naudio = b\"\"\nr = requests.post(url, headers=h, json=payload, timeout=60, stream=True)\nfor line in r.iter_lines():\n    if not line: continue\n    evt = json.loads(line)\n    if evt.get(\"code\") not in (0, None, 20000000):\n        raise RuntimeError(f\"code={evt.get('code')} {evt.get('message')}\")\n    if evt.get(\"data\"):\n        audio += base64.b64decode(evt[\"data\"])\n```\n\n### Speaker catalog (verified working under `volc.service_type.10029`)\n\nFull list at volcengine.com/docs/6561/1257544 \u2014 but availability depends on your instance bundle. Confirmed-working female voices for the typical SeedTTS-2.0 starter instance:\n\n| Speaker ID                                    | \u4e2d\u6587\u540d     | Feel                       |\n| ---                                           | ---        | ---                        |\n| `zh_female_gaolengyujie_moon_bigtts`          | \u9ad8\u51b7\u5fa1\u59d0   | **Best for contemplative/spiritual content.** Mature, restrained, calm. |\n| `zh_female_kailangjiejie_moon_bigtts`         | \u5f00\u6717\u59d0\u59d0   | Warm older-sister storytelling. |\n| `zh_female_shuangkuaisisi_moon_bigtts`        | \u723d\u5feb\u65af\u65af   | Versatile, conversational baseline. |\n| `zh_female_linjianvhai_moon_bigtts`           | \u90bb\u5bb6\u5973\u5b69   | Casual, lifestyle-vlog. |\n| `zh_female_yuanqinvyou_moon_bigtts`           | \u5143\u6c14\u5973\u53cb   | Lively, upbeat. |\n| `zh_female_meilinvyou_moon_bigtts`            | \u7f8e\u4e3d\u5973\u53cb   | Soft, intimate. |\n| `zh_female_shuangkuaisisi_emo_v2_mars_bigtts` | \u65af\u65af\u60c5\u611f\u7248 | Full emotional range \u2014 pair with explicit emotion + scale. |\n\nThese voices return 55000000 against the typical instance even though the doc lists them: `vv_uranus_bigtts`, `wenroushunv_moon_bigtts`, `qingxin_moon_bigtts`, `yingmaoxiaoyuan_moon_bigtts`, `tianxinxiaoling_moon_bigtts`, `shaoergushi_moon_bigtts`. Don't promise them without testing.\n\n### Audio params\n\n`speech_rate` is Volcano's native scale [-50, +100] where the value is a percentage delta (so `-8` means 8% slower). The script passes `--rate -8%` through as `-8`.\n\nUseful emotion presets:\n\n- `emotion=\"calm\"`, `emotion_scale=4` \u2014 contemplative, default for this skill's spiritual-content niche.\n- `emotion=\"gentle\"` \u2014 softer / more intimate.\n- `emotion=\"neutral\"` \u2014 flat / informational.\n- `emotion=\"sad\"` \u2014 melancholic. Use sparingly.\n\nOverride `dub.py` defaults with `VOLC_TTS_EMOTION` and `VOLC_TTS_EMOTION_SCALE` env vars without editing code.\n\n**No English Volcano voices** are wired up in this skill \u2014 for English use edge-tts (next section). Volcano does have English speakers (`en_male_*_bigtts`, `en_female_*_bigtts`) but they aren't typically included in TTS-SeedTTS-2.0 starter instances. Add them by extending the voice routing in `dub.py` once verified.\n\n## edge-tts (Microsoft Edge neural TTS)\n\nFree, no API key, high-quality but less expressive than Volcano. Install into a project venv \u2014 **do not** call it via `uvx` once per segment. Each `uvx` invocation spawns a fresh Python process and the bing endpoint will rate-limit or RST the connection after a handful of rapid hits, breaking mid-render.\n\n```bash\nuv venv .venv\nuv pip install --python .venv/bin/python edge-tts\n```\n\nThen drive it from a single long-lived Python process using `edge_tts.Communicate(...)` directly, with retry-on-failure logic. The bundled `scripts/dub.py` does this.\n\n## Voice selection \u2014 match the original speaker\n\nThere is no perfect cross-language match \u2014 choose gender, age feel, and tone deliberately, then bend with rate/pitch.\n\n### Chinese voices (Volcano preferred, edge-tts fallback)\n\nVolcano's `zh_female_gaolengyujie_moon_bigtts` (\u9ad8\u51b7\u5fa1\u59d0, calm, `speech_rate=-8`) is the validated baseline for mature contemplative female speakers \u2014 equivalent to or better than any edge-tts option for that profile. See the Volcano speaker table above for the rest.\n\nedge-tts catalog (Chinese):\n\n| Voice                              | Gender | Default feel                  |\n| ---                                | ---    | ---                           |\n| `zh-CN-XiaoxiaoNeural`             | F      | Warm, news/novel              |\n| `zh-CN-XiaoyiNeural`               | F      | Lively, young                 |\n| `zh-CN-YunjianNeural`              | M      | Passionate, sports            |\n| `zh-CN-YunxiNeural`                | M      | Sunshine, lively              |\n| `zh-CN-YunyangNeural`              | M      | Professional newsreader       |\n| `zh-HK-HiuMaanNeural`              | F      | Friendly, slightly mature     |\n| `zh-TW-HsiaoChenNeural`            | F      | Friendly                      |\n\n### English voices (edge-tts neural, all multilingual)\n\nAll voices below speak fluent American/British/Australian English; the `*Multilingual*` ones also handle Spanish names, French/Italian loanwords, etc. without mispronunciation.\n\n| Voice                                  | Gender | Default feel                            |\n| ---                                    | ---    | ---                                     |\n| `en-US-AvaMultilingualNeural`          | F      | **Best for warm/mature/caring** \u2014 natural for spiritual or coaching content |\n| `en-US-EmmaMultilingualNeural`         | F      | Cheerful, conversational, younger        |\n| `en-US-AndrewMultilingualNeural`       | M      | Warm, confident, sincere                 |\n| `en-US-BrianMultilingualNeural`        | M      | Approachable, casual                     |\n| `en-US-AriaNeural`                     | F      | Crisp newsreader                         |\n| `en-US-GuyNeural`                      | M      | Steady male newsreader                   |\n| `en-GB-SoniaNeural`                    | F      | British female (RP)                      |\n| `en-GB-RyanNeural`                     | M      | British male (RP)                        |\n| `en-AU-WilliamMultilingualNeural`      | M      | Australian male                          |\n| `fr-FR-VivienneMultilingualNeural`     | F      | Mature European female who also reads English |\n\nFor matching a mature contemplative Spanish female (this skill's canonical use case), start with `en-US-AvaMultilingualNeural` at `--rate -5% --pitch -3Hz`. Do **not** use the news-style `Aria` or `Guy` for spiritual content \u2014 they sound clinical.\n\n### Picking heuristics\n\n- **Mature contemplative female speaker (yoga/spirituality/coaching):** `zh-CN-XiaoxiaoNeural` with `--rate=-8% --pitch=-10Hz` (or Volcano `gaolengyujie`).\n- **Mature professional male:** `zh-CN-YunyangNeural` with `--rate=-5%`. Avoid Yunjian/Yunxi (too energetic).\n- **Young casual speaker:** Defaults; no pitch shift.\n- **Western-mouth feel:** one of the `*MultilingualNeural` voices.\n\n## Always sample before committing\n\n\ud83d\uded1 **Checkpoint \u2014 sample before full dub.** A full-video dub is the most expensive step (TTS API calls + atempo + ffmpeg mux). Before running `dub.py` over the whole SRT:\n\n1. Pick the longest-text cue (worst stretch case) and one short/casual cue (timbre check).\n2. Synthesize 3\u20134 voice/rate/pitch combos at 3\u20138s each.\n3. Show the user the audio panel and ask: \"\u9009\u54ea\u4e2a voice\uff1frate/pitch \u8981\u8c03\u5417\uff1f\u786e\u8ba4\u540e\u6211\u518d\u8dd1\u5168\u7247\u3002\" Wait for explicit pick.\n\nSkip the checkpoint only if the user named a specific voice up front AND has already heard a sample of that voice on this video.\n\nThe script's `scripts/sample_voices.py` (if present) is a thin wrapper for exactly this; otherwise drive the same Python loop the dub script uses.\n\n**Mandatory smoke test before promising any Volcano voice on a new account:** synth one ~5-word cue with that speaker ID first; only quote it to the user if the smoke test returns a non-empty MP3. If the smoke test 401s with `code=45000010` (\"grant not found\"), tell the user they need to \u5f00\u901a the resource in \u706b\u5c71\u5f15\u64ce console \u2014 do not pretend it'll work after a retry.\n\n## Running dub.py\n\n```bash\n.venv/bin/python ~/.claude/skills/wjs-dubbing-video/scripts/dub.py [voice] [rate] [pitch]\n\n# Mature Chinese contemplative female (Volcano):\n.venv/bin/python ~/.claude/skills/wjs-dubbing-video/scripts/dub.py \\\n    zh_female_gaolengyujie_moon_bigtts -8% +0Hz\n\n# Warm English caring female (edge-tts, multilingual):\n.venv/bin/python ~/.claude/skills/wjs-dubbing-video/scripts/dub.py \\\n    en-US-AvaMultilingualNeural -5% -3Hz\n\n# Default Chinese fallback (no Volcano creds needed):\n.venv/bin/python ~/.claude/skills/wjs-dubbing-video/scripts/dub.py \\\n    zh-CN-XiaoxiaoNeural -8% -10Hz\n```\n\nThe script:\n\n1. Reads the SRT (auto-detects `*.zh-CN.srt`, `*.en.srt`, etc., or pass `--srt`).\n2. Synthesizes one MP3 per cue under `dub_work/seg_NN.mp3`.\n3. Probes each clip's actual duration with `ffprobe`.\n4. For each cue: if TTS is longer than the SRT slot, chains `atempo` filters to speed it up; if shorter, pads with silence after.\n5. Inserts silence segments for SRT gaps and any trailing tail so the output audio length exactly matches the source video.\n6. Muxes the new audio into `*_zh_dub.mp4` / `*_en_dub.mp4` keeping the original video stream by `-c:v copy`.\n\nOutput: `<source-stem>_<lang>_dub.mp4` (e.g., `entrevista_zh_dub.mp4`). This is the input for the next step \u2014 `/wjs-burning-subtitles/render.py` \u2014 which composites the final video.\n\n## Filling awkward silences\n\nMandarin takes 60\u201380% of the time Spanish does to say the same thing. With strict cue-by-cue timing, that leaves awkward 2\u20134s silences at the end of most cues. English is closer to ~85% of Spanish. Three levers, in increasing impact:\n\n1. **Slow the native TTS rate.** Changing `--rate` from `+0%` to `-12%` to `-15%` produces clean, natural-sounding slower speech (much better than time-stretching afterward). Try `-12%` first; `-15%`/`-20%` for very contemplative content.\n\n2. **Mild slow-stretch per cue.** When a cue's TTS is still shorter than its slot, run `atempo` between 0.82\u00d7 and 0.95\u00d7. `dub.py` does this automatically: when slack > 0.5s, it sets `atempo = max(0.82, tts_dur / target_dur)` and pads the remainder. Below 0.82\u00d7 the voice starts sounding drugged; above 0.92\u00d7 the stretch is essentially imperceptible.\n\n3. **Expand the target-language text in the worst cues.** When the slot is so long that even 0.82\u00d7 stretch leaves >2s of silence, the cleanest fix is to lengthen the translation. Add natural Mandarin particles (\"\u55ef\uff0c\", \"\u5176\u5b9e\", \"\u4e5f\u5c31\u662f\u8bf4\", \"\u4f60\u77e5\u9053\") or unpack a compressed phrase into its full meaning. This changes the on-screen subtitle, so confirm with the user before doing it. Edit the SRT, regenerate just those segments by deleting their `dub_work/seg_NN.mp3` and re-running `dub.py`.\n\nCombine the levers: native rate `-12%` + stretch-to-fit handles ~80% of cases. Reserve text expansion for the 2\u20133 worst outliers.\n\n## Multi-speaker dubbing (opt-in)\n\n**Only invoke this section when the user explicitly says the source has multiple speakers** (\"interview\", \"two people\", \"dialogue\", \"separate the speakers\", \"different voice for each\", or a direct request to do diarization).\n\nWhen triggered, generate the dub with a different voice per speaker so the listener can follow who's speaking. Two paths:\n\n### Path 1 (recommended for on-camera speakers): visual diarization\n\n`scripts/visual_diarize.py` watches mouth movement per face per frame and tags each cue with the dominant speaker. Self-contained, no API keys, no audio fingerprinting.\n\n```bash\nuv pip install --python .venv/bin/python mediapipe opencv-python\n\n.venv/bin/python ~/.claude/skills/wjs-dubbing-video/scripts/visual_diarize.py \\\n    --video input.mp4 --srt input.en.srt \\\n    --out input.en.diarized.srt \\\n    --report diarization_report.json \\\n    --sample-fps 5 --num-speakers 2\n```\n\nHow it works:\n\n1. Samples N frames per second (default 5).\n2. Runs MediaPipe FaceLandmarker (Tasks API) for up to `--num-speakers` faces per frame, 478 landmarks each.\n3. Measures mouth aperture per face as the vertical distance between inner upper lip (idx 13) and inner lower lip (idx 14).\n4. Bins faces by horizontal screen position (x-quantiles) \u2192 speakers `A`, `B`, ... left-to-right.\n5. For every cue's [start, end] window, integrates per-speaker frame-to-frame mouth-aperture change. Highest mover wins the tag.\n6. Writes a `[A]`/`[B]`-prefixed SRT plus a JSON report with per-cue scores and a confidence ratio (winner / runner-up).\n\nOn first run, downloads the FaceLandmarker model (~3.6 MB) to `/tmp/mp_models/face_landmarker.task`.\n\n**Visual is materially better than guessing from text.** In one validation, manual text-based labels split 6/50 between speakers; visual diarization showed the actual split was 29/27 \u2014 text-based guessing was wildly wrong because both people take similar-shaped turns. Always prefer visual when the speakers are on camera.\n\n**Spot-check low-confidence cues.** Any cue in the JSON report with `confidence_ratio < 1.5` is borderline \u2014 usually overlapping speech or one speaker briefly off-frame. Hand-correct before dubbing.\n\n### Path 2 (fallback): manual tagging\n\nFor very short clips (1\u20132 minutes), or when speakers are off-camera, or when visual diarization fails:\n\n```text\n1\n00:00:00,000 --> 00:00:03,400\n[A] So what about that AI rewrite thing?\n\n2\n00:00:03,400 --> 00:00:08,200\n[B] Right \u2014 let me explain the workflow.\n```\n\nSave as `*.tagged.srt`. Keep the **clean** SRT (without tags) for downstream burn-in via `/wjs-burning-subtitles`.\n\n### Routing voices in dub.py\n\nPass `--voice-map` with `speaker=voice` pairs. The positional voice arg is the default for cues with no tag.\n\n```bash\n.venv/bin/python ~/.claude/skills/wjs-dubbing-video/scripts/dub.py \\\n    en-US-AndrewMultilingualNeural -3% +0Hz \\\n    --srt input.en.tagged.srt \\\n    --voice-map \"A=en-US-BrianMultilingualNeural,B=en-US-AndrewMultilingualNeural\"\n```\n\nVoice-pairing tips:\n\n- **Two of the same gender:** pick voices with audibly different timbre. Brian (casual) + Andrew (warm) works for two American males. Ava (warm female) + Emma (cheerful female) for two females.\n- **Mixed gender:** Ava + Andrew is a clean default.\n- **Accent contrast:** pair `en-US-` and `en-GB-` for distinctness.\n- **Chinese:** mix Volcano voices like `zh_female_gaolengyujie_moon_bigtts` (mature) + `zh_female_kailangjiejie_moon_bigtts` (warm sister).\n\n### Limits\n\nVisual diarization fails when:\n\n- A speaker is consistently off-camera while talking.\n- Camera cuts or zooms make face position unstable across cues.\n- Three or more speakers sit at similar horizontal positions (x-quantile binning is too coarse \u2014 switch to k-means on (x, y) or use audio-based diarization instead).\n\nFor audio-only material (podcasts, voice-overs), fall back to `pyannote.audio` or `whisperx --diarize`. This skill does not yet bundle audio-based diarization.\n\n## Output\n\n- `<source-stem>_<lang>_dub.mp4` \u2014 video stream-copied from source, audio replaced with the time-aligned dub track. Drop-in input for `/wjs-burning-subtitles/render.py`.\n- `dub_work/seg_NN.mp3` \u2014 per-cue TTS clips (kept for resume / per-cue regen).\n\n## Downstream\n\n- **`/wjs-burning-subtitles`** \u2014 to mix the original audio as a low-volume bed, burn the SRT, or both. The final encode happens there in one ffmpeg pass (no cascade). Pass `--video <source.mp4> --dub <source_lang_dub.mp4> [--srt <srt>]` to its `render.py`.\n- The dub-only file (`*_<lang>_dub.mp4`) is technically a finished video and can ship as-is, but it sounds dubbed (because it is). Mixing the original underneath gives the \"professional translation\" feel \u2014 do that in `/wjs-burning-subtitles`.\n\n## Anti-patterns\n\n- \u274c **Calling `uvx edge-tts` once per cue.** Spawns a Python process each time; bing endpoint rate-limits or RSTs mid-render. Use the persistent library path in `dub.py`.\n- \u274c **Trusting `audio_source` without listening.** Always sample a 30 s clip before committing.\n- \u274c **Stretching below 0.82\u00d7 atempo.** Voice starts sounding drugged. Add silence padding or expand text instead.\n- \u274c **Tagging single-speaker SRTs with `[A]`.** Wastes time and the dub sounds the same. Default to one voice.\n- \u274c **Promising a Volcano voice without smoke-testing it on the user's instance.** The doc lists many voices that error with `code=55000000` against typical SeedTTS-2.0 starter bundles. Always synth a 5-word smoke test before quoting.\n- \u274c **Parsing Volcano response as one JSON document.** It's streaming NDJSON; the success terminator is `code=20000000`, not `code=0`. Concatenate base64-decoded `data` chunks for the full MP3.\n","usage":"\u8bfb\u53d6 SKILL.md\uff0c\u5e76\u6309\u9700\u8bfb\u53d6\u914d\u5957\u6587\u4ef6\uff1b\u4e0b\u8f7d ZIP \u83b7\u53d6\u5b8c\u6574\u5305\u3002\u6280\u80fd\u662f\u4efb\u52a1\u65b9\u6cd5\uff0c\u4e0d\u6388\u4e88\u65b0\u589e\u8d26\u6237\u3001\u4ed8\u8d39\u6216\u7cfb\u7edf\u6743\u9650\u3002"}