Volcengine Video Lip Sync: AI Video Editor
Sync any video's lips to a new audio track with Volcengine Lip Sync. Supports scene detection and vocal separation. Try free — pay per generation.
Sync any video's lips to a new audio track with Volcengine Lip Sync. Supports scene detection and vocal separation. Try free — pay per generation.
About this model
Volcengine Video Lip Sync re-times the mouth movements in an existing video to match a new target audio track, producing a natural-looking dubbed or re-voiced result. It supports a fast lite mode tuned for single-person frontal footage and a basic mode that adds scene segmentation and speaker identification for more complex, multi-shot scenes. The output is returned as an MP4 at 25fps, with its duration following the audio track — trimming the video if it runs longer than the audio, or looping it if shorter. Optional vocal separation strips background noise from the driving audio before it's applied.
Cost analysis
| Provider | Cost | Notes |
|---|---|---|
| muapiapp | $0.06/sec of audio (1.3× margin applied) | Pay-per-generation, no subscription required. |
| Fal.ai | Not available | Fal.ai does not offer this specific model. |
| Replicate | Not available | Replicate does not offer this specific model. |
Pay-per-generation, no subscription required.
Fal.ai does not offer this specific model.
Replicate does not offer this specific model.
* Competitor pricing is estimated based on similar model architectures and usage tiers.
Configuration schema
| Parameter | Type | Description | Default |
|---|---|---|---|
| Mode | Enum (2 options) | Service mode. 'lite' is for single-person frontal videos with faster processing. 'basic' is for single-person complex scenes, supporting scene segmentation and speaker identification. | lite |
| Video URL | string | Video URL. Supported resolution: 360p-1080p. Videos above 1080p are compressed to 1080p; below 360p is not supported. Supported formats: MOV, MP4, HDR. Max file size: 500MB. | https://d3adwkbyhxyrtq.cloudfront.net/muapi/data/volcengine-lipsync-video.mp4 |
| Audio URL | string | Target pure vocal audio URL used to drive the video's lip movements. Max file size: 10MB. | https://d3adwkbyhxyrtq.cloudfront.net/muapi/data/volcengine-lipsync-audio.mp3 |
| Separate Vocal | boolean | Enable vocal separation to suppress background noise. | false |
| Scene Detection | boolean | Enable scene segmentation and speaker identification. Only supported in Basic mode. | false |
| Align Audio (Loop Video) | boolean | Supported in Lite mode. Whether to loop the video when the audio is longer than the video. | true |
| Align Audio Reverse | boolean | Supported in Lite mode. Whether to loop the video in reverse (backward). Requires align_audio to be true. | false |
| Template Start Seconds | int | Supported in Lite mode. Start time of the template video, in seconds. | 0 |
Service mode. 'lite' is for single-person frontal videos with faster processing. 'basic' is for single-person complex scenes, supporting scene segmentation and speaker identification.
liteVideo URL. Supported resolution: 360p-1080p. Videos above 1080p are compressed to 1080p; below 360p is not supported. Supported formats: MOV, MP4, HDR. Max file size: 500MB.
https://d3adwkbyhxyrtq.cloudfront.net/muapi/data/volcengine-lipsync-video.mp4Target pure vocal audio URL used to drive the video's lip movements. Max file size: 10MB.
https://d3adwkbyhxyrtq.cloudfront.net/muapi/data/volcengine-lipsync-audio.mp3Enable vocal separation to suppress background noise.
falseEnable scene segmentation and speaker identification. Only supported in Basic mode.
falseSupported in Lite mode. Whether to loop the video when the audio is longer than the video.
trueSupported in Lite mode. Whether to loop the video in reverse (backward). Requires align_audio to be true.
falseSupported in Lite mode. Start time of the template video, in seconds.
0Developer documentation
Prepare your inputs: Upload a source video (360p–1080p, MOV/MP4/HDR, up to 500MB) and a clean vocal audio track (MP3, WAV, AAC, or OGG, up to 10MB) that will drive the lip movements.
Pick a mode: Use mode: "lite" for a single-person frontal video for faster processing, or mode: "basic" for more complex scenes with multiple shots or speakers (enables open_scenedet).
Tune alignment (lite mode): If your audio is longer than the video, align_audio (default true) loops the video to match; align_audio_reverse loops it backward instead. templ_start_seconds sets where in the video the template starts.
Submit and poll: Call the endpoint below, then poll the task-detail endpoint (or use callBackUrl) until the job completes.
curl -X POST https://api.muapi.ai/api/v1/volcengine-video-to-video-lip-sync \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"mode": "lite",
"video_url": "https://your-domain.com/video/example.mp4",
"audio_url": "https://your-domain.com/audio/speech.mp3",
"separate_vocal": false
}'
Frequently asked
It takes an existing video and a target audio track, then regenerates the speaker's mouth movements so they match the new audio — useful for dubbing or re-voicing footage without reshooting.
`lite` mode is optimized for single-person, frontal-facing videos and processes faster. `basic` mode supports more complex scenes, including scene segmentation and speaker identification via `open_scenedet`.
Video: MOV, MP4, or HDR, 360p–1080p resolution (above 1080p is compressed down), up to 500MB, 24–60fps, 1–30 Mbps bitrate. Audio: MP3, WAV, AAC, MP4-audio, or OGG, up to 10MB.
The output duration always follows the audio. If the video is shorter than the audio, it's looped (`align_audio`) — optionally in reverse (`align_audio_reverse`) — to fill the gap. If the video is longer, it's trimmed to the audio length.
When enabled, it runs vocal separation on the input audio to suppress background music or noise before using it to drive lip movement, improving sync accuracy on noisy tracks.
The generated video is returned as MP4 at 25fps, matching the input video's resolution up to 1080p.