CarveHow a long video becomes a vertical clip
faster-whisper transcribes the video with word-level timestamps, PySceneDetect finds the scene boundaries, and Google Gemini 3.0 Flash scores the transcript to pick the 3 to 15 strongest moments of 15 to 60 seconds each. Each moment is then cut with FFmpeg and reframed to 9:16 with MediaPipe face tracking.
The two stages that decide whether a clip is usable are moment scoring and reframing. Everything else is mechanical.
Carve self-hosted is free — run it with Docker on your own machine. Carve Cloud runs the same pipeline on a GPU from $29/month.
What is Carve?
Carve is an AI clip generator that turns long videos (podcasts, webinars, livestreams, interviews) into vertical 9:16 clips for TikTok, Instagram Reels and YouTube Shorts.
The pipeline, stage by stage
Transcription with word-level timestamps
faster-whisper transcribes the source audio and returns a timestamp for every individual word, not just every sentence. Word-level timing is what makes both the clip boundaries and the burned-in subtitles land on the right frame.
Scene boundary detection
PySceneDetect scans the video for cuts and hard visual transitions. These boundaries stop a clip from starting mid-shot, which is the most common reason an auto-generated clip reads as machine-made.
Viral moment scoring
Google Gemini 3.0 Flash receives the timestamped transcript and the scene boundaries together, and returns the 3 to 15 strongest segments of 15 to 60 seconds. Each one is scored on hook strength, emotional payload and how well it stands alone without the surrounding context.
Vertical reframing
Each clip is cropped from 16:9 to 9:16 in one of two modes. TRACK mode follows a single subject with MediaPipe face detection and a YOLOv8 fallback, damped by a stabiliser that holds the camera still inside a safe zone instead of chasing every head movement. GENERAL mode handles group shots and landscapes by keeping the full width over a blurred backdrop.
Subtitles, hooks and effects
Subtitles are generated from the word-level transcript and burned in with FFmpeg. An AI-written hook overlay covers the first seconds. Optional Gemini-generated FFmpeg filter chains handle colour and transitions.
Dubbing and publishing
ElevenLabs dubbing translates the audio into 30+ languages while preserving the speaker's voice, and the dubbed track is re-transcribed so the subtitles match the new language. Finished clips post directly to TikTok, Instagram Reels and YouTube Shorts.
Why does moment scoring need the transcript and the scenes together?
A transcript alone finds a good sentence but has no idea whether the shot cuts halfway through it. Scene boundaries alone find clean cuts with nothing worth saying between them. Passing both to the model at once is what lets it pick a segment that is both quotable and visually intact, which is the difference between a clip a person would watch and one that merely starts and stops in the right places.
Why does the camera hold still instead of following the face exactly?
Because a crop that tracks a face frame by frame produces visible swinging, and the swinging reads as amateur even when the framing is technically correct. The reframing keeps a safe zone around the subject and only moves the crop when they leave it, then damps the movement on the way. A speaker tracker sits on top to stop the crop from flipping between people every time someone nods, and to hold position through brief occlusions.
How long does it take?
On a typical CPU, an 8-minute source video takes roughly 5 to 8 minutes end to end. On an NVIDIA GPU the same video takes about 50 seconds. The gap is almost entirely transcription and encoding; the model call is a small fraction of it.
What does it cost to run?
Carve comes in two editions and they are priced very differently, so it is worth being precise. Carve (self-hosted) is free and free to self-host: Free to self-host. You run it with Docker on your own machine and bring your own API keys. No watermark, no usage cap, no subscription. What it costs you is hardware and time: on a typical CPU an 8-minute video takes 5 to 8 minutes to process. Carve Cloud is the hosted service: The same software with the compute and API keys covered. An NVIDIA GPU clips that same 8-minute video in about 50 seconds. Free plan is 2 videos lifetime with a watermark and no credit card required. Creator plan is $29/month for 10 videos with no watermark. Studio plan is $59/month for 35 videos.
Common questions
- What AI model does Carve use to find viral moments?
- Google Gemini 3.0 Flash. It receives the word-level transcript with timestamps together with PySceneDetect scene boundaries, and returns 3 to 15 segments of 15 to 60 seconds scored on hook strength, emotional payload and whether the segment stands alone without surrounding context.
- How does the automatic vertical cropping work?
- Two modes. TRACK mode follows a single subject using MediaPipe face detection with a YOLOv8 fallback, stabilised so the crop holds still inside a safe zone instead of following every movement. GENERAL mode handles group shots and landscapes by preserving the full width over a blurred backdrop.
- Can it dub clips into other languages?
- Yes, into more than 30 languages via ElevenLabs, preserving the original speaker's voice characteristics. The dubbed audio is then re-transcribed so the burned-in subtitles match the new language rather than the original.