AI video clippers SUCK. Build your own.
Anyone can write their own software and agentic workflows. Reverse engineering tools with AI is not enough. The true edge comes from understanding and solving your unique problems.
Here's how I save 15+ hours a week.
I livestream on X. Editing clips and shorts from livestreams is unglamorous and painstaking, but that's where most views come from. People wonder how I publish so much video content consistently... so here it is. I hope it gives you insights : )
My 30+ agent system doesn't give me choppy, awkward, disjointed clips and shorts. This is the full breakdown of how it works and gets results, with a real run in Claude Code CLI and local models, sped up on the screen recording below.
I built a career in video editing, so I know what a good cut looks like. I also know what a proper workflow and deliverables look like. While my video editing agents churn, I do other work or go for a walk.
And if this looks "complex"... you should try manually editing hours of clips week after week in Davinci or Capcut lmaooo
WHAT IT DOES
I stream live on X weekly through OBS and the input for this run (shown below) was a 2 hour 26 minute stream. The output was a published episode: a corrected transcript (JSON), chapters, shownotes (a Markdown page), 15 clips (MP4 video files) cut and captioned and uploaded to Google Drive, 5 of them queued on YouTube as private videos, and a page live on my site. The run took about 2 hours. 15+ hours down to a few minutes of decision making.
So, I type one command into Claude Code (a custom /show-wrap skill). I answer two questions at the start: is there a guest, and is there anything I want the clip search to look for. In the middle I pick the clips and sign off on the cut plan. The rest runs on its own.
A push notification API hooked into Claude pings my devices whenever it needs me so I can get on with my life and other work without babysitting my machine.
HOW IT WORKS
Seven stages, numbered 0 to 6, plus a final check. Two kinds of agentic workers do the job.
AI agents (separate model calls, each with one narrow job) do the judgment work: reading, proposing, scoring, writing. Code does everything that has to be exact: timestamps, vote counting, cutting, uploading, checking.
Every stage saves a checkpoint (JSON). If a run is interrupted it resumes where it stopped and remembers my two answers. On resume it re-checks each finished stage's output files, by hash (a fingerprint of the file's exact contents) wherever the stage recorded one (agents can hallucinate).
STAGE 0 → Capture and normalize
An agent pulls the replay from X creator studio via headless Playwright browser (Chrome). The main session then fingerprints the file and records it.
Next the file is normalized (re-encoded so the audio and video timing match exactly). A raw X livestream capture has tiny gaps in its audio packets that collapse when the file is decoded, so the transcript slowly drifts against the video.
STAGE 1 → Transcribe
WhisperX (an open-source speech-to-text system) runs locally with the large-v3 model, in the background. A vocabulary of names and terms from earlier shows is fed in so the model hears them right (self learning).
A check confirms the output belongs to this exact file, contains real segments, and has its manifest (the run's record of how it was made). Meanwhile the normalized master is archived to Google Drive.
STAGE 2 → Correct transcripts
Auto AI transcripts mangle names, even from good AI models. Nothing here is edited in place. Every change goes through a ledger (a written record of each fix).
This stream had 1,223 transcript segments and 482 distinct proper nouns. Each one is listed with how often it appears, and each needs a decision: correct it, or confirm it as is. The ledger changed 9 segments.
A second check compares the transcript against every name I have ever published on the show, and flags near misses and known misspellings that survived. A validator agent compares corrected against raw, looking for dropped substance and changes that altered meaning. It gets three passes in total. After that the run stops and asks me, as an escape hatch.
The corrected file is stamped with its correction map, and later stages refuse any file without it, so the raw version can never flow downstream by accident.
STAGE 3 → Topics & chapters
A "topic" is a stretch of the show where I cover one subject, and it becomes a HH:MM:SS chapter in the shownotes. This is the hardest step to get right with AI.
A live show wanders: I read a post, riff on it for ten minutes, drift to something else, and come back later. A single model pass over 2.5 hours tends to lose the middle, and two readers rarely agree on exactly where a topic starts.
So five boundary-finder agents each read the whole transcript independently and propose where topics start. Code groups proposals that land within 17.5 seconds of each other, and a boundary needs 4 of the 5 to count. Three more calls then find where each chapter ends cleanly.
Agents can lie, so how do I know an agent read all 2.5 hours? The transcript goes to it as a file with unique markers woven through it, and it has to list the markers back in order. An agent that read the first page and the last cannot produce them. A response whose markers don't reconcile is thrown out.
Each answer comes back through a relay (a step that carries it from the agent to the files) that stamps a hash on it, and the collector rejects anything hand-placed or edited.
If one agent fails, it gets one retry on an identical prompt. If two fail, the run hard-stops. Two failing the same way means something is wrong, and re-running an unchanged prompt burns calls that cannot succeed.
Chapters are checked before the shownotes render: at least three, the first at zero, at least 10 seconds apart, and the last one starting past the 80 percent mark.
STAGE 4 → Find the clips
Three generator agents each read the whole show and propose clips. They quote the exact words where a moment starts and ends, and code finds the time in the word-level transcript. A quote that doesn't resolve is discarded with a reason.
Code merges the three lists. Two proposals count as the same moment only if they overlap in time and their titles share words.
Three scorer agents then rate each candidate high, medium, or drop and code settles it: "keep" versus "drop" is decided first, a tie goes to "keep", and the tier is the majority among the keeps.
A researcher agent looks up real X usernames and writes a grounded one-line summary for each clip. Another agent writes a summary of the whole show and a list of every brand and entity mentioned (web searching). One last agent checks any clip that only one run proposed, to see whether it cuts away just before its payoff (important for vibes).
On this stream: 27 candidates scored, 9 high, 17 medium, 1 dropped, 26 shown to me.
STAGE 5 → My gate (push notification)
This is the part I do (faster than reviewing with a human editor). The pool is grouped by tier, with timestamps, vote counts, entities with handles, and a summary for each clip. If the scorers dropped a clip that at least one run liked, it is shown too, so the disagreement is mine to settle.
I approve and the system tells me exactly which candidates I skipped, so I know what I left out. For YouTube I go through the stream in four equal quarters and pick from each. Easy peasy.
Then I see a dry run of the cut plan: how each clip will be treated, what will be removed, and a sample frame of the vertical crop. It states plainly what happens next: files cut, uploaded to my Drive, inserted as private YouTube videos, and an entry pushed to my site. My approval is tied to a hash of the exact approved clip list.
The cut, the Drive clip upload, the Drive shownotes upload, and the YouTube insert each re-hash that list when they run and refuse if it doesn't match. Change one clip after I approve and the approval is void. Re-rendering an approved clip keeps my approval, and changing the set of clips does not.
This stream: I approved 15 of the 26 and picked 5 for YouTube.
STAGE 6 → Cut, caption, publish (hard part)
Each approved clip is routed by length: Short (a vertical 9:16 version only, with burned-in captions, under 2mins), Clip (over 2mins a landscape master plus a captioned vertical), Highlight (5mins and up, a paced render, with dead air tightened and off-topic stretches in the middle removed).
Pacing is conservative. A gap in the transcript only proposes a cut, and the audio waveform can veto it. If there is audible sound in more than 5 percent of the gap, any brief loud sound, or the gap is shorter than 0.12 seconds, the pause stays. Removing off-topic speech is deliberate and produces its own list of every removal for me to review. Everything is loudness-normalized in two passes to -14 LUFS (a common streaming loudness level).
Captions run two to three words at a time with the active word highlighted, burned into the vertical version only. If captioning fails, the clip ships without them and the reason is recorded.
Four automatic check agents run on every cut before upload. One looks for words cut in half at the start or end, or a clip that opens on dead air. One measures audio levels, splices, and dead air. One reports what pacing removed and whether captions landed. One checks each file's audio timing. A clip that fails the first or last check is held back alone with its file kept, and the rest ship.
This stream: 15 approved clips became 16 files. 13 ran 2 minutes or longer, 1 was a short, and 1 ran just under 2 minutes. The short and the under-2-minute clip are the 2 captioned vertical versions, which is why 15 clips make 16 files.
Each file goes to Google Drive as an upsert (create, or replace the existing file), so a re-render replaces and never duplicates.
Before the shownotes page and the YouTube copy go out, the run looks up every username on X. This stream checked 27 handles. Each run adds to a ledger of known handles that is now over 300 (saves future work.
For the 5 YouTube picks, three writer agents each draft chapters from the clip's own transcript, code merges them and fixes the timing, and a fourth agent writes the title, description, and tags from a written style guide for my voice. I read the copy before anything is inserted. Everything goes in as a private video, and making one public is my step in YouTube Studio.
Last, the website entry. It is built from the checkpoint alone: runtime, a frame from the show, and the guest field from my first question. The site's own checks run, the entry is pushed, and the run waits until the page is confirmed live.
THE FINAL CHECK
Before the commit, an automatic text check runs over everything the run wrote. Then a fact reviewer agent re-fetches each claim in the shownotes from its source, not from memory. After a fix it gets two more tries, and a third failure holds the commit.
A publication audit then compares every link on the published page to what is live on Drive. For this stream: 0 problems across 16 published links and 16 live clips.
...AND IT'S DONE
I get another ping on my phone.
I have a Google Drive folder with all the clips ready to post, chapters and shownotes, transcript, my website is updated, the full recording is there, my website is updated, videos are ready to publish on my YouTube creator dashboard and I can go about my day.
CONSTANT EVALUATION
After each session, I have agents review the runs of previous streams to diagnose issues, harden the system and update dependancies or A/B different methods. I will soon be incorporating TypeSafe Jev for faster decision making on timestamped transcript work, possibly saving an hour of agent work and hundreds of thousands of expensive tokens.
That is what I mean by evaluation. Every stage has a check, and every check has a named reader: code that blocks the run, or me. In my design, a result that nobody acts on counts as a defect.
Each run also leaves the next one with more. The handle ledger, the names vocabulary, and the list of names I have published all carry over, and every run reads them before it starts.
THANKS FOR WATCHING
A few of you have said you want a system like this for your own work. I would build it the way I built mine: start from how you actually work, then check every step. That is why it has to be bespoke.
DM me here or reach me through my site : )
Below I'll attach the Google Drive folder with everything shown from the episode and more. A big shoutout to all my viewers and supporters, it's amazing what you can do with AI but the key is...
Know what problem you are trying to solve!
See the system in action below...