AI Clips My Livestream: 3 Hours In, 6 Shorts Out

July 12, 2026 · Türkçe oku

I Streamed for Three Hours, Then Couldn’t Face the Editing

Where are the best 30 seconds of a three-hour livestream? To find them you have to rewatch the whole thing, take notes, then crop every cut to vertical and burn in captions. I did this by hand exactly once: 3 hours of streaming, 2.5 hours of clipping. I skipped it on the second stream. And the third. Streams piled up, the Shorts channel stayed empty.

The problem was never a lack of ideas — it was friction. The last thing you want after a stream is to rewatch three hours of yourself.

So I built a pipeline that watches the stream, picks the viral moments itself, reframes them to vertical, burns in captions, and schedules them to YouTube. I no longer touch any of it: the stream ends, and the next day Shorts queue up. Here’s the full architecture, the scoring rubric I use (copy-paste ready), and the three lessons I learned running it live.

The Five Stages of the Pipeline

This isn’t one magic model — it’s five chained steps, each eating the previous one’s output:

1. Mic check (pre-screen). Three audio samples are pulled from the recording (at the quarter, half and three-quarter marks) and run through a small speech model to ask: is anyone actually talking here? A stream where the mic stayed muted, or that was just background music, gets rejected before it’s even downloaded. Without this step the system happily tries to process three hours of silence.
2. Transcription (word-level timestamps). The whole stream is transcribed with timestamps per word, not per sentence — karaoke captions are impossible otherwise.
3. Viral moment selection (LLM + rubric). The transcript is split into overlapping windows, each window goes to a language model, and every candidate cut is scored against a 9-signal rubric. Only cuts above the threshold become clips.
4. Vertical composition. Each candidate becomes a 1080×1920 vertical video: webcam box on top, the active region of the screen zoomed in below, word-by-word karaoke captions in the middle. The webcam box and the active region are found automatically via face detection plus frame-difference analysis.
5. Scheduling and publishing. Clips that pass QC are uploaded to YouTube as private with a future publish time — one clip per weekday at 12:00. If I don’t like one, I can cancel it in Studio before it goes live: the automation runs, but the veto stays with me.

Real Numbers: What Does One Stream Cost?

The obvious question: isn’t feeding a three-hour transcript to a language model expensive? Here are the numbers straight out of the engine’s own summary output for two real runs:

So having AI read an entire stream, score it, and refine the clip boundaries costs less than half a dollar. Compared to the hours it saves, the model cost doesn’t even register. The expensive resource was never the model — it was your three hours.

The Secret Isn’t the Model, It’s the Rubric

Ask a model to “find viral clips in this stream” and it hands you a summary of the conversation, not a viral moment. The difference lies in describing what you’re looking for numerically. The engine’s rubric has 9 signals, each worth 0-2 points, 18 max. Nothing below the threshold ever becomes a clip.

Copy it, adapt it to your content:

SCORING RUBRIC — each signal is worth 0, 1 or 2 points (18 max). Threshold: 13.

1. SINGLE CONCEPT   — 2 = one clear idea | 0 = topics tangled, jumps with "anyway"
2. TWEETABLE LINE   — 2 = a line you could rip out and tweet standalone
3. MINI-ARC         — 2 = problem→solution arc closes on itself | 0 = starts mid-thought
4. HOOK STRENGTH    — 2 = the first 1-2 sentences ALONE stop the swipe (claim/question/number)
5. VISUAL SUPPORT   — 2 = there's a demo ("let me open this", "you can see here" = demo signal)
6. AROUSAL / EMOTION— 2 = surprise, strong claim, contrarian take (high arousal = shares)
7. STANDALONE CLARITY — MOST CRITICAL. 2 = 100% understandable ripped out of context
                        0 = STARTS with "this/that/as I said/like I mentioned"
8. FITS THE FORMAT  — 2 = naturally 20-50 seconds (~50-160 words), single topic
9. CTA CONNECTABLE  — 2 = topic connects to the product/community without forcing it

If nothing clears the threshold, return an empty list. Don't force it.

The signal I argued with myself about most was #7. Without “standalone clarity,” the model keeps picking cuts you loved live but that open with “and that’s exactly why…”. A viewer sees that in their feed, has no context, and swipes in 1.5 seconds. That’s why a cut starting with a back-reference can never score full marks — the rule is baked into the engine.

”Clip This Part” — Marking Moments Out Loud, Mid-Stream

The rubric is good, but it can’t catch everything. Sometimes I’m explaining something on stream and I know, right then, that it’s a clip. The model might score it 12 and drop it; I’d score it 20.

The fix turned out to be simple: I mark it out loud — “you can clip this part,” “clip this one.” The engine searches the transcript for those phrases and turns whatever the command points at (usually the arc behind the command, with its natural start and end) into a forced clip. It doesn’t have to clear the rubric.

One small but critical detail: the clip must end before the command sentence — the viewer should never hear me saying “clip this.” The engine handles that with a separate boundary-refinement pass.

The general lesson: leave a channel for human marking inside your automation. Instead of chasing full autonomy, the thing that improved quality most was an interface where a human can cheaply tell the system what they already know — here, just speaking it out loud.

Three Lessons From Running It Live

1. A pipeline with no pre-screen will happily process garbage. I added the mic check after the fact. Before that, the system downloaded and transcribed an entire stream where the microphone had stayed muted — and the speech model hallucinated words out of the silence. Watching an AI try to pick viral clips from invented sentences is not fun. Put a “is this input even worth processing?” gate at the front of every pipeline.

2. Automation will fill your disk. A three-hour 1080p stream download is not small. The pipeline deletes the raw video once the clips are cut — that’s not an optimization, it’s a requirement. Otherwise the disk fills up after a few streams and the pipeline dies quietly.

3. Auto-publish ≠ uncontrolled publish. Clips don’t go straight to public: they’re uploaded private with a future publish time, and I can kill any of them inside that window. Every point where your automation becomes irreversible is a point that will wake you up in a panic later.

What You Can Do Today — Checklist

If you want to automate your own clips, in order:

• Transcribe your recording with word-level timestamps (sentence-level output won’t do — karaoke captions need words).
• Split the transcript into overlapping 10-15 minute windows (the overlap is what stops a clip from getting lost on a window boundary).
• Embed the 9-signal rubric above in your prompt, state the threshold, and score at temperature 0.
• Refine boundaries for surviving cuts in a separate call (“don’t start or end mid-sentence”).
• Do the vertical composition with ffmpeg: camera on top, zoomed active screen region below, karaoke captions in the middle.
• Schedule as private + future publish time, and keep a veto window for yourself.
• Pre-screen at the front of the pipeline, disk cleanup at the back.

Where This Pipeline Plugs In

This clip engine isn’t a standalone toy — it’s one of the things my AI agent team built. I wrote about how I run that team on autopilot with sprints in running an AI agent team on autopilot. And the biggest problem that team had — starting from zero every session — I tackled in giving your AI agent persistent memory.

One email a week: I share this pipeline’s code, prompts and failures — the automations that work and the ones that die — once a week. Subscribe with the form at the end of this page; unsubscribe in one click.

Want to build your own clip engine? I share the rubric, the ffmpeg composition settings and the scheduling flow verbatim in the newsletter — copy it, adapt it to your channel. Subscribe with the form below.

Conclusion

Clipping a stream is no longer a decision for me — it’s a background process. What made that possible wasn’t a better model; it was a rubric that describes numerically what I’m looking for, a pre-screen gate that makes failure cheap, and a veto window where a human can still step in. That’s usually the pattern that works in AI automation: don’t assume the model is smart — write the criteria yourself.

Author: Murat Baskıcıoğlu