Inside our resumable, review-gated video production pipeline

Our studio runs a local pipeline that turns a history topic into a finished, captioned 1080p video that waits for human review. Plain Python scripts handle the work, one per step, and a single command runs them all. We built it this way because each stage fails in its own way. Narration depends on a text-to-speech API, photos come from public archives that rate-limit, and editorial calls depend on a model. Keeping the steps separate lets us test, rerun, and fix each one without redoing the rest.

Seven steps run in order: script, narration, visuals, captions, render, description and thumbnail. Step one writes a script grounded in web search through a local agent CLI in one-shot mode, and our code assigns the segment ids rather than trusting the model to number them. Narration comes from ElevenLabs' timestamped endpoint, and we turn its character-level alignment into word-level timing. When a step exits cleanly, the orchestrator records it in a plain JSON state file. A killed run resumes where it stopped, and the start-at-step flag stops with an error if any earlier step isn't recorded.

Visuals are archival photos found by searching the Library of Congress and Wikimedia Commons. A model picks the best one or two photos for each keyword. Then a vision pass looks at the actual pixels of all of a section's candidates in one call, because a title or credit line can describe the wrong subject. Captions are sentence-aware groups of about five words, burned in with ffmpeg and libass. The render builds Ken Burns clips cut to the exact narration length. Nothing is uploaded: the finished video goes into a queue marked pending review.

Failure handling was the hardest decision. An early version replaced model review with deterministic heuristics, and later wrappers skipped the vision check without saying so when a call failed. We removed every timeout and fallback from those calls. A failed, empty or unparseable response now retries after 10, 30 and then 60 seconds, repeating indefinitely, and a heartbeat log line shows that long waits are still alive. Only a missing CLI binary raises an error. Network calls keep a limit: a 429 waits three times at most, then fails just that keyword.