Aaron Hoffman · Design-led AI Product Builder
Aaron Hoffman · Design-led AI Product Builder
Case Study · The Steadcast Knowledge Engine
No babysitter. No five-figure surprise bill. No hallucinated quotes.
A lot of "production AI" in 2026 is a demo with a Stripe button on it. This is what it looks like when it's built to run on its own: the Steadcast Knowledge Engine, the AI infrastructure behind a podcast app that's live in the Mac App Store today.
The Job
Transcripts, per-episode highlights, section outlines, framings, a weekly newsletter. At catalog scale, across a hundred-plus shows, without a human reviewing every output and without burning a budget.
The hard part isn't getting a model to produce something. It's getting a system to produce the right thing every time, catch itself when it doesn't, and keep costs boring.
Here is one pass through the pipeline, end to end.
Bad drafts never reach a reader. They never even reach storage: the gate runs before anything persists, and failures go back to the models with specific feedback about what to fix.
The Output
Everything below is live at steadcast.co right now, produced and published by the pipeline above — including the magazine front page in the header, which nobody laid out. No human reviewed these pages before they shipped; the eval gate did. See it live.
Highlights & Transcript
Each episode page leads with highlighted moments, then the full timestamped transcript. Every quote is verified word-for-word against the source audio, and every "jump to" timestamp snaps to a real transcript segment. There is no path for a hallucinated quote or an invented timecode to reach this page.
Per-Episode Pages
Show notes, a one-line framing, highlights, the full transcript, and a one-click path into the Mac app, generated for each of a hundred-plus shows without anyone touching a CMS. The same routing and eval discipline runs on every single one.
The Weekly Newsletter
Each week the system composes a newsletter issue wrapping a curated list, runs it through the eval gate before it can persist, and sends it to subscribers. Same voice rules, same banned-pattern checks, same cost ceiling as everything else. Browse the issues.
The Discipline
Transcription jobs sit in a database-backed queue that a self-hosted GPU worker pulls from safely: no double-processing, no lost work. The worker sends heartbeats. If the home GPU drops, a cloud fallback notices within five minutes and takes over automatically. Nobody gets paged, because nothing needs a person.
Every piece of editorial prose passes an automated eval suite before it's allowed to touch the database: deterministic checks (57 banned hype phrases, 14 banned openers, length, structure) plus an LLM-judge pass scored against a threshold that ratchets up every quarter.
The editorial voice lives in 410 structured, machine-readable rules built from the publication's own corpus and reference publications. When the voice guide changes, a codegen check keeps the enforcement rules in sync, so the guide and the gate can't drift apart.
Structured extraction runs on one model. Prose runs on another, because testing showed the first one couldn't satisfy the voice constraints. That routing layer also means no single-vendor lock: when a model gets deprecated, the slot gets re-pointed, not rebuilt.
Quoted "highlight" moments are checked word-for-word against the transcript, and section timestamps are snapped to real transcript boundaries. Zero hallucinated quotes, zero invented timestamps, by construction rather than by hope.
Per-job budget caps, monitoring, and a rule that nothing is called shipped until it has run clean in production for a week. The first viral moment should be good news, not a five-figure bill.
The Numbers
After the gate went in, the expanded checks produced zero false positives across five consecutive fresh outputs spanning five genres, each clean on the first attempt. A pre-gate catalog audit had found voice violations in 3.3% of existing content; new violations are now blocked before they persist.
Why This Matters To You
If you're a founder or operator being pitched "production-ready" AI right now, ask the person you're talking to four questions:
If the answers are vague, you're buying a demo. I built this system end to end: design, infrastructure, eval discipline, and the product around it. Steadcast was accepted into the Mac App Store, a third-party production filter most AI work never faces.
A one-week audit, a production-grade build, or ongoing fractional product leadership. Pricing is on the page, not behind a call.
See how we can work togetheror email hello@doubleabattery.com