Host Ran Aroussi (Old School / New Tech) shares a recent “war story” from his agency Automaze: a hard-to-reproduce mobile bug where calls were sometimes dropped only on a client’s devices, which the team couldn’t reproduce for 2–3 weeks. After two long sessions using a Droid Factory setup with agents and a “small council” debate between models (e.g., Fable and Sol), he got the team unstuck, found the issue, and then turned the session logs into an internal guide and public article on his methodology. Key practices include distrusting agent conclusions (treat “preexisting/flaky/unrelated” as hypotheses), demanding proof (including his Proof library requiring video evidence), using extensive unit and end-to-end tests, asking for a numeric confidence level before production, controlling environments, handling merge conflicts with full review/testing, switching models for independent review, and re-spec’ing from scratch to detect drift from the implementation plan.

Why Software Factories Still Need Humans in the Loop

Ran Aroussi discusses “software factories” as automated systems that manage the full software development and deployment cycle, arguing the industry is moving from a single workstation to virtual, always-on cloud environments with agents running off-machine. Prompted by Dex (HumanLayer) describing a fully automated “lights-out” factory that produced “slop,” and Uncle Bob’s claim he doesn’t read agent-written code when surrounded by extreme constraints, Ran explains why he doesn’t believe in lights-out automation. He outlines Automaze’s internal factory, Cloop: tickets/PRDs are created and grounded in an indexed knowledge graph of the codebase; multiple models generate and critique PRDs; a human approves; agents implement in parallel, run unit/e2e/quality tests, simplify and profile code, then perform code and security review, looping up to five times before escalating to a human. Humans remain essential for alignment, enforcement, and final validation via PR review and temporary deployments, avoiding false positives, runaway costs, and untrustworthy results.

---

My new book, "Company-Scale Agentic AI: The operator's guide to a company that runs on intelligence", is out.
Amazon link: https://www.amazon.com/dp/B0HCDKK79L

Company-Scale Agentic AI: Why AI Isn’t Stupid, It’s Blind (and How to Fix It)

In this episode of Old School / New Tech, Ran Aroussi explains what “company-scale agentic AI” really requires and why most AI rollouts fail: the model isn’t stupid, it’s blind to how the business actually operates. He shares why he wrote a new executive-focused book, Company-Scale Agentic AI (a free follow-up to Production Grade Agentic AI), and outlines a five-part loop—observe, understand, build, run, compound—built around an always-on “company brain” that absorbs emails, chats, calls, and files to map processes, relationships, and bottlenecks. Ran emphasizes the difference between deterministic automations and true agents, argues for starting with human-gated execution to build trust, and highlights role-based access control via middleware as essential for organization-wide deployments. He also describes a browser “morning brief” workflow that keeps tasks from falling through the cracks and urges teams to adapt AI to existing tools instead of forcing employees to change how they work.

00:00 Welcome and Topic
00:29 Why I Wrote It
02:20 AI Is Blind
06:45 Building Company Brain
07:28 The Five Step Loop
11:15 Automation vs Agents
13:53 Gated Readiness Dial
17:01 Role Based Access
20:58 Morning Brief Extension
22:34 Meet People Where They Work
24:57 Loop Recap and Wrap

-----

This podcast is sponsored by Automaze, the fractional CTO partner for founders and operators. Whether you’re building a high-tech MVP or modernizing internal ops with AI and automation, Automaze can help you scale without the overhead of a full-time team.

Learn more: automaze.io