A company Jeff Patton worked with this year shipped a glossy new release to 2.65 million customers. AI helped them get it out on time. It didn't break. There were no bugs. Nobody liked it.

Jeff, author of User Story Mapping, joins me to talk about dual track development. He says the book needs a second edition: it described a discovery team and a delivery team, but after 10 years of coaching he sees two tracks and one team, with engineers and testers pulled up into discovery.

Jeff asks the whole team, on a scale of 1 to 5, how confident they are that people will see a feature and want to try it. If one says 3 and another says 5, ask what they'd need to learn to be sure. Keep using is the hard one to test: release something usable to a small group, and in Jeff's words, "don't build it at scale."

We also get into release to learn versus release to earn, and what I've seen in client GitHub repos since they adopted AI: a lot more output, but the share of work going toward customer results is flat or down. Jeff on asking AI what customers think: "the AI is not your customer."

His homework: do an opportunity canvas for the feature you're working on right now. If you can't say who it's for, what problem it solves and how you'll measure it, stop building it.

Chapters

00:00 A release customers did not like

00:21 Meet Jeff Patton

04:54 Discovery and delivery together

07:33 Two tracks within one team

09:46 Expose the beliefs behind a feature

12:52 Where does the team's confidence differ?

13:58 A usable release that teaches you something

16:03 Learning about technical risk

17:16 Ship to learn, earn, or burn

23:24 AI output and customer outcomes

27:40 AI opinion and actual customer evidence

33:18 An exercise for the next feature

34:46 Where to learn more from Jeff

Find Jeff: https://jpattonassociates.com/

How much of the feedback on a release actually needs to block it?

David Rasch looks for the checks his team can automate so nobody has to keep thinking about them. If an agent repeatedly makes the same mistake, a deterministic check can catch it before a person spends another review pointing it out. That leaves the person reviewing the work to decide what matters.

We talk about where to spend that judgment: the data model, changes to the interface, and the parts of a system that need careful design. Code may be easier to replace now, but someone still has to understand what the software is supposed to do.

David gives an example of an AI review getting caught up in a race condition for a task that runs about once a month. In that case, he decided the extra complexity wasn't worth it. I share an experiment where repeatedly responding to review feedback left my project unable to do anything at all.

The useful question is what needs fixing before this release, and what can become a separate ticket whose priority you decide later. A patient reviewer can always find more work. You still need a person who can decide when the work is worth doing.

Useful questions from the conversation

Which repeated review comments could become automatic checks?

Which parts of this change need someone to understand the design?

Does this issue need to stop the release, or can you decide its priority separately?

Chapters

00:00 When AI agrees with everything

00:12 Meet David Rasch

02:12 What you still need to understand

05:02 Data foundations before more code

10:00 Building and selling as an engineering education

14:42 Which parts need careful design?

19:12 Automating repeatable review checks

21:20 Code is easier to rework

22:31 When review becomes a loop

25:02 Human judgment and work worth doing

28:55 Where to find David

Find David: https://www.linkedin.com/in/rasch

Listen to Episode 36 of Build & Break Through: https://buildandbreakthrough.riverside.com/e/making-ai-useful-with-david-rasch

What would help your customers more: another round of faster AI output, or a better way to see what you're building?

Josh Colter, the majority owner of Cardboard, has put a lot of thought into that question. He bought the company with a bet on AI and has been combining story maps with detailed instructions so he can see the big picture while the models work through the details.

His comparison comes from the Wright brothers. He says "they thought about control first." A bigger engine is useful once you can steer. Josh applies that idea to building software: make the work visible, understand the customer behavior you're trying to change, and keep releases small enough to inspect.

We talk about the difference between having AI drive and learning to operate it yourself, why a long specification can hide something a human needs to see, and how Josh planned and shipped dark mode in three days after finding years of requests from users of a competing product.

Useful questions from the conversation

Can you see the work clearly enough to tell where it's going?

Can the people building it understand the plan as readily as the AI can?

What customer request are you trying to meet, and how will you check the result?

Chapters

00:00 Control before thrust

00:45 Show intro

00:53 Meet Josh Colter

01:26 Buying Cardboard and betting on AI

02:46 Chauffeurs and airmen

08:47 Why a bigger engine needs control

11:05 Make the work visible

15:20 Story maps, specifications and human understanding

20:32 Planning and shipping dark mode

25:26 Checking what you've actually built

26:35 Where to find Josh and Cardboard