Over the last several episodes of Talking Product, Collin and I have been exploring how we increase the probability that our AI investment will translate into business value.

Recent news has shown how organisations are having to confront the economics of AI more directly: token budgets being consumed faster than expected, tighter controls on usage, the challenges of moving from prototype to production, how to shape experiments around coherent hypotheses tied to a business outcome, and whether the use of GenAI is actually translating into customer value.

In Episode 15, we explore the mindset shift that I have experienced through working with the DSPy framework and becoming familiar with the work of Omar Khattab. This is not a technical discussion. Instead, Collin and I continue to explore the role non-technical senior leaders should play in shaping the AI journey.

I believe anyone in an AI leadership role should familiarise themselves with Omar’s work if they haven’t done so already. I have found him to be a grounded, coherent voice in a world inundated by hyperbole and breathlessness.

Omar Khattab is an Assistant Professor at MIT. His doctorate is from Stanford. His research focuses on how we program, evaluate and optimise systems built around language models. He created DSPy, an open-source framework for building and optimising LLM applications programmatically, rather than relying primarily on manually engineered prompts.

In this episode, Collin and I explore:

• What belongs to LLM inference and what belongs to good ol’ deterministic programming.
• Defining tight, measurable evaluations — yes, we’ve been banging on about this since covering Chip Huyen’s book.
• The power of optimisation — don’t just believe us, go and look at the use cases on the DSPy website.
• Building on that optimisation, how lower-cost models can sometimes deliver equal and even better performance.
• The role non-technical senior leaders should play in shaping the AI journey.

If you like the session, please subscribe through your preferred podcast service and leave us a smashing five-star review. It helps us build an audience and attract guests to the show.

And, as always, feedback is more than welcome.

Thanks.

Links referred to in the show:

Omar's home page: https://omarkhattab.com/

Two podcasts:

How Foundation Models Evolved: A PhD Journey Through AI's Breakthrough Era: https://podcasts.apple.com/us/podcast/how-foundation-models-evolved-a-phd-journey-through/id1740178076?i=1000737229448

DSPy: Transforming Language Model Calls into Smart Pipelines // Omar Khattab: https://podcasts.apple.com/gb/podcast/dspy-transforming-language-model-calls-into-smart-pipelines/id1505372978?i=1000637537521

I also found this post helpful: https://x.com/lateinteraction/status/1921565300690149759