"Explainable AI", aka XAI, refers to a suite of techniques to help AI system developers and AI system users understand why inputs to the system resulted in the observed outputs.

Industries such as healthcare, education, and finance require that any system using mathematical models or algorithms to influence the lives of others is transparent and explainable.

In this episode, Jake and David review what XAI is, classical techniques in XAI, and the burgeoning area of XAI techniques specific to LLM-driven systems.

Prompt engineering involves a lot more than simply getting smarter with how you structure the prompts you enter in an LLM browser interface.

Furthermore, a growing body of peer-reviewed research provides us with best practices to improve the accuracy and reliability of LLM outputs for the specific tasks we build systems around.

In this episode, Jake and David review evidence-based best practices for prompt engineering and, importantly, highlight what proper prompt engineering requires such that most of us likely cannot call ourselves prompt engineers.

Lots of people like to talk about the importance of prompts, context, and what is sent to an LLM. Few discuss the even more important aspect of an LLM-driven system in evaluating its output.

In this episode, we discuss traditional and modern metrics used to evaluate LLM outputs. And, we review the common frameworks for obtaining that feedback.

Though evals are a lot of work (and easy to do poorly), those building (or buying) LLM-driven systems should be transparent about their process and the current state of their eval framework.