Voice AI agent evaluation — why it's fundamentally harder than text, how cascade failures derail conversations invisibly, and ServiceNow's open-source framework to establish industry evaluation standards. Featuring real audio examples showing authentication failures, leaked reasoning, and latency problems.

WHAT WE COVER

TARA BOGAVELLI — Research Engineer, ServiceNow
Leading the open-source voice agent evaluation framework. Explains why existing benchmarks don't measure what matters and what ServiceNow is releasing to establish industry standards.

KATRINA STANKIEWICZ — Staff Machine Learning Engineer, ServiceNow
Cascade model architecture expert. Breaks down STT → LLM → TTS failure modes, named entity transcription challenges, and real audio example analysis.

GABRIELLE GAUTHIER MELANÇON — Staff Applied Research Scientist, ServiceNow
Multi-language evaluation specialist. Reveals why Large Audio Language Models lag behind, the native speaker requirement, and bot-to-bot simulation methodology.

CHAPTERS
0:00 Introduction — The evaluation gap
1:11 ServiceNow's Open-Source Framework Announcement — Tara Bogavelli
2:43 Meet the Researchers
3:43 Voice-Specific Challenges — Tara Bogavelli
5:03 Cascade Architecture: STT → LLM → TTS — Katrina Stankiewicz
7:57 The Named Entity Problem — Katrina Stankiewicz
10:06 Evaluation Metrics: Accuracy vs Experience — Gabrielle Gauthier Melançon
11:23 Bot-to-Bot Testing at Scale — Gabrielle Gauthier Melançon
14:30 The LALM Gap: Why Audio AI Judges Struggle — Tara Bogavelli
16:57 Real Audio Example: Flight Rebooking Gone Wrong
21:58 Breaking Down the Failures — Katrina Stankiewicz 28:30 Wrap-Up & Resources

KEY INSIGHTS

The Cascade Failure Problem: STT → LLM → TTS errors propagate invisibly Named Entity Transcription: The #1 enterprise blocker—names, confirmation codes, emails break authentication Accuracy vs Experience: Perfect task completion means nothing if users hang up due to poor experience LALM Gap: Large Audio Language Models lag behind text LLMs—human evaluators remain essential Latency Kills Conversations: Five-second pauses make users think the call dropped, breaking the experience even when tasks complete Open-Source Framework: ServiceNow releasing evaluation tools, metrics, and bot-to-bot simulation methodology for the industry.

LEARN MORE

Website: https://servicenow.github.io/eva/ GitHub:
https://github.com/servicenow/eva Blog Post:
https://huggingface.co/blog/ServiceNow-AI/eva Dataset: https://huggingface.co/datasets/ServiceNow-AI/eva

ABOUT

Hosted by Bobby Brill. ServiceNow Insights podcast explores AI research, real-world applications, and the people building the future of work. #VoiceAI #AIEvaluation #ServiceNow #MachineLearning #OpenSource #ConversationalAI #STT #TTS #LLM #VoiceAgents #AIResearch #Podcast

See omnystudio.com/listener for privacy information.

AI governance at scale — what it means, how to do it, and what regulations you need to know now. Host Bobby Brill brings together five ServiceNow experts across two conversations for a complete 20-minute briefing on governing AI in the enterprise.

━━━━━━━━━━━━━━━━━━━━━━━━
WHAT WE COVER
━━━━━━━━━━━━━━━━━━━━━━━━

RAVI KRISHNAMURTHY — VP, AI Platform, ServiceNow
Why hidden AI is one of the biggest unmanaged risks in the enterprise — and why governance is an accelerator, not a brake.

PETER WEIGT — Responsible AI, ServiceNow
The innovation paradox: how AI Control Tower makes governance a team sport and breaks down the silos that slow AI deployment down.

SAMPADA CHAVAN — AI Control Tower, ServiceNow
How AI Control Tower was built, what the discovery problem really looks like, and why compliance must be baked into the AI lifecycle — not bolted on at the end.

ANDREA LAFOUNTAIN — AI Legal, ServiceNow
The three regulatory frameworks every enterprise needs to know: EU AI Act, Colorado AI Act, and NIST. Plus: the compliance strategy that scales across all of them.

NAVDEEP GILL — Responsible AI, ServiceNow
The math on enterprise AI compliance — why it's exponential — and how AI Control Tower's automated discovery keeps you ahead of it.

━━━━━━━━━━━━━━━━━━━━━━━━
CHAPTERS
━━━━━━━━━━━━━━━━━━━━━━━━

0:00 Introduction
1:23 The Hidden AI Problem — Ravi Krishnamurthy & Sampada Chavan
5:33 AI Control Tower in Practice — Peter Weigt & Sampada Chavan
7:37 The Regulatory Landscape — Andrea LaFountain & Navdeep Gill
14:38 Compliance in Action & Key Deadlines
17:05 Wrap-Up

━━━━━━━━━━━━━━━━━━━━━━━━
KEY DATES TO KNOW
━━━━━━━━━━━━━━━━━━━━━━━━

EU AI Act enforcement: August 2026
Colorado AI Act enforcement: June 2026
NIST AI RMF: Voluntary framework, increasingly referenced by regulators

━━━━━━━━━━━━━━━━━━━━━━━━
LEARN MORE
━━━━━━━━━━━━━━━━━━━━━━━━

ServiceNow AI Control Tower: https://www.servicenow.com
NIST AI Risk Management Framework: https://www.nist.gov/artificial-intelligence

━━━━━━━━━━━━━━━━━━━━━━━━
ABOUT THIS PODCAST
━━━━━━━━━━━━━━━━━━━━━━━━

Hosted by Bobby Brill. A ServiceNow podcast exploring the people, technology, and ideas shaping the future of work.

#AIGovernance #ServiceNow #AIControlTower #ResponsibleAI #EUAIAct #EnterpriseAI #AICompliance #FutureOfWork #NowAssist #Podcast

See omnystudio.com/listener for privacy information.

What is Agentic AI — and what can it actually do for your business? In this episode of the podcast, host Bobby Brill brings together three conversations with the people building Agentic AI at ServiceNow into one 20-minute briefing.

Whether you're leading a team, managing a platform, or just trying to understand what your company is investing in - this episode gives you the full picture.

━━━━━━━━━━━━━━━━━━━━━━━━

WHAT WE COVER

━━━━━━━━━━━━━━━━━━━━━━━━

RITA CASTILLO — VP of Design, Platform & AI

The clearest explainer of Agentic AI you'll find. Rita walks through the full evolution from scripted automation to agentic systems, what makes agentic different, and how the orchestrator-and-agent hierarchy works at ServiceNow.

AMRUTHA RAMESH — Visiting Researcher, ServiceNow

Amrutha built Agent Ada — a real AI agent designed to turn enterprise data into actionable insights. She explains how Ada understands your goal, adapts to who's asking, and delivers analysis your team can actually use.

ISSAM LARADJI & ANITHA RAGHAVAN — Research Scientist & Product Manager, ServiceNow

How do you know if your AI's insights are trustworthy? Issam and Anitha built InsightBench — a benchmark that uses AI to evaluate AI — to answer exactly that question.

━━━━━━━━━━━━━━━━━━━━━━━━

CHAPTERS

0:00 Introduction
1:30 What Is Agentic AI? — Rita Castillo, VP of AI Design
6:00 How Agents Work as a Team
10:30 Meet Agent Ada — Amrutha Ramesh, Visiting Researcher
16:00 Can You Trust Your AI? — Issam Laradji & Anitha Raghavan
21:30 Wrap-Up

━━━━━━━━━━━━━━━━━━━━━━━━

GUESTS

━━━━━━━━━━━━━━━━━━━━━━━━

Rita Castillo — VP, AI Design at ServiceNow

Amrutha Ramesh — Visiting Researcher, ServiceNow

Issam Laradji — Research Scientist, ServiceNow

Anitha Raghavan — Product Manager, ServiceNow

━━━━━━━━━━━━━━━━━━━━━━━━

LEARN MORE

━━━━━━━━━━━━━━━━━━━━━━━━

ServiceNow Agentic AI: https://www.servicenow.com

More episodes:

Narrative Analytics with Agent Ada: https://youtu.be/k1P9o2glq90

InsightBench & AgentPoirot: https://youtu.be/0_ikQQ82qAk

Understanding Agentic AI with Rita Castillo: https://youtu.be/JyFdcQChYLs

━━━━━━━━━━━━━━━━━━━━━━━━

ABOUT THIS PODCAST

━━━━━━━━━━━━━━━━━━━━━━━━

Hosted by Bobby Brill. ServiceNow INsights podcast explores the people, technology, and ideas shaping the future of work.

#AgenticAI #ServiceNow #AIAgents #ArtificialIntelligence #FutureOfWork #EnterpriseAI #DataAnalytics #NowAssist #MachineLearning #Podcast

See omnystudio.com/listener for privacy information.