AI spent the last few years competing on intelligence. Now the next major AI race is about speed.

As AI shifts from simple chatbots to agents that reason, write code, call tools, search the web, and coordinate other agents, latency compounds. A task that requires dozens or hundreds of sequential model calls can quickly turn seconds into minutes — or even hours. That makes inference speed a fundamentally different problem in the agentic AI era.

In this episode of So What About AI Agents?, Philippe Trounev sits down with Vasanth Mohan of SambaNova Systems to unpack what actually makes AI faster — from model architecture and memory bandwidth to batching, specialized accelerators, and next-generation inference hardware.

Vasanth explains the two major stages of inference — prefill and decode — and why they create very different hardware bottlenecks. They discuss operator fusion, parallelization across chips, memory bandwidth, and why reducing latency often comes with a significant cost tradeoff.

They also explore why coding agents are one of the clearest use cases for premium inference, how AI providers may eventually offer commodity, premium, and ultra-fast tiers of tokens, and why businesses willing to pay for speed may gain a meaningful productivity advantage.

Vasanth also shares early performance figures for SambaNova's upcoming SN50 architecture, including benchmark results around 800 tokens per second on a large model, compared with roughly 300–400 tokens per second on GPUs in the cited comparison.

We also get into a slightly crazier question: are AI agents already helping engineers design the next generation of AI hardware? The answer is increasingly yes — although humans are still very much in the loop.

In this episode:

• Why AI agents make inference speed dramatically more important
• Why sequential agent workflows create a latency bottleneck
• Prefill vs. decode explained
• GPUs vs. specialized AI accelerators
• The relationship between speed, batching, throughput, and cost
• Why memory bandwidth matters for large AI models
• What SambaNova's SN40 and SN50 architectures are designed to solve
• 800-token-per-second AI inference
• Why coding agents benefit so much from faster models
• Commodity vs. premium vs. ultra-fast AI inference
• Whether faster AI becomes a competitive advantage
• How AI agents are already being used in hardware engineering
• Why AI infrastructure may become just as important as the models themselves

Chapters

00:00 — The AI race is shifting from intelligence to speed
00:39 — Why AI agents suddenly need faster inference
02:38 — What actually makes an AI model run faster?
05:30 — When does ultra-fast inference matter?
08:08 — Does model architecture determine inference speed?
10:05 — How do you optimize AI from model to hardware?
12:44 — How fast can AI inference actually get?
15:28 — The sequential latency problem with AI agents
16:39 — Where faster AI creates the most value
18:30 — Will premium AI inference become a competitive advantage?
20:11 — What does AI inference actually cost?
23:10 — How many tokens can AI hardware generate?
24:46 — Building private AI infrastructure
28:02 — How much faster can AI eventually become?
30:14 — Are AI agents already designing AI hardware?
32:50 — Why today's “fast” AI will eventually feel slow
33:34 — What happens next in AI infrastructure

AI attackers can now move at a speed and scale that traditional security operations were never designed to handle.In this episode of **So What About AI Agents**, Philippe Trounev sits down with **Neal Iyer, Director of Product Management for AI at Splunk Security**, to talk about what happens when autonomous AI agents become part of both the attack and defense side of cybersecurity.Neal breaks the new threat landscape into three forces: **scale, speed, and sophistication**. AI can give relatively unsophisticated attackers capabilities that previously required much more expertise, while autonomous attacks can move so quickly that a security team celebrating a five-minute detection time may already be too late.We get into:• How AI changes the economics of cyber attacks• Why cheap AI agents can dramatically increase the number and sophistication of attacks• Why traditional SOC response times may no longer be fast enough• How prompt injection creates a new attack surface for defensive AI agents• Why simply deploying more security agents isn't enough• Building a security “harness” around autonomous agents• Runtime monitoring, guardrails, observability, and human escalation• When security agents should act autonomously — and when a human needs to stay in the loop• Using business context to distinguish between compromising an intern's laptop and locking out a CEO• Why security tools need better integration for agentic operations• How organizations can start testing their agents against prompt injection and other attacks today• Why Splunk and Cisco are exploring purpose-built small language models for cybersecurity• The role of open-source, self-hosted, and multi-model AI strategies• Why AI security economics may force companies to rethink how they store and process security dataWe also get into the broader question of whether frontier AI companies can really replace specialized enterprise software — or whether impressive demos fall apart when customers need reliability, support, governance, and measurable outcomes.The bigger takeaway is that the future SOC probably isn't humans versus AI attackers.It's **agents fighting agents — with humans designing the systems, permissions, guardrails, and escalation paths that keep those agents under control.**Subscribe to **So What About AI Agents** for conversations with the people actually building and deploying AI agents in production.https://www.docsie.io

In this episode of **So What About AI Agents**, we break down **10 AI agents and workflows we’ve used to drive traffic, generate demand, nurture leads, and automate parts of our go-to-market operation**.This episode was recorded a little while ago, so my own setup has evolved since then. Today I use **fewer, more capable agents with much more advanced workflows**, rather than trying to automate everything with a huge number of separate agents. But the core ideas in this conversation are still useful if you're trying to figure out where agents can actually create leverage in marketing and GTM.We cover agents for:• LinkedIn intent detection and outbound• Lead nurturing based on actual customer behavior• Automated newsletters• Social media content planning and distribution• Paid-ad creative and optimization• Turning videos into blogs, clips, and other content• Programmatic SEO and AI-search visibility• Building glossary and topic-cluster content at scale• Google Search Console and analytics-driven optimization• Cross-channel content repurposingWe also talk about what **doesn’t** work particularly well: generic cold email, blindly automating social media, expensive always-on AI employees, bad AI-generated content, and agents that cost more to operate than the value they create.The bigger lesson is that you probably don’t need one magical “AI employee” doing everything.You need a small number of well-defined agents with clear jobs, good data, specific triggers, and enough human oversight to keep them from doing something stupid.If you're building an AI-native marketing or GTM stack, this episode gives you a practical starting point for deciding what is actually worth automating — and what probably isn't.Subscribe to **So What About AI Agents** for conversations about how companies are actually building, deploying, and operating AI agents in the real world.https://www.docsie.io