Standard AI benchmarks measure speed and raw token throughput, but how do you measure whether artificial intelligence is actually completing enterprise work correctly?

In this episode of Utilizing AI, co-hosts Stephen Foskett and Brad Shimmin welcome Ryan Shrout, President and GM of Signal 65, to introduce "Pinnacle"—a new benchmark designed to evaluate real-world, agentic workflows.

Ryan breaks down why traditional hardware and model metrics miss the mark on business value, how Pinnacle evaluates performance across clean "governed" versus messy "as-found" data environments, and what recent test runs reveal about token efficiency, hallucination resistance, and hardware scaling.

From comparing frontier models to exposing the hidden costs of max-reasoning modes, this discussion offers a grounded look at quantifying true enterprise AI performance.

This and more on Utilizing AI, part of Futurum Media and the Futurum Podcast Network.

When cutting-edge AI capabilities shift overnight from exclusive enterprise clouds to permissive open-weights models accessible to anyone, IT leaders are left navigating a chaotic landscape of unprecedented power and heightened security risks.

In this episode of Utilizing AI, hosts Stephen Foskett, Brad Shimmin, and Olivier Blanchard unpack the realities of today’s expanding AI toolset—from high-performing open-source models coming out of China to Stripe's acquisition of OpenRouter.

The panel explores the stark tradeoffs between affordable accessibility, data privacy, and AI supply chain security. They also debate the rise of local hardware powerhouses for on-premise execution, the volatile "tokenomics" of reasoning models, and why the enterprise world desperately needs stable, Long-Term Support (LTS) release channels for AI software.

This and more on Utilizing AI, part of The Futurum Group Podcast Network.

Enterprise IT is currently facing a dizzying explosion of new AI models, frameworks, and platforms—making integration, rather than security or governance, the primary roadblock to actual business success.

In this episode of Utilizing AI, host Stephen Foskett is joined by Futurum Group analysts Brad Shimmin and Guy Currier to explore how organizations can effectively navigate this overwhelming "embarrassment of riches."

The panel breaks down the real-world friction of non-deterministic model behavior, sudden API performance shifts, and hidden tokenomics costs, evaluating whether emerging tools like model routers offer a true fix or merely add complexity.

Ultimately, they offer practical guidance for enterprise architects: double down on precise scope management, prioritize output quality and consistency over pure speed, consider open-source models for stability, and manage AI as a grounded IT asset rather than an all-knowing solution.

This and more on Utilizing AI, part of The Futurum Group Podcast Network.

What if the key to stripping away apocalyptic hyperbole and demystifying artificial intelligence lies in putting open-weights models directly into the hands of everyday creators worldwide?

In this episode of Utilizing AI, host Stephen Foskett and guest Guy Currier explore how the shift toward open-weights models and lower hardware requirements is breaking down the geographical and economic barriers of AI development.

Drawing parallels to historic industry shifts, they analyze how moving away from centralized, proprietary systems empowers students, global developers, and edge users to experiment without exorbitant costs.

By moving past "doomsday marketing" and embracing smaller, function-specific tools, widespread access allows society to better understand AI’s real capabilities, puncture the hype, and foster grassroots innovation.

This and more on Utilizing AI, part of The Futurum Group Podcast Network.

Although we are bombarded by headlines suggesting that AI will decimate the global workforce, the likely scenario is far more nuanced.

This episode of Utilizing AI brings Nick Patience, AI Platforms Practice Lead, and Olivier Blanchard, Devices Research Director, of the Futurum Group to discuss the latest data on the impact of AI on the job market.

When we look at the history of industrialization and technology, we see the value of investing in people and improving their skills rather than simply automating their tasks. We see the same effect as AI comes to software development and other aspects of business, reducing hiring of junior engineers and risking de-skilling senior developers as well.

Still, history suggests that society will adapt and new jobs will emerge, allowing the next generation to flourish and be productive in new ways.

This and more on Utilizing AI, part of The Futurum Group Podcast Network.

Not every task demands a multi-trillion parameter model with massive a context window, and we are seeing increasing success with small models designed with efficiency in mind.

This episode of Utilizing AI features Nick Patience and Brad Shimmin of The Futurum Group discussing the push and pull between frontier models and small domain-specific models.

Quantization, novel open weights models, and even local models give us all sovereignty over our data, at least to some extent. They also give us financial control, an increasingly important concern as AI service providers are charging increasingly large amounts for model use.

Patience and Shimmin are uniquely positioned in the industry to hear from all constituencies in the AI world, and provide their observations on the desire to take control of their software and model stack. All of these firms are starting to re-focus on efficiency, performance, and return on investment to optimize their AI applications even as the state of the art keeps pushing forward.

This and more on Utilizing AI, part of The Futurum Group Podcast Network.

Autonomous AI agents are officially entering the workforce, but can your business actually trust them to make critical decisions?

In this episode of Utilizing AI, hosts Stephen Foskett and Nick Patience sit down with Christian Kleinerman, EVP of Product at Snowflake, to discuss the high-stakes world of agentic AI and enterprise trust.

They unpack how AI tools are moving from simply answering questions to taking real-world actions, why AI correctness depends entirely on a strong data foundation, and how new frameworks like agent identities and semantic governance are giving organizations the confidence to scale autonomous workflows safely.

This and more on Utilizing AI, part of The Futurum Group Podcast Network.

This episode of Utilizing AI was presented by Snowflake.

Email questions Utilizing AI Podcast here.

So-called frontier models have gotten better and better, but users are increasingly focused on AI-powered applications rather than treating the chatbot as the application.

This episode of Utilizing AI features Stephen Foskett, Brad Shimmin, and Guy Currier of The Futurum Group discussing the impact of Tokenomics on the development of AI models.

Until recently, AI models were offerd a loss-leader pricing or even free to entice users to adopt them. But once the providers began charging for them, architects and users are considering alternatives. This has given rise to specialized small models for agentic applications like Meta's Muse Spark, Google's Gemma, and many others besides.

Companies are also pushing AI processing to the edge, with Apple's Siri running transparently on-device or in their cloud compute without the user's involvement. We are returning to a world where task-specific applications will be the product with the AI model being a resource used to deliver value rather than as a destination in itself.

This and more on this episode of Utilizing AI, part of The Futurum Group Podcast Network.