You connect an LLM to a few tools, give it a task, and suddenly it looks like the system can reason, plan, and act on its own.

But production is a different story.

In this episode of My Data Guest, I spoke with Dipanjan Sarkar, AI engineer and community leader at Analytics Vidhya, about what it really takes to build agentic AI systems that work reliably outside a notebook.

The main point was clear: agentic AI is not just a prompt, a framework, or a clever demo. It is an engineering problem.

A production system has to deal with messy user requests, tool failures, long context, security risks, hallucinations, monitoring, and evaluation. These are not details you add at the end. They need to shape the system from the beginning.

One topic we discussed was context engineering. In agentic systems, the model does not only receive a user prompt. It may also receive tool outputs, retrieved documents, previous steps, instructions, and business rules. If the context becomes too long or poorly structured, the system can fail in unpredictable ways.

As Dipanjan put it:

“Context window limitations cause system crashes.”

We also talked about hallucinations. Giving an LLM access to tools does not magically solve the problem. The model can still choose the wrong tool, misunderstand the output, or produce an answer that sounds correct but is not grounded in reality.

Another key point was evaluation. Asking the model how confident it is is not enough.

“Confidence scores are unreliable in LLMs.”

For real systems, teams need custom evaluation metrics, tracing, and monitoring. It is not enough to check the final answer. You also need to understand the steps the agent took to get there.

Security is another major concern. Once an agent can access APIs, documents, databases, or internal tools, the risks become much larger. Prompt injection, data leakage, unsafe actions, and poor governance all become real production problems.

The conclusion of the episode is simple: if you want to build agentic AI for production, start thinking like an engineer from day one.

Design for failure.Add observability.Evaluate the actual workflow.Set clear permissions.Keep humans in the loop where needed.

Agentic AI has huge potential, but the companies that succeed will not be the ones with the flashiest demos. They will be the ones that build systems that are reliable, observable, and safe enough to be trusted.

Listen to the episode

In this conversation, we cover:

* common mistakes when deploying agentic AI

* why demos often fail in production

* context engineering

* hallucinations and model limitations

* debugging and monitoring strategies

* evaluation challenges

* security and governance

* what may happen in the next few years