『What Happens When Your AI Agent Finds Two “Correct” Answers?』のカバーアート

What Happens When Your AI Agent Finds Two “Correct” Answers?

What Happens When Your AI Agent Finds Two “Correct” Answers?

無料で聴く

ポッドキャストの詳細を見る

【Amazonプライム会員限定】今ならプレミアムプランが4か月 月額99円。

10月19日まで。※適用条件あり
Two teams pitch the same figure for ‘active customers’ right before a leadership meeting. One dashboard displays 40,000 while the other dashboard displays 52,000. No mistakes were detected by the teams; however, the difference in the stats is due to one team or employee deploying a different data filter and the other using another join path.Now there's an AI agent involved, who is simply asked to 'retrieve the most recent number of active customers'. Not realising that there are two versions, it selects one, states that figure with complete confidence, and then goes about its business. In that moment, the semantic gap ceases to be merely an internal discussion and becomes a business risk.On this episode of the Don't Panic! It's Just Data podcast, host Shubhangi Dua, Podcast Producer and B2B Tech Journalist at EM360Tech, is joined by Ananya Devraju, a Business Intelligence Developer at Premier Inn, to discuss the semantic gap and how AI is rendering an old data problem. They look at the situation in which metric definitions are spread across five different dashboards rather than kept in a single central location, and explain why an AI agent can produce a perfectly valid SQL query yet return a completely incorrect answer.The problem, consistent even with AI in the picture, is governance. AI has just made the issue of governance more apparent. “It was never a data quality problem, but it's a governance of meaning problem, and it's been sitting there long before AI showed up. AI has just made it louder now,” Devraju tells Dua.Devraju will be presenting a talk at Big Data LDN (BDL) this year on Thursday, September 24, on When Migrations Break Your Metrics: Rebuilding Data for Commercial Analytics. The talk is scheduled for 3:20 pm to 3:50 pm in the Data Architecture Modernisation Theatre.TakeawaysSemantic gaps existed before AI; AI has only made them louder and faster.The same metric is displayed on different dashboards, each using different date logic, filters, or join paths.There can be five different definitions of an "active user" across the five dashboards, all of which are incompatible.SQL can be syntactically correct and yet provide the wrong answer to a business question.Validation should be placed at the definition level, not at the SQL level.AI agents can query the raw staging tables rather than referring to the governed marts by name alone.Incorrect joints or the wrong grain result in numbers that look plausible but are actually wrong.According to dbt Labs research, 72 per cent of peopleprioritise AI-assisted coding while 24 per cent prioritise validation.Incorrect data is more dangerous than having no data; it is the confident errors that pose a risk.Since end users are unable to check the answers at the time they occur, trust has to be established earlier on.Provide a semantic layer that is governed and has version control, with designated owners for each metric.Ad hoc SQL statements on no side-channel should generate the "official" figures that are reported.Chapters00:00 Introduction to the Semantic Gap in AI02:56 Understanding Data Interpretation and Governance05:56 The Importance of Centralized Definitions09:05 AI Hallucinations and Data Validity12:02 Navigating Governance in AI14:56 The Role of AI in Data Validation17:56 Final Thoughts on AI and Data MeaningGuestAnanya Devraju — Business Intelligence Developer, Premier InnHostShubhangi Dua — Podcast Producer & B2B Tech Journalist, EM360TechListen to more episodes of Don't Panic! It's Just Data on EM360Tech.#SemanticGap #AIAgents #DataGovernance #DataAnalytics #SemanticLayer #AIData #BusinessIntelligence #DataEngineering #EnterpriseAI #DataQuality
adbl_web_anon_alc_button_suppression_t1
まだレビューはありません