Here’s a quick wrap of the three papers we found interesting over the last few weeks with some take home points.

• 1:00 - Clinical knowledge in LLMs does not translate to human interactions
• 06:45 - From Tool to Teammate: A Randomized Controlled Trial of Clinician-AI Collaborative Workflows for Diagnosis
• 11:55 - Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study

More details in the show notes on our website.

Episodes | Bluesky | info@medicalattention.ai

In this episode, we’re lucky to be joined by Alexandre Sallinen and Tony O’Halloran from the Laboratory for Intelligent Global Health & Humanitarian Response Technologies to discuss how large language models are assessed, including their Massive Open Online Validation & Evaluation (MOOVE) initiative.

0:25 - Technical wrap: what are agents?

13:20 - What are benchmarks?

• 18:20 - Automated evaluation

• 20:10 - Benchmarks

• 37:45 - Human feedback

• 44:50 - LLM as judge

Read more about the projects we discuss here:

• Meditron

• Learn about the MOOVE or contact our team if you'd like to be involved

• Listen to the LiGHTCAST including their recent excellent outline of the HealthBench paper

More details in the show notes on our website.

Episodes | Bluesky | info@medicalattention.ai

Here’s a quick wrap of the three papers we found interesting over the last few weeks with some take home points.

• 0:35 - Superhuman performance of a large language model on the reasoning tasks of a physician
• 06:20 - MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks
• 11:45 - Identifying and mitigating algorithmic bias in the safety net

More details in the show notes on our website.

Episodes | Bluesky | info@medicalattention.ai

We’re trying out a new episode format! We’ll be doing a quick wrap of the top three papers we found interesting over the last few weeks with some take home points.

• 01:30 - Patient Reactions to Artificial Intelligence-Clinician Discrepancies
• 06:50 - AI-based volumetric six-tissue body composition quantification from CT cardiac attenuation scans for mortality prediction
• 11:10 - Large-scale Local Deployment of DeepSeek-R1 in Pilot Hospitals in China

More details in the show notes on our website.

Episodes | Bluesky | info@medicalattention.ai

In this episode, we tackle some common myths about how generative AI works, why this is the case, implications for healthcare and some quick fixes. These myths include 1) that LLMs can explain their reasoning 2) that LLMs can express uncertainty, 3) that LLMs can a) do maths, b) manage temporal data c) apply guidelines d) handle negation and finally that 4) that AI will replace clinicians.

02:00 Technical update - DeepSeek, other new models

10:00 - AI mybusting

• 15:50 - LLMs can explain their reasoning

• 21:50 - LLMs can express uncertainty

• 26:40 - LLM blindspots

• 41:50 - AI will replace clinicians

Episodes | Bluesky | info@medicalattention.ai

In this episode, we discuss algorithmic bias and fairness in healthcare. We explain what this is, the different definitions of “fairness”, explore the ways in which bias can enter the machine learning pipeline and some ways to combat it.

01:00 Technical update - NeurIPS

06:50 Technical update - ChatGPT-o3

14:00 - Algorithmic bias

Episodes | Bluesky | info@medicalattention.ai

We’ve returned after an accidental hiatus, just in time for the end of the year. In this episode, we’re joined by the team behind the American Medical Informatics Association (AMIA) Year in Review - Professor James Cimino, Pushkala Jayaraman and Dr Humayera Islam to talk about the main themes in healthcare AI for 2024.

01:00 Technical update (ChatGPT-o1, agentic AI) 10:25 AMIA Year in Review

Episodes | Bluesky | info@medicalattention.ai

We’re back! This is the start of our regular discussions about healthcare AI topics and recent literature. On today’s episode - the 10 commandments of decision support, the checkered history of EMRs, clinicians as “moral crumple zone” for AI models and much more.

01:18 Technical update (Llama 3.1, Phi releases)

06:25 Main discussion

41:20 Article round-up

Episodes | Twitter | info@medicalattention.ai