In one of the most popular episodes of the year, Legalbenchmarks.ai Founder Anna Guo discusses her organization's research that tests whether artificial intelligence custom-made for legal tasks better than general AI tools.

Anna is a former BigLaw lawyer who left the practice to become an entrepreneur and now focuses her energies on quantifying the utility of AI in the legal industry. Anna's initial anecdotal research for colleagues quickly revealed a strong community interest in a systematic approach to evaluating legal AI tools. This led to the creation of Legalbenchmarks.AI, dedicated to finding out where the promise of humans plus AI is truly better than humans alone or AI alone.

The core of the research involves measuring the "delta," or the extent to which AI can elevate human performance. To date, Legalbenchmarks.ai conducted two major studies: one on information extraction from legal sources and a second on contract review and redlining.

Key Findings from the Studies:

• Accuracy vs. Qualitative Usefulness: The highest-performing general-purpose AI tools (like Gemini) were often found to be more accurate and consistent. However, the legal-specific AI tools often received higher marks in qualitative usefulness and helpfulness, as they align more closely with existing legal workflows.

• Methodology: The testing goes beyond simple accuracy. It includes a three-part assessment: Reliability (objective accuracy and legal adequacy), Usability (qualitative metrics like helpfulness and coherence for tasks such as brainstorming), and Platform Workflow Support (integration, citation checks, and other features).

• Human-AI Performance: In the contract analysis study, AI tools matched or exceeded the human baseline for reliability in producing first drafts. Crucially, the data demonstrated that the common belief that "human plus AI will always outperform AI alone" was false; the top-performing AI tool alone still had a higher accuracy rate than the human-plus-AI combo.

• Risk Analysis: A significant finding was that legal AI tools were better at flagging material risks, such as compliance or unenforceability issues in high-risk scenarios, that human lawyers missed entirely. This suggests AI can act as a crucial safety net.

• Strengths Comparison: AI excels at brainstorming, challenging human bias, and performing mass-scale routine tasks (e.g., mass contract review for simple terms). Humans retain a significant edge in ingesting nuanced context and making commercially reasonable decisions that AI's instruction-following can sometimes lack.

In the most listened to episode of 2025, author of "Systemology" and business systems expert, David Jenyns, discusses the importance of systemizing business processes, even in the nuanced field of law. Jenyns debunks the myth that legal work is too bespoke to be systemized, explaining how just like any other business, documenting legal workflows unlocks scalability and creativity. He shares his journey from digital SEO agency owner to becoming a systems expert and outlines the seven steps of his Systemology framework.

• Key Topics:

  • The importance of documenting processes in any business, including legal services.

  • How systemization can create space for creativity in legal work.

  • The Systemology framework and its seven steps: Define, Assign, Extract, Organize, Integrate, Scale, and Optimize.

  • The role of a "systems champion" in implementing a systems culture.

  • Addressing common misconceptions about systemizing legal work.

  • Practical tips for getting started with systemizing processes.

• Resources Mentioned:

• • Systemology book by David Jenyns

  • System Hub software

  • The E-Myth by Michael Gerber

  • Systemology Podcast (Mentioned interview with Crow Estate Planning firm)

Editing and Production: Grant Blackstock

Theme Music: Home Base (Instrumental Version) by TA2MI

• Want to keep up to date about new episodes? Technically Legal Update List.

• Want to learn more about Percipient (percipient.co)?

• Follow Chad on Linkedin: Chad Main | LinkedIn

• Follow the podcast on LinkedIn: Technically Legal | LinkedIn

• Follow the podcast on Instagram: Technically Legal | Instagram

• Follow the podcast on X: Technically Legal | X