Software Engineering Institute (SEI) Podcast Series · Members of Technical Staff at the Software Engineering Institute

From Data to Performance: Understanding and Improving Your AI Model

·27 min·10 clips

Linda Parker-Gates hosts from the SEI's Washington, D.C. office. Chrisann Nolan and Nick Testa join her, both new to the podcast series. They are working on a project they are glad to talk about: a tool for helping programs better understand and improve AI performance. The conversation starts with introductions and a little background. Nick says he is a bit of an oddball and explains that his PhD is in evolutionary biology. He talks about studying the molecular, genetic, and developmental mechanisms behind shape and size. The episode then moves into the kind of reasoning the tool is meant to support. The speakers show how bias can creep in when an apparent relationship has a hidden cause. One example uses vehicles, coolant, maintenance events, and heat waves to show a common-cause problem. In that setup, more coolant and more failures can move together without one causing the other. The same logic comes up in the shark-and-ice-cream story. Higher temperatures can drive both ice cream sales and shark activity, which makes the correlation misleading. The hosts keep the explanation plain on purpose. They return to the point in simpler language so the listener can follow the logic step by step. The discussion also brings in collider bias as another way analysis can go off track. The tone stays measured and practical instead of flashy. The episode is built around making abstract statistical ideas easier to use in real programs. It ends with the SEI's standard contact and distribution information. The result is a grounded conversation about how to read AI performance more carefully before drawing conclusions.

As heard by us

A practical discussion of how bias and confounding can distort AI performance.

An institutional but approachable conversation about how bias and confounding can distort AI performance, and how to think through the data before trusting the model.

Read the full review in PlayNext →

Why you'd press play

See how coolant, heat, shark attacks, and ice cream sales help explain AI model bias.

Read the full recommendation in PlayNext →

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from Software Engineering Institute (SEI) Podcast Series