Part two of two with Roz Hirsch.

Roz has been applying for jobs and not getting interviews she would once have gotten easily. She's been rejected in under an hour. She's been rejected at midnight, by companies where nobody was awake to read anything.

Her expertise is language assessment, so she makes the argument nobody else is making. A resume is an assessment. Assessments require a validity argument, meaning evidence that the decisions you make with them are the right decisions. Validity is a property of the decision, not of the instrument. And validation has to happen at every single company that adopts a screening tool, because a tool validated somewhere else for something else has not been validated for you.

So, has anyone gone back and reread the rejections? Roz cites hiring managers who did, after two full cycles that produced no hires, and found people who should not have been rejected. That's a validity failure, and the near-instant rejection timestamps suggest nobody is checking.

We also get into what the screen is actually reading. Roz's point is that it isn't only the resume and cover letter. It's postal code, financial history, whatever else is available, and the inference that nobody from that postal code works here so this person probably won't either. I bring in Uber's pickers and ants, airline pricing, and the casual nursing algorithms that offer lower wages to people whose credit history says they'll accept.

Then the same argument turned on education. Roz on the professor whose take-home midterm produced near-perfect scores and whose in-class final didn't, and why the bell curve was the problem before AI ever showed up. Why she doesn't have a cheating problem. And the classroom exercise she runs with AI image generation, where students ask for one bear and keep getting several.

Links

• Roz's Random Ramblings: Language, History, and Other Adventures
• Roz on LinkedIn
• Women Writin' 'Bout AI
• Carol Chappelle, Iowa State and The Applied Linguistics Encyclopedia
• Image Description Games: Twin Pics, Say What You See, and Promptle
• Enshittification by Cory Doctorow and our show about the same
• Uber pickers and ants
• The Brown Professor story about AI and cheating
• The TOEFL (Test of English as a Foreign Language)
• Part one of Kim & Roz talkin' 'bout AI

Leave us a comment or a suggestion!

Support the show

Contact us: https://www.womentalkinboutai.com/

Kimberly's friend Roz Hirch is the guest on this two-part series. Roz is a linguist, a college instructor in Medicine Hat, Alberta, and a language assessment specialist. She is also out of work for the summer for the first time in her life. Kimberly suggested that she read The Artist's Way by Julia Cameron, and that led to Roz asking ChatGPT and Claude for book recommendations as well. She wrote a question describing herself and her situation and asked ChatGPT and Claude for reading recommendations. Something in ChatGPT's answer bothered her enough that she took the identical question, word for word, and texted it to friends and family to see what people would do with it.

The machines gave her thirteen books and seven. The humans gave her one, or two, or none. Three titles appeared on both AI lists. Not one appeared on both an AI list and a human list. The AI books all pointed the same direction, which was creating a portfolio career, company of one, multipotentialite, like build an umbrella and put everything under it. The people who actually know Roz told her to write.

Roz and I analyzed the responses from the humans and the bots, and because Roz has a background in theater as well as linguistics, she reaches for the difference between naturalism, which is how people talk, and realism, which is how we think people talk. We look at what humans do that machines don't, such as dropping the subject, hedging in nearly every response, and knowing when to stop, which Grice's maxim of quantity covers and which one model violated thirteen times over. We also analyze the speech act itself, recommendations. A recommendation ordinarily requires the speaker to have read the thing and to stake something on it. The form survives in the AI answers. The function is hollowed out, because there is nobody there to have been inspired.

Links

• Roz's Random Ramblings: Language, History, and Other Adventures
• Roz on LinkedIn
• Women Writin' 'Bout AI
• John Searle, "The Chinese Room"
• Mini Philosophy, Jonny Thomson, the episode on online versus face-to-face conversation
• How to Be Everything, Emilie Wapnick
• Range, David Epstein
• The Wealthy Barber, David Chilton
• The Artist's Way, Julia Cameron

Leave us a comment or a suggestion!

Support the show

Contact us: https://www.womentalkinboutai.com/

Devon Cantwell-Chavez studies global urban climate change governance. She and Kimberly met because of an antagonistic LinkedIn post (not between the two of them), and then discovered that, in many ways, they came up the same way. They both were Teach for America corps members, both early believers in classroom technology, and both landed somewhere far more critical. This conversation is about what gets lost when we design friction out of learning and research.

They get into the myth of the digital native and why "nobody knows how file folders work anymore," the frictionless interfaces that train us to just ask instead of think, and Goodhart's Law, the idea that a measure stops being effective once it becomes a target. Devon lays out how her research team built friction back in on purpose with a no-first-use policy, low-stakes-only translation tools, and a tagging system so every use of AI is on the record.

The episode closes on why AI can't be replicated the way rules-based software can, what that means for qualitative research, and where Devon finds hope, on the lawns of rural Michigan, in t-shirts and yard signs against data centers.

Mentioned in this episode:

• The AI Con, Emily Bender and Alex Hanna
• Being Wrong: Adventures in the Margin of Error, Kathryn Schulz
• Right Kind of Wrong: The Science of Failing Well, Amy Edmondson
• Kimberly's Substack that discusses the following:
  • Don Norman on design responsibility
  • Rosina Lippi-Green on communication as a two-way street
• Goodhart's Law
• Devon's viral LinkedIn post
• The chess-cheating study (in understandable language) or the research manuscript preprint
• The Brown University exam experiment
• Box Elder County data center coverage
• Digital Natives
• Non-Consensual Sexual Imagery
• Devon's t-shirt
• Cancellation of data center projects: https://www.datacenterwatch.org/report

Fact Checking Notes:

Traditional spell checkers were primarily dictionary- and rule-based, later augmented with statistical language models and machine learning. Modern writing assistants (including current Grammarly features) increasingly combine traditional spelling and grammar checking with large language models.

Leave us a comment or a suggestion!

Support the show

Contact us: https://www.womentalkinboutai.com/