Submit when the Science is Done
Learn how to make agentic AI trustworthy in clinical trials, with your experts in control, always.
In this episode of the Verisian Community Podcast, we welcome Ricardo Baeza-Yates, a renowned computer scientist, to discuss the world of Large Language Models (LLMs) and their implications. Ricardo delves into the mechanics of LLMs, their popularity, and the challenges they pose. He discusses the inherent lack of reasoning and understanding in LLMs, explores the limitations in reproducibility, and emphasizes the importance of transparency in AI development. Ricardo shares his insights on how these issues impact the reliability and application of LLMs, particularly in contexts requiring precision and trustworthiness.
Guest: Ricardo Baeza-Yates, director of research for Experiential AI (Institute for Experiential AI) at Northeastern University, researcher in responsible and ethical AI.
Host: Tomás Sabat Stöfsel, CEO & Co-Founder, Verisian
Topics: What LLMs are and how they work, the difference between imitating understanding and genuine understanding, whether LLMs are more dangerous than nuclear weapons, open versus closed-source model debates, AI regulation in China versus the EU, why LLMs struggle to extrapolate beyond training data, frameworks for safely deploying LLMs in high-stakes applications, why LLMs lack real knowledge and memory, whether LLMs can reason, whether statistical programmers will become obsolete, a vision for AI's role in human work
The full episode can be found on YouTube, Apple Podcasts or Spotify.
Ricardo is deliberate about avoiding human verbs (read, write, think, understand, learn) when describing LLMs, arguing these systems imitate human language by statistically reproducing patterns from massive training corpora rather than genuinely processing meaning. Asked whether imitating understanding well enough is functionally equivalent to understanding, he draws on the human-chimpanzee DNA comparison: despite sharing 99.9% of our DNA, the remaining 0.1% produces capabilities (language, abstraction) entirely absent in chimps. He argues LLM "understanding" sits closer to a chimpanzee's than a human's, and that repeated AI failures on cases a human would never get wrong, an autonomous vehicle failing to recognize a cyclist at night, or a chatbot failing to recognize the danger in a conversation about suicide, are direct evidence that pattern-imitation isn't the same as comprehension.
Asked about claims that LLMs pose a greater danger than nuclear weapons because software has a far lower barrier to misuse than physical weapons infrastructure, Ricardo reframes the premise: the danger was never the technology itself but the humans deploying it, a pattern he traces through the history of atomic power, radiation, and chemical weapons. His practical conclusion is that the real requirement isn't "humans in the loop" but "humans in control", sufficient oversight that an LLM, for instance, could never be positioned to trigger a weapon. He extends this into a broader historical pattern: technologically advantaged groups have always used that advantage to exploit less-advantaged ones (slavery, colonialism), and argues the current era's version of that exploitation is subtler, cultural manipulation and disinformation delivered at LLM-enabled scale.
Discussing the open-versus-closed model debate (Meta/Llama versus OpenAI), Ricardo notes Llama's "open" status arguably originated from an unauthorized leak of model weights rather than a deliberate strategic choice, which Meta then embraced as a market position. He sees genuine tradeoffs on both sides: open models enable healthy competition but also broaden the pool of people who can misuse them. On regulation, he highlights that China's generative AI regulation (proposed for public comment in April 2023) preceded the EU's AI Act and, in his assessment, addresses a broader set of concrete concerns, copyright, misinformation, hallucination, and mental health risk, more thoroughly than commonly assumed in Western commentary, while noting that framing AI-driven disinformation purely as a threat to democracy (as historian Yuval Noah Harari has argued) undersells that it threatens any form of government, not just democratic ones.
Ricardo's central technical argument is that LLMs excel at interpolating within the boundaries of their training data, producing a sharply focused, "fisheye" view of whatever topic is prompted, while struggling badly to extrapolate beyond it, which requires genuine understanding rather than statistical prediction. He illustrates this with a hallucinated academic paper attributed to him that never existed (a known LLM failure mode when it lacks grounding knowledge), and with the fatal 2018 autonomous vehicle case in which the system failed to classify a cyclist because its training data likely never included a bicycle at night. His point: extrapolation is precisely where human creativity and judgment operate, and precisely where AI failure risk concentrates, which matters enormously for a field like clinical statistical programming where reproducibility, not 99% accuracy, is the actual requirement.
For high-stakes applications, Ricardo recommends predicting the risk or complexity of a given case using standard (non-LLM) predictive techniques, then tiering the response accordingly: low-risk cases can be automated with minimal oversight, medium-risk cases require mandatory human review before any output is used, and the highest-risk cases should bypass the LLM's output entirely, going straight to human (or committee) judgment. He flags a specific cognitive-bias risk in this framework: people tend to over-trust system output, partly from a kind of imposter syndrome relative to the AI and partly because deferring to the system's recommendation offers a convenient way to disclaim personal responsibility for a bad outcome, something he stresses is not actually true, the decision remains the human's.
Ricardo explains that LLM architectures are built, by design, to predict the next most probable token, which constitutes a kind of predictive memory rather than genuine, fact-grounded knowledge; retrieval-augmented approaches (he cites Perplexity as an example) attempt to link outputs to verifiable facts, but as a post-processing step layered on top of a fundamentally predictive architecture, it isn't fully reliable. He frames genuine reasoning as requiring three things together: knowledge (to distinguish true from false), reasoning proper (to verify a logical claim, as in a formal mathematical proof), and contextual understanding, roughly equivalent to human common sense, which depends on the same extrapolation capacity LLMs largely lack. He also cites Stanford research (a NeurIPS best-paper winner) showing that so-called "emergent" LLM capabilities are, under correct evaluation metrics, standard learning curves rather than genuinely novel behavior, a finding he says even leading AI researchers have sometimes overinterpreted.
Asked directly whether programmers, and statistical programmers specifically, face obsolescence, Ricardo doesn't predict disappearance but a shift in the role's center of gravity from writing programs to checking and validating them, with a specific concern that an LLM-generated program could contain an error pattern no human programmer would ever produce, and therefore one a human reviewer might struggle to even think to look for. He argues that as programs grow more complex, the time saved writing code may simply be spent verifying it instead, which paradoxically increases the need for elite, deeply experienced "Jedi" programmers (his term, drawn from a personal essay applying the same idea to medicine), both to handle the hardest cases directly and because sustained AI improvement itself depends on having people who can still out-program the AI. He's explicit that there's no shortcut to that expertise: becoming a "Jedi" programmer requires writing extensive original code from scratch, the same apprenticeship path that produced landmark individually authored software like the Linux kernel or Donald Knuth's TeX.
Closing on a deliberately non-predictive note (he declines to forecast the next five years, joking Murphy's Law makes predictions unwise), Ricardo lays out what he'd like to see instead of the current cost-cutting, headcount-reduction trajectory: AI investment aimed at growing revenue and human capability rather than replacing people, paired with a rigorous, case-by-case benefit-versus-harm analysis for critical applications. He closes with a widely circulated framing he endorses explicitly: AI should do the tasks people don't want to do, freeing humans for the ones they do, illustrated by a tweet he cites: "I don't want AI to write for me and then I can do the laundry. I want AI to do the laundry, and then I have time to write."
"Imitating understanding is understanding, and I'm on the side that I say no, that's not the same." — Ricardo Baeza-Yates
"We are the danger, not the technology. Technology is neutral. Depending on how we use it, it's dangerous." — Ricardo Baeza-Yates
"What is hard for the system to do is to extrapolate outside the data, because that will need understanding of the data. So that's where humans are good at." — Ricardo Baeza-Yates
"Instead of programmers, I think we will need to have program checkers... the more complicated the program, the more complicated the checking." — Ricardo Baeza-Yates
"I don't want AI to write for me, and then I can do the laundry. I want AI to do the laundry, and then I have time to write." — quoted by Ricardo Baeza-Yates
Do large language models actually understand what they're processing?
According to Ricardo Baeza-Yates, no. He argues imitating understanding statistically is not the same as genuine understanding, comparing the gap to the small DNA difference between humans and chimpanzees that nonetheless produces entirely different capabilities, and points to AI failures on cases a human would never misjudge as evidence of that gap.
Are LLMs more dangerous than nuclear weapons?
Ricardo calls this framing a fallacy: the danger comes from how humans deploy a technology, not the technology itself, a pattern he traces through the history of atomic power and chemical weapons. He argues the real requirement is sufficient human control over AI systems, not simply "humans in the loop."
Why do LLMs struggle with tasks outside their training data?
Ricardo describes LLMs as excellent at interpolating within familiar topics (a "fisheye view") but fundamentally weak at extrapolating beyond their training data, which requires genuine understanding rather than statistical prediction, the same limitation behind incidents like a self-driving car failing to recognize an unfamiliar road scenario.
How should organizations safely deploy LLMs in high-stakes settings?
Ricardo recommends a risk-tiered framework: predict a case's risk or complexity using standard techniques, automate only low-risk outputs, require human review for medium-risk cases, and route the highest-risk cases directly to human (or committee) judgment without relying on the LLM's output at all.
Will AI make statistical programmers obsolete?
Ricardo doesn't predict obsolescence but a redefined role centered on program checking and validation rather than pure code generation, alongside continued demand for elite, deeply experienced programmers who can catch AI-introduced errors and keep out-programming the AI itself.
The Verisian Community Podcast brings together experts in clinical trials to exchange innovative ideas and best practices central to clinical reporting, submission and review. Aligned with Verisian's mission to accelerate the evaluation and market launch of new medical treatments, each episode features expert insights, with guests ranging from statistical programmers to medical writers, to discuss the challenges and opportunities of the latest software and technology.
You can listen to us on YouTube, Apple Podcasts or Spotify.
See Verisian in action. Share a few details and we’ll be in touch to schedule your demo.
Your request has been received. A member of the Verisian team will contact you shortly to arrange a demo.
