Certainty, Uncertainty, and the Future of Prompts | David Cox (VP, IBM Research)

Listen to full episode:

Summary: In this episode, Ed and Anant sit down with David Cox (VP, AI Foundations at IBM Research) to explore one of the most important questions in modern AI: how do we combine the power of large language models with the reliability of traditional software systems? They discuss why the future of AI may depend on smaller models, deterministic control flow, and cleaner abstraction boundaries — treating generative AI as a computational element inside a broader program rather than a standalone "agent" doing everything on its own. The conversation also touches on prompt brittleness, the limits of long context windows, and why developers need better tools to build defensively around model uncertainty, along with the interdisciplinary roots of AI, from neuroscience to computer science.

Chapters:

0:00 — Opening the episode

1:53 — Prompt injection and poisoning model output

5:08 — AI browsers and prompt injection attacks

8:59 — AI anxiety and the personal impact of AI

11:19 — Why AI needs a multidisciplinary perspective

15:42 — David Cox on his background in neuroscience

21:41 — LLMs and control flow limitations

22:40 — Determinism through software decomposition

29:20 — Models being wrong a significant portion of the time

35:09 — Clean abstraction boundaries

39:46 — Activated LoRA and inference-time specialization

45:44 — Divide and conquer for better LLM performance

49:48 — Closing remarks and wrap-up

Sound Bites:

“We are all kind of barreling through with the promise of AI, and yet at the same time, dangers are lurking down below.”

“Any kind of impact of AI on humanity will require a multidisciplinary perspective.”

“We can actually make these systems much more deterministic simply by breaking things up.”

“People in ‘LLM land’ just put everything in the context and say, figure it out.”