Making Small Models Cool Again | Kate Soule (Director, IBM Granite)

Listen to full episode:

Summary: In this episode, Anant and Ed are joined by Kate Soule (Director, IBM Granite) to explore why small language models are becoming increasingly important in modern AI systems. The conversation covers the rise of agentic workflows, the trade-offs between large and small models, and why the future of practical AI may depend less on “one model to rule them all” and more on an ensemble of specialized tools.

Chapters:

0:00 — AI Agents, Silicon Valley, and the Doomer Debate

11:52 — Kate Soule on Granite and Why Small Models Matter

24:58 — Why Small Models Are a Natural Fit for AI Agents

32:02 — Beyond Fine-Tuning: Adapters, Retrieval, and Specialization

37:59 — Are Models Becoming Commodities? What Comes Next

45:01 — Evaluating AI Agents and the Boundary Between Models & Apps

49:50 — Next Up: Bringing More Structure to Prompts

Sound Bites:

“There is no one model to rule them all.”

“You can only write an email so well.”

“We need to build agents differently if we’re going to really take advantage of small language models.”

“The boundary between agents and models is perhaps the next frontier.”