The Hidden Economics of AI Model Routing | Tomás Hernando Kofman (Co-founder + CEO, Not Diamond)

 

Listen to full episode:

Summary: Not Diamond Co-founder + CEO Tomás Hernando Kofman joins Anant and Ed to explore the latest advancements in AI model routing, the security implications of self-replicating AI worms, and the future of diverse AI ecosystems. Tomás shares insights on building smarter, cost-efficient AI infrastructure and the importance of model diversity.

Chapters:

03:35 Anthropic's insights on agentic breakout and model replication

05:01 Web GPU and the potential for distributed inference malware

11:14 Not Diamond's thesis on diverse model ecosystems

13:32 Model routing in coding and long-horizon workloads

16:55 Reinforcement learning for model recommendation and cost savings

20:19 Handling long sessions and cache management in AI inference

22:34 Features and rewards in AI model routing policies

25:29 Crowd learning and enterprise model management

30:02 Why coding is the first focus for model routing

31:58 Model routing architecture and enterprise needs

34:32 The role of gateways and provider agnosticism

36:08 Cost implications and token economics in AI inference

44:56 Benchmarking AI in interactive, long-horizon settings

46:02 Future trends in AI model diversity and ecosystem health

52:30 Building an AI company in a competitive landscape

57:45 The importance of VC backing and ecosystem support

Sound Bites:

"Agents can replicate independently of the model"

"Costs will explode as models become more capable"

"Many diverse models shift power away from labs"

Previous
Previous

Is AI Changing the Rules of Venture Capital? | Swaroop (Kittu) Kolluri (Founder + Managing Director, Neotribe Ventures)

Next
Next

The Death of Marketing Execution | Denise Persson (CMO, Snowflake)