Listen to full episode:
Summary: Not Diamond Co-founder + CEO Tomás Hernando Kofman joins Anant and Ed to explore the latest advancements in AI model routing, the security implications of self-replicating AI worms, and the future of diverse AI ecosystems. Tomás shares insights on building smarter, cost-efficient AI infrastructure and the importance of model diversity.
Chapters:
03:35 Anthropic's insights on agentic breakout and model replication
05:01 Web GPU and the potential for distributed inference malware
11:14 Not Diamond's thesis on diverse model ecosystems
13:32 Model routing in coding and long-horizon workloads
16:55 Reinforcement learning for model recommendation and cost savings
20:19 Handling long sessions and cache management in AI inference
22:34 Features and rewards in AI model routing policies
25:29 Crowd learning and enterprise model management
30:02 Why coding is the first focus for model routing
31:58 Model routing architecture and enterprise needs
34:32 The role of gateways and provider agnosticism
36:08 Cost implications and token economics in AI inference
44:56 Benchmarking AI in interactive, long-horizon settings
46:02 Future trends in AI model diversity and ecosystem health
52:30 Building an AI company in a competitive landscape
57:45 The importance of VC backing and ecosystem support
Sound Bites:
"Agents can replicate independently of the model"
"Costs will explode as models become more capable"
"Many diverse models shift power away from labs"

