Summary
This episode explores why AI infrastructure must be built differently from traditional cloud infrastructure, with a focus on AI-first design rather than treating AI as just another workload. Corey Sanders of CoreWeave explains how training jobs depend on tightly coordinated GPUs, storage, and orchestration, where small failures or slowdowns can disrupt the entire run. The discussion also covers how AI development is shifting from manual dashboard analysis toward agent-led iteration loops, including tools like CoreWeave’s ARIA for AI research and iteration. Inference is presented as the emerging application layer for AI, increasingly composed of multiple models, agents, and specialized calls working together. The episode closes with a broader vision that software interfaces may move away from websites and buttons toward agent-based interactions and AI-first experiences.
Key Takeaways
- 1AI infrastructure needs to be designed as AI-first, not as a generic cloud add-on.
- 2Training workloads are extremely sensitive to failures and bottlenecks across the stack.
- 3The AI development workflow is moving from manual analysis toward agent-led iteration.
- 4Inference is becoming the main application layer for AI, and it will likely be multi-model and agentic.
- 5AI interfaces may replace many websites and button-based workflows with agent-driven interactions.
- 6Enterprises may soon need dedicated AI engineering teams to build and maintain AI-powered products.
Notable Quotes
""What got us successfully here today does not necessarily get us successfully to the next wave, right?""
""The concept of a failure in a job can mean a whole lot of things, right? There's certainly sort of failure from an infrastructure perspective.""
""We’ve recently launched a what we call, Aria, a core of Iria, which is short for AI research and iteration agent.""
""I think we'll go through the same progression with AI and probably faster to the point where we will have AI engineering teams at every single enterprise company.""
Episode questions
Why does CoreWeave believe AI infrastructure needs a different design than normal cloud infrastructure?
Because AI workloads depend on tightly interconnected GPUs, specialized storage, and optimized orchestration. Sanders argues that generic cloud assumptions—like simply adding more compute on demand—do not work well for large AI training jobs.
What problems cause AI training jobs to slow down or fail?
Failures can come from hardware issues, GPU slowdowns, storage bottlenecks, or orchestration problems. In large training runs, even one straggling GPU can impact the whole job, which is why observability and straggler detection matter.
How does ARIA change the AI research workflow?
ARIA is designed to analyze experiments continuously and recommend the next step, helping move from manual chart review to an iterative agent-driven loop. The goal is to make experimentation faster and more accessible to more practitioners.
What does CoreWeave mean by an AI loop?
It means continuously improving inference, prompts, models, and evaluations in a closed feedback loop. Traces, registries, sandbox evaluations, and agent guidance all help users identify what to change and then validate whether the change worked.