Afleveringen
-
Stanislas Polu is Co-Founder & CTO of Dust β the enterprise AI agent platform used by 51,000 workers at 3,000+ companies. Before Dust, he spent three years on OpenAI's research team under Ilya Sutskever, working on mathematical reasoning in language models, and prior to that was an engineer at Stripe. He brings a rare combination of frontier AI research and product-building experience to the enterprise agent space.
MCP, Agents & the $40M Bet on Multiplayer AI // MLOps Podcast #384 with Stanislas Polu, Co-Founder & CTO of Dust
π€ What is Dust? β How Dust enables teams to build and deploy AI agents powered by internal company data, and why the "multiplayer AI" model is winning in enterprise.
π§ From OpenAI Research to Startup Founder β Stanislas's journey from studying mathematical reasoning in LLMs under Ilya Sutskever to co-founding an enterprise AI company in Paris with Gabriel Hubert.
π MCP & Standardization β Why the Model Context Protocol matters, what's trivial vs. what's transformative about MCP, and how Dust integrates MCP-compatible servers for enterprise workflows.
π The $40M Series B β What Dust is building with fresh funding, the bet on human-agent collaboration as the future of work, and what "multiplayer AI" actually means in practice.
π The Outer-Loop Era β Stanislas's framework for thinking about where AI agents create the most value: not just automating tasks, but rewiring how work gets done across entire organizations.
β οΈ What Most Enterprise AI Gets Wrong β The biggest mistakes companies make when deploying AI agents, why adoption fails, and how Dust achieves 70%+ weekly adoption rates.
π Building Reliable Agent Infrastructure β Lessons from scaling to thousands of companies: observability, governance, data security, and why enterprise AI is harder than it looks.
π οΈ Horizontal vs. Vertical AI Platforms β Why Dust chose to build a horizontal enterprise agent platform and how that decision shapes product, go-to-market, and technical architecture.
This episode is essential for AI/ML engineers, enterprise AI leads, and anyone building or deploying AI agents at scale inside organizations.
π Links & Resources:
β’ Dust: https://dust.tt
β’ Stanislas Polu on X/Twitter: https://x.com/spolu
β’ Dust on LinkedIn: https://www.linkedin.com/company/dust-tt
β’ Dust $40M Series B announcement: https://dust.tt/blog
β’ "The Outer-Loop Era" talk by Stanislas (dotconferences): https://www.youtube.com/watch?v=_outer_loop
β’ Dust + Stripe MCP integration: https://stripe.com/customers/dust
β’ Dust + Datadog observability case study: https://datadoghq.com/case-studies/dust
β±οΈ Timestamps
[00:00] Future of Work
[00:19] Dust Scaling Lessons
[04:44] Human-Agent Collaboration
[14:24] Pod as Workspace
[22:30] Work Flow Optimization
[29:37] Multiplayer Collaboration Vision
[39:55] Token Economics and Inference
[47:20] AI Pricing Challenges
[52:36] Dust vs Co-work
[57:06] Agentic Work Infrastructure
[1:04:23] Stateful Sandbox Challenges
[1:09:58] Product Use Case Discussion
[1:14:05] Agent Data Interaction Needs
[1:20:09] Wrap up
#EnterpriseAI #AIAgents #Dust
-
James Everingham is the CEO and Co-founder of Guild.ai β the AI agent control plane for production teams. With roots at Netscape, Instagram (Head of Engineering), and Meta (Head of Dev Infra, leading a 1,000-person org), James brings rare, hard-won expertise to the challenge of operating AI agents at scale.
From Single-Player to Multi-Player: Operating AI Agents at Scale // MLOps Podcast #383 with James Everingham, CEO and Co-founder of Guild.ai
In this episode, James unpacks what actually breaks when you move from a single AI agent to a fleet of them β and what engineering leaders need to build before it's too late.
π― Single-Agent vs. Multi-Agent Systems β Why "single-player" AI workflows don't survive contact with production reality, and what the shift to multi-agent coordination actually demands from your infrastructure.
π The Agent Control Plane β What it is, why every engineering org needs one in 2026, and how Guild.ai is building the neutral layer to deploy, govern, and share agents across any framework or model.
β οΈ Non-Determinism at Scale β Why AI agents behave like employees, not software, and why you need workforce-style governance β not just observability tooling β to manage them.
πΈ Token Spend & Cost Visibility β How teams running agents in production are flying blind on cost, and what Guild shows you that your current stack doesn't.
ποΈ Lessons from Meta's DevMate β How Meta's AI coding agent went from experiment to submitting 50% of all diffs, and what that journey teaches every engineering leader about scaling agents safely.
π¦ Agent Identity & Governance β Why every agent needs an identity, what happens when they don't have one, and how agent sprawl becomes a governance crisis fast.
π Sharing Agents as Infrastructure β Why Guild treats agents as shared production infrastructure rather than one-off scripts, and how that changes the economics of AI investment.
π οΈ Framework Agnosticism β Why betting on a single agent framework is a losing strategy, and how to build for a multi-model, multi-framework world from day one.
Essential viewing for engineering leaders, AI platform teams, and founders building production-grade agentic systems.
π Guild.ai: https://guild.ai
π James on X/Twitter: https://x.com/jevering
π James on LinkedIn: https://www.linkedin.com/in/jameseveringham
π Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
β±οΈ Timestamps
[00:00] Context Transfer Challenges
[00:51] Control Plane for Agents
[02:17] Effective Agent Policies
[09:23] Agent Governance Policies
[15:34] Developer Tool Adoption
[22:02] Knowledge Sharing and Open Source
[24:59] Simulated Deployments and Confidence
[29:36] Agent Workloads vs Human Workloads
[39:55] AI as a Customer
[47:59] Agent Hub vs Autonomy
[53:21] Wrap up
#AgenticAI #AIAgents #AIEngineering
-
Zijn er afleveringen die ontbreken?
-
Thiago Cardoso is the Director of Data & AI at iFood and the architect behind iFood Pago's AI agent platform. This fintech system serves millions of restaurants across Brazil through WhatsApp and the iFood app. In this episode, he breaks down what it actually takes to ship agentic AI in production at scale.
The Control-vs-Magic Spectrum Building Agents // MLOps Podcast #382 with Thiago Cardoso, Director of Data & AI at iFood
π€ WHAT WE COVER:
πΉ Control vs. Magic β Thiago's spectrum model for thinking about AI agents, from deterministic pipelines to fully autonomous systems
πΉ iFood Pago Explained β How iFood's embedded fintech arm uses AI agents to provide credit, loans, and financial services to restaurants
πΉ WhatsApp as an AI Interface β Why WhatsApp is the primary channel for merchant interactions in Brazil and how agents are deployed there
πΉ Multi-Agent Architecture β Why single monolithic agents break down and how to split them into sub-graphs with specialized contexts and tool sets
πΉ Context Engineering β Why what you put in the agent's context window is more important than the model itself
πΉ Human-in-the-Loop Design β How to build trust with merchants while minimizing friction in agentic workflows
πΉ LangGraph in Production β How Thiago's team uses LangGraph to build stateful, multi-agent pipelines
πΉ Debugging with AI β Generating on-the-fly HTML/JavaScript visualization tools to investigate data pipeline problems
πΉ The Cost of Software Going to Zero β What happens to demand when software becomes nearly free to build
πΉ Personalization at Scale β Serving millions of restaurants with AI that knows their business context
π― This episode is for AI engineers, ML practitioners, and fintech builders who want to understand what production agentic AI looks like beyond the demos.
π LINKS & RESOURCES:
Thiago Cardoso on LinkedIn: https://www.linkedin.com/in/thiagoncc/
iFood: https://www.ifood.com.br
iFood Pago: https://ifoodpago.com.br
ZenML iFood Case Study: https://www.zenml.io/llmops-database/building-a-hyper-personalized-food-ordering-agent
LangGraph: https://www.langchain.com/langgraph
β±οΈ TIMESTAMPS
[00:00] Control vs Magic in AI
[00:18] Foodpago Fintech Ecosystem
[08:59] Scaling Personalization with AI
[15:04] Chat UI Evolution
[20:22] Context Layer in Systems
[26:39] Agent Growth Dynamics
[33:39] Job Evolution with Open Claude
[39:54] AI and Software Costs
[41:50] Wrap up
#AIAgents #Fintech #iFood
-
Sherwood Callaway is the founder of Sazabi (YC P26), the AI-native observability platform built for engineering teams who ship fast. He previously founded and exited a YC company β now he's back, betting that logs are all you need to replace Datadog.
Logs Are All You Need: Rethinking Observability with AI Agents // MLOps Podcast #381 with Sherwood Callaway, the Founder of Sazabi
π What's covered:
πͺ΅ Logs vs. The Three Pillars β Sherwood makes the case that the traditional observability stack (metrics, logs, traces) is overkill. In 2026, with AI agents in the loop, logs alone are sufficient β and dramatically simpler to instrument.
π¨ AI-Generated Alerts, Not AI-Evaluated Alerts β Instead of using AI to triage your noisy alert stream, Sazabi generates the alerts autonomously from your logs and codebase β so you never configure a monitor again.
π€ Agent Sandboxing & Bash Access β How Sazabi gives its AI agent a persistent bash sandbox with CLI tool access, why every other action routes through that sandbox, and how RLS database permissions keep the agent from doing damage.
π§ Agentic Memory via Git β Sazabi's novel approach to persisting agent memory across threads using Git branches β enabling multiple parallel sub-agents to share findings without bloating the context window.
π Multi-Agent Parallelization β How Sazabi spawns sub-agents and background agents on-demand to investigate production issues in parallel, the way Claude Code displays a live to-do list of agent work.
π Why Evals Are Hard (and What They Built Instead) β An honest conversation about the difficulty of evaluating agentic systems, log-based eval proxies, and why Sazabi still doesn't buy third-party eval tooling.
β‘ MCP Servers, Skills Bloat & Context Management β The tradeoffs between MCP servers and local skill files, progressive tool disclosure, and why context window management is the hidden bottleneck in production agent systems.
π― Building a Moat in 2026 β Sherwood and Demetrios debate what a defensible advantage actually looks like when every AI tool can be cloned fast. Spoiler: "We built it first" is not a moat.
π Beta Launch & Who It's For β Sazabi is in closed beta and opening the waitlist. If your team uses Cursor or Claude Code and you have production traffic you can't afford to break, this is built for you.
π Perfect for: AI engineers, SREs, DevOps teams, and founders building production-grade agent systems who are questioning whether their current observability stack is overbuilt.
π Links & Resources
π Sazabi: https://sazabi.com
π Sazabi on Y Combinator: https://www.ycombinator.com/companies/sazabi
πΌ Sherwood Callaway on LinkedIn: https://www.linkedin.com/in/sherwood-callaway
π° SiliconANGLE coverage: https://siliconangle.com/2026/04/08/startup-sazabi-bets-on-logs-and-ai-agents-to-replace-traditional-observability-stacks/
π» MLOps.community: https://mlops.community
β±οΈ Timestamps
[00:00] Genetic Agent Evolution
[00:33] Dethroning Datadog
[03:13] Sazabi vs Traditional Observability
[10:47] MCP vs CLI Paradigm
[15:12] Sandbox Usage for Agents
[24:28] Genetic Prompt Optimization
[32:34] Eval and Agent Spawning
[38:45] RL Environment Tensions
[45:40] Sazabi is hiring!
[46:10] Wrap up
#Observability #AIAgents #DevTools
-
Joe Maionchi (Co-founder & COO) and Rod Christensen (Co-founder & Chief Architect) of RocketRide join the MLOps Community to walk through AIDE β the AI Integrated Development Environment. RocketRide is an open-source AI pipeline platform that lets developers build, debug, and run production-grade agentic AI workflows directly from their IDE, with support for 13+ LLM providers, 8+ vector databases, and full multi-agent orchestration.
AI Is Fast. AI Projects Are Slow. Let's Fix That. // MLOps Podcast #378 with JRocketRide's Joe Maionchi (Co-founder & COO) and Rod Christensen (Co-founder & Chief Architect)A huge shout-out to β¨RocketRideβ© for this collaboration!
π What's covered:
ποΈ Why AI infrastructure needs standardization β how coding agents produce inconsistent "glue code" across projects and why a typed node graph fixes it
β‘ Efficiency AI vs. Opportunity AI β the two paths companies take with generative AI, and which one actually compounds growth
π Multi-agent pipeline orchestration β running CrewAI, LangChain, and DeepAgent side-by-side to benchmark which works best for your use case
π° Cutting LLM costs in half β design-time strategies for routing tasks to cheaper models without sacrificing output quality
π Pipeline observability & debugging β logging every node step in dev and production so you can pinpoint exactly where a 10-step pipeline breaks
πΌοΈ Beyond text: image, video & audio nodes β frame grabbing, OCR, Whisper transcription, and speech-to-text running on shared GPU infrastructure
π RocketRide Cloud β one-click deploy from local to cloud with dynamic GPU scaling and cost-efficient shared inference
π§ Intentionality in agentic development β why moving fast with AI agents creates "crappy code fast" and how skills/context files change the equation
π MCP support & framework-agnostic design β swap any model, tool, or framework without rewritesThis episode is essential for AI engineers, ML practitioners, and developers building production LLM applications who want to stop reinventing infrastructure and start shipping.
π Links & Resources:
β’ RocketRide website: https://rocketride.ai
β’ RocketRide open source (GitHub): https://github.com/rocketride-org/rocketride-server
β’ AIDE VS Code Extension: https://rocketride.org
β’ MLOps Community: https://mlops.community
β’ Discord: https://discord.gg/Hd4PukFt2H
β±οΈ Timestamps
[00:00] Cost Savings in AI
[00:21] AI, Developer, and Software Development Evolution
[02:51] Intentionality in Software Development
[10:51] Model Skill Optimization
[17:08] Primitives in AI Systems
[29:00] Coding Agent Challenges
[37:09] RocketRide Inspiration
[44:42] Coding Agents and Documentation
[47:40] RocketRide Cloud Overview
[56:27] Wrap up
-
BuzzHPC Roundtable episode: Architecting Modern AI Systems: Platforms, Agents, and Integration
Join the Community: https://go.mlops.community/YTJoinIn
Get the newsletter: https://go.mlops.community/YTNewsletter
MLOps GPU Guide: https://go.mlops.community/gpuguide
Big shout-out to BuzzHPC for the collaboration!
// Abstract
As AI systems evolve into more autonomous, agent-driven architectures, the way we design platforms, tools, and infrastructure is rapidly changing. In this session with BuzzHPC, we explore the shifting boundary between platforms and tools, what developers expect platform providers to handle versus what they want to control and build themselves.
We unpack what modern agentic stacks look like today, how teams are structuring them in production, and where these architectures are heading as systems become more complex and distributed. A key focus will also be on agent interoperability, how different agents communicate, coordinate, and operate within shared environments.
Finally, we share insights and lessons from a recent AI hackathon delivered in partnership with Bell, Buzz, Mila, and KHP, highlighting how these concepts are being tested and applied by builders in real-world scenarios.
// Bio
Allen Roush
Allen has held senior technical and AI leadership roles at companies like Oracle and Intel. He's very active in the AI research space and open source communities. He's passionate about improving the creativity and coherence of AI systems.
FrΓ©dΓ©ric BΓ©nard
FrΓ©dΓ©ric is Senior Director of AI Applications Development at Mila (Quebec AI Institute), where he leads a team focused on building the engineering foundations for applied AI systems. His work centers on translating cutting-edge research into scalable applications, including AI-driven platforms and agent-based systems used across research and industry collaborations.
Shuo Wang
Shuo leads the Responsible AI Office for Bell Canada, where all AI use cases are reviewed and assessed for potential harm and bias. Previously, he led a team of data scientists to expand a large-scale ML program to improve customer support effectiveness.
// Related Links
Website: https://www.buzzhpc.ai/
~~~~~~~~ βοΈConnect With Us βοΈ ~~~~~~~
Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore
Join our Slack community [https://go.mlops.community/slack]
Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]
Sign up for the next meetup: [https://go.mlops.community/register]
MLOps Swag/Merch: [https://shop.mlops.community/]
Connect with Demetrios on LinkedIn: /dpbrinkm
Connect with Allen on LinkedIn: /allen-roush-27721011b/
Connect with FrΓ©dΓ©ric on LinkedIn: /benard/
Connect with Shuo on LinkedIn: /shuow/
-
Big news: the MLOps Community is joining the Linux Foundation to become the official user community of the new Agentic AI Foundation (AAIF).
The AAIF is the neutral home for open source projects like the Model Context Protocol (MCP), goose, and AGENTS.md, co-founded by Anthropic, Block, and OpenAI. With that governance and scaffolding now in place, the open source agent ecosystem has room to scale, and the MLOps Community is right in the middle of it.
Everything you love about the community from the past six years keeps going, and we are adding even more on top.
What this means:
- Official user community: MLOps Community becomes the user community of the Agentic AI Foundation under the Linux Foundation.
- The projects: MCP, goose, and AGENTS.md now live under one open, neutral governance structure built to scale.
- Nothing goes away: The podcast, the global meetups, the weekly newsletter, the Slack workspace, and the virtual events all continue.
- New: Ambassador Program: Just opened for applications, so you can get more involved in the community.
- AgentCon EU: September 17 and 18 in Amsterdam.
- AgentCon North America: October 22 and 23 in San Jose.
- A possible new name: The podcast may become "Agentic Conversations," because honestly all we talk about is agents. Tell me what you think in the comments.
If you build with AI agents or follow the open source agent ecosystem, this is the update to bookmark. This is MLOps Community 2.0.
Links and Resources:
- MLOps Community: https://mlops.community
- MLOps Community 2.0: https://mlops.community/blog/mlops-community-2-0
- Agentic AI Foundation: https://aaif.io
- Ambassadors: https://aaif.io/ambassadors
- Linux Foundation AAIF announcement: https://www.linuxfoundation.org/press/linux-foundation-announces-the-formation-of-the-agentic-ai-foundation
- AgentCon and MCPCon events: https://events.linuxfoundation.org/aaif-events/
- Model Context Protocol (MCP): https://modelcontextprotocol.io
- goose: https://goose-docs.ai
- AGENTS.md: https://agents.md
Timestamps (approximate, adjust before publishing):
00:00 The big announcement
00:12 Joining the Linux Foundation's Agentic AI Foundation
00:30 Why it matters: MCP, goose, and AGENTS.md
00:48 What is not changing: podcast, meetups, newsletter, Slack
01:15 What is new: the Ambassador Program
01:30 AgentCon EU in Amsterdam and North America in San Jose
01:55 A new name for the podcast: Agentic Conversations?
02:10 MLOps Community 2.0
#AgenticAI #MCP #LinuxFoundation
-
Guthrie Cooper (Senior Group Product Manager, AI & Robotics) and Nidhi Sharma (Global Head of Engineering AI & Incubation) from Just Eat Takeaway.com join the MLOps.community to pull back the curtain on how one of Europe's largest food delivery platforms is running an internal innovation engine. From autonomous delivery robots to agentic AI voice assistants, they share what it actually takes to build like a startup inside a 40,000-person company.
Inside Just Eat's AI Lab: Voice Agents & Agentic Commerce // MLOps Podcast #377 with Just Eat Takeaway.com's Guthrie Cooper (Senior Group Product Manager, AI & Robotics) and Nidhi Sharma (Global Head of Engineering AI & Incubation)
π€ Delivery Robots β How JET partnered with RIVR and DELIVERS.AI to deploy physical AI ground robots in Zurich, Milton Keynes, and Bristol, and what the first pilots taught the team
π§ AI Incubation at Scale β How Nidhi's team built a dedicated incubation unit to fast-track AI experiments without the red tape of a large enterprise
ποΈ AI Voice Assistant β The story behind JET's new voice-first food ordering experience, and the ML challenges of building a conversational concierge at scale
π¦Ύ Physical AI vs. Software AI β Why deploying wheeled-legged robots in real cities is fundamentally different from shipping a model update, and the MLOps implications
π Corporate Innovation Playbook β The frameworks Guthrie and Nidhi use to move from idea to pilot in weeks, not quarters, inside a large org
π¦ Innovation as a Platform β How JET is thinking about turning its delivery infrastructure and AI capabilities into a reusable platform for new business lines
π Startup Partnerships β What makes a good external innovation partner (vs. building in-house), and how JET evaluates robotics and AI startups for pilots
β‘ Agentic AI & Accessibility β How agentic AI is being used to make food ordering genuinely accessible for blind and low-vision users
Whether you're an ML engineer at a large company trying to get AI into production, a product leader navigating corporate innovation, or a startup founder looking to partner with a platform player β this conversation is packed with practical lessons.
π Links & Resources:
Just Eat Takeaway.com: https://www.justeattakeaway.com
RIVR (physical AI delivery robots): https://www.rivr.ai
DELIVERS.AI (UK delivery robots): https://www.delivers.ai
Prosus (JET parent company): https://www.prosus.com
MLOps.community: https://mlops.community
β±οΈ Timestamps
[00:00] AI Innovation Incubator Strategy
[03:16] Everyday Convenience Expansion
[07:03] Context Ownership in Ecosystems
[17:35] LLM Integration and Discovery
[24:02] Whoop Notifications Grievances
[33:01] Expanding Beyond Food
[48:20] Innovation Lab Failures
[51:22] Rory Sutherland's Alchemy
[1:03:23] Latency and Conversational Design
[1:13:42] Drone Delivery Efficiency
[1:18:06] Wrap up
#AgenticCommerce #VoiceAI #DroneDelivery
-
Pramod Krishnan is a Managing Director - AI Managed Services at PwC, specializing in enterprise AI transformation β helping large organizations move from AI experimentation to production operating models. In this episode with Demetrios, Pramod breaks down exactly what the OpenClaw wave means for enterprises, and the control frameworks PwC uses before a single agent touches production.
Huge thanks to β PwCβ for supporting this episode!
Autonomous Agents at Work: From OpenClaw Hype to Enterprise Reality // MLOps Podcast #378 with Pramod Krishnan, Managing Director - AI Managed Services at PwC US.
π OpenClaw & the Agentic Hype Cycle β Why the fastest-growing open-source agent project in history (190K+ GitHub stars in weeks) is a forcing function for enterprise AI governance, and what most organizations are getting wrong.
ποΈ 3-Tier Work Classification β Pramod's framework for categorizing any agentic task as reversible, sensitive, or consequential β and how the approval gates, controls, and blast radius differ for each tier.
π‘οΈ The Guardrails Stack β A concrete list of non-negotiable guardrails: allow-listed tool calls, prompt injection defense, credential protection, toxic output filtering, and more β straight from PwC's production deployments.
π 5-Part Auditability Framework β How to make AI agents truly auditable across quality (LLM-as-judge), performance, safety, cost, and security β and why OpenTelemetry alone isn't enough.
π° Agent Cost & ROI Tracking β Why successfully deployed agents are generating the hardest financial measurement problems enterprises have ever faced, and what a real cost-tracking architecture looks like.
π Agent Security in Depth β From API key harvesting attacks to credential leakage to malicious actor scenarios: what security controls PwC requires before any agent goes live.
βοΈ The Minimum Control Stack β The non-negotiables Pramod would walk in with on a Monday before clearing any agent for production: what they are, why they matter, and how to implement them.
π Human-in-the-Loop Design β The difference between "human in the loop" (approves every action) and "human on the loop" (monitors and intervenes) β and how to choose the right pattern based on consequence level.
π€ AI as a Force Multiplier β How Pramod thinks about AI ownership, intellectual authorship, and making sure humans remain deliberate and responsible even as agents accelerate output.
This episode is essential for ML engineers, platform architects, CIOs, and AI product managers who are moving beyond demos into real enterprise agentic deployments.
π Links & ResourcesPramod Krishnan on LinkedIn: https://www.linkedin.com/in/pramod-potti-krishnan/
MLOps.community: https://mlops.community
OpenClaw project: https://openclaw.ai
BCG on OpenClaw + Enterprise: https://www.bcg.com/publications/cios-openclaw-and-the-new-wave-of-ai-agents
PwC 2026 AI Business Predictions: https://www.pwc.com/us/en/tech-effect/ai-analytics/ai-predictions.html
Timestamps:
[00:00] AI in Enterprise
[02:04] AI System Failures
[08:01] Agent Decision Tracing
[13:07] Agent Design Tension
[16:21] Agent Control Stack Essentials
[20:20] LLM Cost and FinOps
[26:16] Agent Attack Surfaces
[30:00] Tools as Attack Vectors
[33:47] Human in the Loop
[37:00] AI Ownership and Accountability
[41:42] Wrap up. Shoutout to Pramod and PwC!
-
Hamza Tahir, co-founder of ZenML, joins the show to cut through the hype around long-running agents β arguing that at the end of the day, an agent is just a while loop that talks to a model, calls a tool, and writes to a file system. He covers the architecture of agent harnesses (inner and outer), what durable execution actually guarantees (and what it doesn't), and why the ML pipeline paradigm is a cleaner mental model than transactions for most agent workloads.
Hamza also announces Kitaru β ZenML's new open-source execution runtime for async Python agents β built on five years of running ML workloads in enterprise environments.
What we get into:
Agents are while loops: The surprising simplicity under all the tooling: a brain (LLM), hands (tool calls), and a file system, stacked recursively
Inner harness vs outer harness: Why Pydantic AI owns the inner loop while production deployment needs a separate runtime layer
What "long-running" actually means: Why the infrastructure we need to build is about extrapolating the future, not defining a time window today
Durable execution demystified: What checkpointing actually guarantees (infra failures, pod death, network drops) vs. what it never will (external state, bad LLM outputs, Snowflake rollbacks)
ML pipelines vs transactions: Why bursty containers in Kubernetes map more naturally to agent workloads than microsecond-latency queue workers β and why Hamza argues against the complexity tax
Anthropic opening the harness: Why letting other models run Claude Cowork is a "boss move," and what it means for the one-harness vs one-model debate
Human-in-the-loop, done right: The pod-kill-and-resume pattern, and why warm pools matter less when your agent runs for days
Kitaru: ZenML's new open source durable execution runtime: zero-config local, Kubernetes/SageMaker/Vertex in production, built on Pydantic AI integration
Arguing with Claude about Temporal: Hamza's story of spending hours getting an LLM to admit ZenML and Temporal solves the same problem
If you're architecting agents for production, picking between Pydantic AI, LangGraph, and Temporal, or just want to understand what "durable execution" actually means β this is the episode.
// LINKS & RESOURCES
Kitaru on GitHub: https://github.com/zenml-io/kitaru
Kitaru launch blog post: https://www.zenml.io/blog/kitaru-launch
Kitaru on Hacker News: https://news.ycombinator.com/item?id=47520115
Hamza Tahir on LinkedIn: https://www.linkedin.com/in/hamzatahirofficial/
ZenML: https://www.zenml.io/
Timestamps
[00:00] While Loop Checkpointing
[00:24] Long-Running Agents Explained
[01:28] Agent Harness Model Definitions
[06:30] Durability and State Recovery
[11:03] Agent Systems Layers
[18:45] Durability in Agent Systems
[22:07] ML Pipeline vs Transactions
[29:23] Durability vs Guarantees
[33:13] Durability vs Chaos Engineering
[39:50] Kitaru Naming and Purpose
[40:38] Wrap up
#AIAgents #DurableExecution #OpenSource
-
Rafael (Head of Innovation, iFood) and Daniel (Data and AI Manager, iFood) pull back the curtain on ILO-Agent β iFood's conversational AI ordering system built for 200 million users across Latin America. Recorded live at AI House Amsterdam, this conversation goes deep into the engineering and product decisions behind building recommendation systems and agentic AI, and why the speed of your AI's response might actually be destroying user trust.
The Latency Goldilocks Zone Explained // MLOps Podcast #376 with iFood's Rafael Borger (Head of Innovation) and Daniel Wolbert (Data and AI Manager)
π Recommendation Systems at Scale β Why personalizing for 200M users with wildly different food tastes, budgets, and cultures is a fundamentally different problem than standard ML
π€ ILO-Agent Deep Dive β What iFood's conversational AI agent actually does, how it handles open-ended requests ("a romantic dinner for two, my wife hates onions"), and where it's headed
β±οΈ The Latency Goldilocks Zone β The fascinating insight that LLM responses can be too fast (users don't trust them) or too slow (users abandon) β and how to find the sweet spot
π§ Perceived vs. Actual Latency β Why showing progress indicators and partial results can make a 6-second response feel instant, and how iFood uses this in production
π The Tinder for Food Experience β How iFood is experimenting with swipe-based discovery to solve "I don't know what I want to eat" for millions of undecided users
π£οΈ Voice vs. Text AI Interfaces β Why voice ordering limits you to 6 items in 30 seconds, and why text-based agents need radically different output design
π Agent-to-Agent (A2A) Architectures β What happens when your customer support agent and your ordering agent need to collaborate, and the standardization challenges ahead
π Measuring Product-Market Fit for AI β Why the Sean Ellis / Chanel score method breaks down in Brazil, and what iFood uses instead
ποΈ Scalability vs. Ecosystem Health β The real tension between consuming partner APIs aggressively and keeping the food delivery ecosystem sustainable
π Building AI for Global-Local Markets β Why one-size-fits-all AI products fail and how iFood builds for cultural and economic diversity simultaneously.
This episode is for ML engineers, AI product managers, and data scientists building production AI systems at scale β especially if you're working on recommendation, retrieval, or agentic systems in consumer apps.
π Links & Resources
MLOps.community: https://mlops.community
AI House Amsterdam: https://aihouse.amsterdam
iFood: https://www.ifood.com.br/
iFood AILO launch coverage: https://tiinside.com.br/en/10/10/2025/ifood-lanca-ailo-assistente-de-ia-que-inaugura-pedidos-por-conversa/
iFood AI case study (AWS): https://aws.amazon.com/solutions/case-studies/ifood-bedrock/
Related MLOps Community talk β "From Zero to AILO" by Nishikant Dhanuka & Chiara Caratelli: https://home.mlops.community/public/videos/from-zero-to-ailo-lessons-learned-from-building-ifoods-ai-agent-nishikant-dhanuka-and-chiara-caratelli-2025-11-25
ZenML LLMOps database write-up on iFood's hyper-personalized agent: https://www.zenml.io/llmops-database/building-a-hyper-personalized-food-ordering-agent-for-e-commerce-at-scale
β±οΈ Timestamps
[00:00] Recommending the unknown
[00:18] Ailo Hyperpersonalization Insight
[06:24] Predictive Personalization Insights
[09:13] "Jet skis" of innovation
[17:45] Consumer Behavior and Chatbots
[26:33] Perceived Latency and Engagement
[33:22] AI-driven UI Evolution
[38:17] LCM Voice Mode Inquiry
[45:20] Chat as Interface
[47:46] Wrap up
-
Nicolas Alejandro Bogliolo is the AI PM at Despegar, the largest online travel agency in Latin America, and the engineer-product-hybrid behind Sofia, the GenAI travel concierge that beat most of the OTA world to a working multi-agent system.
Before MCP was a standard and before LangChain was widely adopted, his team had already shipped their own orchestration layer and tool protocol in production. This conversation is a rare look at what it takes to build an agentic system that actually books trips, runs on WhatsApp, and keeps adding capabilities without falling over.
Building MCP Before MCP Existed: Inside Despegar's Sofia Agent // MLOps Podcast #375 with Nicolas Alejandro Bogliolo, AI PM at Despegar
What we cover:
- Chappi, the brain of Sofia: how Despegar built an internal orchestration layer when there was nothing off the shelf- Building "MCP before MCP": the custom tool-calling protocol that predated the Anthropic standard- Multi-agent architecture by vertical: flights, hotels, activities, and cars each own their own flow
- Decentralized agent ownership: how any squad in the company can build a flow with central supervision
- Sofia on WhatsApp: making messaging the consumer control center, the way Slack became it for the enterprise
- The five-phase travel arc Sofia covers: dreaming, planning, anticipation, in-trip, and post-trip
- KPI evolution: why "in-scope conversation rate" topped out near 96 percent and what they measure now
- The flight-delay-claim use case and why filing claims through a chatbot is a perfect agent task
- Group trip planning in WhatsApp groups: the next frontier for travel agents
- Sofia as channel of choice: the WeChat-style vision for an agent that handles your entire trip
- Why Despegar held off on giving Sofia the ability to bargain with customers, for now.
Whether you are building production agents, running an OTA, or just curious about how an AI travel concierge actually works under the hood, this episode is full of grounded, in-production lessons from a team that had to invent the patterns the rest of us are now adopting.
Links and Resources:
Despegar: https://www.despegar.com
Sofia announcement: https://investor.despegar.com/news-presentations/news-releases/news-details/2024/Despegar-revolutionizes-the-tourism-industry-introducing-the-regions-first-Generative-AI-Travel-Assistant
Sofia coverage on PhocusWire: https://www.phocuswire.com/despegar-debuts-genai-travel-assistant-remembers-previous-interactions
MLOps Community: https://mlops.community
Subscribe for more agent and AI infra deep dives
Timestamps
[00:00] Sophia Travel Concierge AI
[00:38] Sophia Multi-Agent System
[06:00] AI Limitations in Practice
[13:52] Travel Planning Exploration
[18:03] Group Travel Decision Making
[21:32] Agent Ecosystem Design
[30:14] Sofia's Travel Assistant Vision
[33:35] Orchestration and MCP Design
[40:13] Sophia Negotiation Concerns
[40:47] Wrap up
#AIAgents #MCP #AgenticAI
-
This episode is brought to you by the MLflow team. Check out more information at MLflow.org.
What does it actually take to build voice AI at a billion-interaction scale? This episode features an ex-Amazon voice AI engineer who built customer support systems handling 2 billion+ interactions β now working on next-gen voice agent platforms. Anurag digs deep into the real engineering tradeoffs, design patterns, and use cases that separate production-grade voice agents from demos.
Voice Agent Use Cases // MLOps Podcast #374 with Anurag Beniwal, Member of the Technical Staff at ElevenLabs
ποΈ Topics covered:
πΉ Cascaded vs. speech-to-speech β Why cascaded systems still win in production, and how to make them feel natural without sacrificing control
πΉ Latency masking β Foreground/background model architecture and how to buy yourself time while deep retrieval runs
πΉ Constellation of models β Using Haiku for tool calling, fine-tuned smaller models for response generation, and why "one model for everything" breaks at scale
πΉ Turn-taking & ASR challenges β Why voice is harder than chat: accents, noise, silence detection, and domain-specific fine-tuning
πΉ Level 1 vs Level 2 customer support β Why today's agents max out at Level 1 and what it takes to capture Level 2 expert judgment
πΉ Inbound vs. outbound sales agents β Where voice agents are already winning, and why inbound lead qualification beats cold outbound
πΉ Booking, reservations & concierge β The clearest near-term wins for voice agents across hospitality, home services, and SMBs
πΉ Continual learning from natural language feedback β How to build agents that improve from real operator feedback without ML expertise
πΉ Conversational TTS β Why passing full conversation history to your TTS model changes everything for tone consistency
πΉ User tiers for voice platforms β Non-technical business owners vs. developers vs. enterprise: why one interface doesn't fit all.
If you're building production voice agents, evaluating voice AI vendors, or scaling AI-first customer support β this episode is packed with hard-won lessons from someone who's done it at Amazon scale.
π Links & Resources:
MLOps.community: https://mlops.communityGoogle Scholar: https://scholar.google.com/citations?user=g_QB5WgAAAAJ&hl=en&o
Amazon science page: https://www.amazon.science/author/anurag-beniwal
Join the Community: https://go.mlops.community/YTJoinIn
Get the newsletter: https://go.mlops.community/YTNewsletter
MLOps GPU Guide: https://go.mlops.community/gpuguide
β±οΈ Timestamps
[00:00] Cascaded Systems Control Challenge
[05:35] Voice vs Chat Complexity
[14:16] MLflow's open source platform
[15:03] AI Model Constellations
[23:00] Model Constellations Use Cases
[31:40] Voice vs Text Context
[33:54] Voice as Thought Capture
[42:11] Cascaded vs Speech-to-Speech Debate
[50:02] Wrap up
-
Jesse Vincent is the Founder & CEO of Prime Radiant and creator of Superpowers β the most-used Claude Code plugin in the world. He built the first agentic software development methodology from scratch while managing MIT interns in the early 2000s, and hasn't written a line of code manually since October.
The Creator of Superpowers: Why Real Agentic Engineering Beats Vibe Coding // MLOps Podcast #373 with Jesse Vincent, Founder & CEO of Prime Radiant
In this conversation, Jesse walks Demetrios through the full Superpowers system: why he thinks most developers are still approaching agentic coding wrong, how he designs skills that force LLMs to stop rationalizing and actually follow rules, and what he's building next at Prime Radiant β including Green Field, an unreleased tool for reverse-engineering legacy codebases into specs. This one is for developers who want to go beyond "vibe coding" and build AI-assisted workflows that actually scale.
π§ Topics Covered
π§ The Superpowers Methodology β How the brainstorming skill extracts what you actually want before you hand work to an agent, and why most developers skip this step
π Spec-Driven Development & Plan Files β Why Jesse insists on TDD, DRY, and YAGNI for every agentic task, and how planning skills generate per-task context blocks agents can actually execute on
π Debugging with Agents β Jesse's systematic approach to root cause analysis, reproduction cases, and the 30 years of debugging instinct he's baked into a skill
π Pressure Testing LLM Skills β How Claude fires up sub-agents and stress-tests its own rules to catch rationalization before it shows up in production
π οΈ Clearance IDE β Jesse's new Markdown-native development environment built for humans working alongside AI, with a history pane for file navigation
π¦ Green Field (Unreleased) β A toolset for turning old codebases or built products into clean specs β not yet public but dropping soon from Prime Radiant
π§βπΌ Management as the Magic Trick β Why the real unlock of tools like Superpowers is that they make every developer a manager, and why that transition is hard the first time
βοΈ Software Ethics in the Agent Era β Reverse engineering, license washing, open source cloning, and whether the value of software itself is collapsing
π Links & Resources
Prime Radiant: [https://prime-radiant.com](https://prime-radiant.com/)
Superpowers on GitHub: https://github.com/prime-radiant-inc
Clearance IDE: https://github.com/prime-radiant-inc (check repo)
MLOps.community Slack: https://go.mlops.community/slack
MLOps.community website: [https://mlops.community](https://mlops.community/)
β±οΈ Timestamps
[00:00] Greenfield Toolset Insights
[00:27] Superpowers Kit Evangelism
[08:06] Hyperbolic's GPU Cloud
[17:48] Debugging Skill Creation
[22:12] Skill Extraction Strategy
[31:15] Smallest Harness
[41:06] Software supply chains
[48:56] Visual Precision Challenges
[54:09] Creative Feedback Loops
[1:04:24] MLflow's Gen AI
[1:05:55] Wrap-up
-
Maggie Konstanty is an AI Product Manager at Prosus, one of the world's largest consumer internet companies, where she builds and evaluates AI agents for food ordering and ecommerce at scale. She's been inside the messy reality of LLM evaluation longer than most β and her take is unfiltered.
It's 2026, and We're Still Talking Evals // MLOps Podcast #372 with Maggie Konstanty, AI Product Manager at Prosus
π§ͺ Why accuracy metrics lie β Maggie breaks down why "95% accurate" tells you almost nothing about whether your agent is actually working in the real world, and what to measure instead.
ποΈ Pre-ship vs. production evals β Your eval suite before launch will not survive first contact with real users. Maggie explains the structural disconnect and how to close the gap.
π» The silent failure: user drop-off β Users who are unhappy don't complain β they just leave. Discover why drop-off analytics are one of the most underutilized eval signals in production.
π― Instruction to fail: the 20-evaluator trap β Setting up 20 types of evaluators not connected to your product goal is a fast path to wasted time. How to design evals that are tied to real outcomes.
π½οΈ The "surprise me" edge case β A real example from Prosus's food ordering agent and what it reveals about how users actually behave vs. how PMs imagine they do.
π€ LLM-as-a-judge: the limits β Why Maggie doesn't lean on LLM-as-a-judge for accuracy measurement, and what approaches she uses instead for production-grade evaluation.
π οΈ Arize/Phoenix & eval tooling critique β A candid take on the current state of eval platforms, why she spent a whole day fighting the UI, and why mature teams often go back to custom code.
𧬠Eval as team DNA β Evals aren't a launch checklist. Maggie makes the case that they need to be a constant practice embedded in team culture β and why alignment on "what good looks like" is harder than any technical implementation.
π’ When to stop optimizing β What happens when your eval score approaches 100%, and how to know when it's time to shift focus to a different metric or flow.
π¬ Red teaming with incentives β A fun tactic: running adversarial eval sessions where engineers compete to break your agent for an Amazon gift card.
This is required watching for AI PMs, ML engineers, and applied AI teams who have moved past "getting evals set up" and are now struggling with making them actually matter.---
π Links & Resources
Maggie Konstanty on LinkedIn: https://www.linkedin.com/in/maggie-konstanty
Prosus: [https://www.prosus.com](https://www.prosus.com/)
MLOps.community: [https://mlops.community](https://mlops.community/)
Arize AI / Phoenix (mentioned): [https://arize.com](https://arize.com/) / [https://phoenix.arize.com](https://phoenix.arize.com/)
MLOps.community Slack: https://go.mlops.community/slack
β±οΈ Timestamps
[00:00] Evaluations and User Alignment
[00:18] Eval Lifecycle in Production
[06:05] LLM Accuracy and Judging
[15:30] Evals vs Tests in AI
[22:39] Profanity as Frustration Signal
[29:23] Impact-weighted performance
[32:22] Eval Tooling Pros and Cons
[38:10] Build vs Buy Dilemma
[39:35] Wrap up
-
This episode is brought to you by Hyperbolic and the MLflow team. Check out more information at hyperbolic.ai and MLflow.org.
Why AI Coding Agents Are Moving to the Cloud β With Zach Lloyd, CEO of Warp
Zach Lloyd is the founder and CEO of Warp, the AI-native terminal and agentic development platform trusted by over a million developers. Before Warp, Zach was a product lead at Google on Google Docs β giving him a uniquely deep intuition for what it means to build truly collaborative developer tools at scale.
Why Agents are Driving Software Development to the Cloud // MLOps Podcast #371 with Zach Lloyd, CEO of Warp
What we cover:
ποΈ Why agents belong in the cloud, not local sandboxes β Zach breaks down why the "set up a local dev box for your agent" approach is fundamentally flawed and what cloud-native agent execution actually looks like in practice.
π GitHub is losing collaborative code review β One of the episode's sharpest takes: the hero features of GitHub, like collaborative code review, are migrating into agent workbenches. Zach explains why this shift is structural, not cyclical.
π± "Just-in-time apps" are replacing SaaS β The era of long-lived, learn-to-use-it software may be ending. Zach argues that agents will generate ephemeral, purpose-built interfaces on demand β and why most current app categories are at risk.
π€ Introducing Oz β Warp's cloud orchestration platform β A first look at how Oz works, how Demetrios is already using it to automate podcast production, and what multi-agent orchestration looks like in a real team environment.
ποΈ Agent observability and why it matters β Debugging, compliance, context management, and handoff/steering: Zach outlines the three pillars every engineering team needs before trusting agents with production work.
π Agent chaos is real β access control for AI β Why giving agents too much context is just as dangerous as giving them too little, and how Warp thinks about scoped agent permissions as you scale.
π¦ SaaS for agents will look nothing like SaaS for humans β The 25-year investment in human-friendly UI is irrelevant for agents. Zach explains what the new infrastructure layer for AI workers will actually need.
β‘ Open-weight models will commoditize the coding agent space β With Nvidia investing $2B in open-weight models, Zach believes the current cost advantage that frontier labs hold is temporary β and how Warp is positioning for that world.
𧩠Multi-agent orchestration patterns β Parallel agents, agent-to-agent handoffs, and why there's no single "right" pattern yet. Warp's Oz platform is being built for flexibility, not prescription.
This episode is essential for engineering leaders, platform engineers, and any developer trying to understand where their daily workflow is headed in the next 18 months.
π Links & Resources:
Warp: https://www.warp.dev
Warp Oz platform: https://oz.dev
Zach Lloyd on X/Twitter: https://x.com/zachlloyd
MLOps Community: https://mlops.community
MLOps Community Slack: https://go.mlops.community/slack
β±οΈ Timestamps
[00:00] Agentic Coding Review Shift
[00:29] Warp Collaboration vs Sandboxes
[05:22] Continuous Co-Creation in Teams
[07:00] Hyperbolics GPU Cloud
[07:56] Skill Governance Framework
[14:41] Agents vs Browsers Analogy
[21:31] PR Provenance in Warp
[27:58] Agent System Commandments
[37:44] Harness vs ADE
[42:03] Adversarial Review Technique
[45:26] GitHub Limitations for Agents
[49:07] MLflow's GenAI
[50:06] Wrap up
-
This episode is brought to you by the MLflow team. Check out more information at MLflow.org.
Mihail Eric is Head of AI at Monaco and Adjunct Lecturer at Stanford University, where he teaches CS146S: "The Modern Software Developer" β the first course in the world dedicated to how AI is transforming every stage of the software development lifecycle. With 12+ years building production AI systems at Amazon Alexa, Storia AI (YC S24), and early-stage startups, Mihail has one of the most grounded, practitioner-level takes on what it actually means to be a software engineer in 2026.
The Modern Software Engineer // MLOps Podcast #370 with Mihail Eric, Head of AI at Monaco
π§ What the modern software engineer actually looks like β why the job description has fundamentally shifted from writing code to designing systems and directing agents
βοΈ Agents require more thinking, not less β why the engineers getting the most out of coding agents are the ones who invest the most upfront in architecture, planning, and codebase structure
π Inside Stanford's "Modern Software Developer" course β what Mihail teaches in the first CS course in the world focused entirely on AI-transformed software development
ποΈ From writing code to designing systems β how the best developers are repositioning themselves as architects of agentic workflows rather than line-by-line coders
π The Build System: how to run agents at scale β practical lessons from building multi-agent pipelines, parallel subagent batches, and automated retrospectivesπ What junior engineers should actually focus on β the skills that remain irreplaceable and the paths that still produce strong software engineers in an AI-first world
π Building Monaco's AI-native revenue engine β what it's like building AI infrastructure for a fast-moving $35M-funded startup disrupting enterprise CRM
π― How to ace AI engineering interviews β Mihail's framework for demonstrating real AI engineering competence beyond prompt engineering basics. Essential watching for software engineers, ML practitioners, and engineering managers who want an honest, practitioner-level view of where the profession is going β from someone who's both teaching it at Stanford and building it in production.
π Links & Resources
Mihail Eric on LinkedIn: https://www.linkedin.com/in/mihaileric/
Mihail's website: https://www.mihaileric.com
Stanford course "The Modern Software Developer": https://themodernsoftware.dev/
Maven course β AI Software Development: From First Prompt to Production Code: https://maven.com/the-modern-software-developer/ai-course
Free AI Engineer interview prep course: https://course.aiengineermastery.com/
Monaco (AI-native revenue engine): https://monaco.com
MLOps.community Slack: https://go.mlops.community/slack
β±οΈ Timestamps
00:00 Intro β Mihail Eric & Monaco
04:00 What has actually changed for software engineers in 2026
09:00 Inside Stanford's "Modern Software Developer" course
15:00 Why agents require more human thinking, not less
21:00 From writing code to designing systems β the architect mindset
27:00 The Build System: running agents at scale in production
33:00 What junior engineers should focus on right now
39:00 Building AI infrastructure at Monaco
44:00 How to demonstrate real AI engineering competence
49:00 Skills that will remain irreplaceable
52:00 Rapid fire/closing thoughts
-
Maher Hanafi is an engineering leader who went from zero AI experience to self-hosting LLMs at enterprise scale β managing GPU costs, optimizing inference with TensorRT LLM, and building an AI platform for HR tech. In this conversation, he breaks down exactly how his team cut latency by 70%, reduced GPU spend through counterintuitive scaling strategies, and navigated the messy reality of taking AI from proof-of-concept to production.
How We Cut LLM Latency 70% With TensorRT in Production // MLOps Podcast #369 with Maher Hanafi, SVP of Engineering at Betterworks
Key topics covered:
The AI Iceberg β Why the invisible work behind AI (performance, latency, throughput, cost, accuracy) is harder than building the features themselves
GPU Cost Optimization β How upgrading to more expensive GPUs actually saved money by reducing total runtime hours
TensorRT LLM Deep Dive β Rewiring neural networks to match GPU architecture for 50-70% latency reduction
Cold Start Solutions β Using AWS FSx, baking models into container images, and cutting minutes off spin-up times
KV Cache & In-Flight Batching β Why using one model per GPU with maximum KV cache beats cramming multiple models together
Scheduled & Dynamic Scaling β Pattern-based scaling for HR tech workloads (nights, weekends, end-of-quarter spikes)
Verticalized AI Platform β Building horizontal AI infrastructure that serves multiple HR product verticals
AI Engineering Lab β How junior vs. senior engineers adopted AI coding tools differently, and the cultural shift that followed
Agentic Coding in Practice β Navigating AI coding agent costs, quality control, and redefining the SDLC
Chinese Models & Compliance β Why enterprise customers block DeepSeek/Qwen and the geopolitics of model training data
This episode is for engineering leaders building AI in production, MLOps engineers optimizing GPU infrastructure, and anyone navigating the gap between AI demos and enterprise-scale deployment.
Links & Resources:
TensorRT LLM: https://github.com/NVIDIA/TensorRT-LLM
NVIDIA Run: ai Model Streamer (cold start optimization): https://developer.nvidia.com/blog/reducing-cold-start-latency-for-llm-inference-with-nvidia-runai-model-streamer/
vLLM vs TensorRT-LLM comparison: https://northflank.com/blog/vllm-vs-tensorrt-llm-and-how-to-run-them
Timestamps:
[00:00] Optimizing GPU Usage and Latency
[00:21] Learning AI as Leadership
[04:34] AI Cost Centers
[13:56] Throughput and Infrastructure Efficiency
[18:10] Scaling and Unit Economics
[24:14] Championing AI ROI
[36:11] Queue to Value Engine
[41:30] Failed Product Features
[46:12] Agentic Engineering Costs
[58:49] AI Self-Hosting in Engineering
[1:04:40] Wrap up
-
Rob Ennals is the creator of Broomy, an open-source IDE designed for working effectively with many agents in parallel. He previously worked at Meta, Quora, Google Search, and Intel Research. He has a PhD in Computer Science from the University of Cambridge.
Getting Humans Out of the Way: How to Work with Teams of Agents // MLOps Podcast #368 with Rob Ennals, the Creator of Broomy
Join the Community: https://go.mlops.community/YTJoinIn
Get the newsletter: https://go.mlops.community/YTNewsletter
MLOps GPU Guide: https://go.mlops.community/gpuguide
// Abstract
Most people cripple coding agents by micromanaging themβreviewing every step and becoming the bottleneck.
The shift isnβt to better supervise agents, but to design systems where they work well on their own: parallelized, self-validating, and guided by strong processes.
Done right, you donβt lose controlβyou gain leverage. Like paving roads for cars, the real unlock is reshaping the environment so AI can move fast.
// Bio
Rob Ennals is the creator of Broomy, an open-source IDE designed for working effectively with many agents in parallel. He previously worked at Meta, Quora, Google Search, and Intel Research. He has a PhD in Computer Science from the University of Cambridge.
// Related Links
Website: https://robennals.org/
https://broomy.org/
https://learnai.robennals.org/ (not yet announced, but should be by the time of the podcast)
~~~~~~~~ βοΈConnect With Us βοΈ ~~~~~~~
Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore
Join our Slack community [https://go.mlops.community/slack]
Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]
Sign up for the next meetup: [https://go.mlops.community/register]
MLOps Swag/Merch: [https://shop.mlops.community/]
Connect with Demetrios on LinkedIn: /dpbrinkm
Connect with Rob on LinkedIn: /robennals/
Timestamps:
[00:00] Agent Optimization Strategies
[00:21] Visual Regression Explanation
[05:35] Automated QA for Videos
[13:05] Verification System Design
[19:48] Agent Selection Strategies
[30:48] Parallel Agent Management
[35:30] Containerization and Cost Estimation
[42:48] Shifting to Agent Orchestration
[50:10] Wrap up
-
Kashish Mittal is a Staff Software Engineer at Uber, working on large-scale distributed systems and core backend infrastructure.
Fixing GPU Starvation in Large-Scale Distributed Training // MLOps Podcast #367 with Kashish Mittal, Staff Software Engineer at Uber
Join the Community: https://go.mlops.community/YTJoinIn
Get the newsletter: https://go.mlops.community/YTNewsletter
MLOps GPU Guide: https://go.mlops.community/gpuguide
// Abstract
Kashish zooms out to discuss a universal industry pattern: how infrastructureβspecifically data loadingβis almost always the hidden constraint for ML scaling.
The conversation dives deep into a recent architectural war story. Kashish walks through the full-stack profiling and detective work required to solve a massive GPU starvation bottleneck. By redesigning the Petastorm caching layer to bypass CPU transformation walls and uncovering hidden distributed race conditions, his team boosted GPU utilization to 60%+ and cut training time by 80%. Kashish also shares his philosophy on the fundamental trade-offs between latency and efficiency in GPU serving.
// Bio
Kashish Mittal is a Staff Software Engineer at Uber, where he architects the hyperscale machine learning infrastructure that powers Uberβs core mobility and delivery marketplaces. Prior to Uber, Kashish spent nearly a decade at Google building highly scalable, low-latency distributed ML systems for flagship products, including YouTube Ads and Core Search Ranking. His engineering expertise lies at the intersection of distributed systems and AIβspecifically focusing on large-scale data processing, eliminating critical I/O bottlenecks, and maximizing GPU efficiency for petabyte-scale training pipelines. When he isn't hunting down distributed race conditions, he is a passionate advocate for open-source architecture and building reproducible, high-throughput ML systems.
// Related Links
Website: https://www.uber.com/
Getting Humans Out of the Way: How to Work with Teams of Agents // MLOps Podcast #368 with Rob Ennals, the Creator of Broomy: https://www.youtube.com/watch?v=ie1M8p-SVfM
~~~~~~~~ βοΈConnect With Us βοΈ ~~~~~~~
Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore
Join our Slack community [https://go.mlops.community/slack]
Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]
Sign up for the next meetup: [https://go.mlops.community/register]
MLOps Swag/Merch: [https://shop.mlops.community/]
Connect with Demetrios on LinkedIn: /dpbrinkm
Connect with Kashish on LinkedIn: /kashishmittal/
Timestamps:
[00:00] Local dataset caching
[00:30] Engineers Evolving Roles
[04:44] GPU Resource Management
[10:21] GPU Utilization Issues
[21:49] More GPU War Stories
[32:12] Model Serving Issues
[39:58] Reflective Learning in Coding
[43:23] Workflow and Reflective Skills
[52:30] Wrap up
- Laat meer zien
