How 80,000 companies build with AI: products as organisms, the death of org charts, and why agents will outnumber employees by 2026 | Asha Sharma (CVP of AI Platform at Microsoft)
Summary
In this insightful episode, Asha Sharma, CVP of AI Platform at Microsoft, discusses how over 80,000 companies are building AI products today, unveiling key transformations reshaping product development and organizational structures. One central theme is the shift from viewing products as static artifacts to dynamic, evolving organisms that continuously improve via data-driven feedback loops, or 'metabolic loops.' Sharma emphasizes post-training—fine-tuning AI models with proprietary and synthetic data—as becoming more critical than pre-training massive foundational models, enabling companies to build competitive moats through customized AI. She introduces Microsoft's novel 'seasons' planning framework, which replaces static roadmaps with flexible, adaptive strategies that respond rapidly to fast-paced AI innovation waves, such as GPT-5’s emergence and the rise of AI agents. The concept of the 'agentic society' is explored, where autonomous AI agents drastically outnumber human employees by 2026, transforming rigid hierarchical org charts into fluid, task-centric work charts and networks, with significant implications for workforce design and management. The rise of full-stack AI builders—a new breed of polymaths combining skills across product, design, and AI engineering—is reshaping team roles and accelerating go-to-market timelines. UI paradigms are evolving dramatically; static graphical user interfaces (GUIs) give way to code-native and stream-based text interfaces better suited for interaction with large language models and agent-based systems, enabling on-the-fly adaptation and personalization. Sharma also underscores the importance of embedding AI where users already work—integrating AI agents into existing tools like Microsoft Copilot—rather than creating isolated AI silos. Scaling AI-driven agent workforces demands robust infrastructure for observability, evaluation, fine-tuning, and self-healing to maintain quality at massive scale. Case studies such as Dragon Medical One demonstrate how expert-annotated data and continuous optimization significantly boost AI acceptance and performance, highlighting the new operational KPIs required for AI product teams. Finally, Sharma reflects on leadership cultural lessons from Satya Nadella and discusses ongoing debates about AI’s role in strategic planning, model scaling economics, and the coexistence of diverse AI interaction modes. The episode thoroughly addresses the current challenges and opportunities in AI product strategy, organizational evolution, and technology adoption critical to both enterprises and startups.
Key Takeaways
- 1Products are evolving from static artifacts into dynamic organisms that continuously learn and improve through ongoing interactions and fine-tuning of AI models.
- 2Post-training, including fine-tuning models with proprietary and synthetic data, is becoming more crucial than upfront pre-training of large foundational models for building sustainable competitive advantages.
- 3The advent of an 'agentic society' signifies a future where AI agents outnumber human employees, reshaping organizations from hierarchical charts into fluid, decentralized 'work charts' focused on autonomous task execution and throughput.
- 4Microsoft's newly adopted 'seasons' planning framework replaces traditional fixed roadmaps with adaptive cycles that align with rapid and unpredictable advancements in AI technologies.
- 5The rise of full-stack AI builders—individuals or small teams proficient across product, engineering, design, and AI model tuning—is transforming traditional organizational roles and accelerating product development.
- 6User interfaces are transitioning from static graphical interfaces (GUIs) to code-native, stream-based, and adaptable interfaces that evolve automatically based on user interactions and align better with large language model (LLM) architectures.
- 7The guiding principle in AI product development is 'the loop, not the lane,' emphasizing continuous, data-driven feedback loops over fixed organizational silos for optimizing product output, cost, and user experience.
- 8Integrating AI where users already operate—such as embedding AI agents within existing productivity tools and workflows—is essential for adoption and effectiveness, rather than relying solely on new, isolated AI chat interfaces.
- 9Scaling and managing large fleets of autonomous AI agents necessitate advanced observability, automated routing, fine-tuning, and self-healing infrastructures to maintain reliability, quality, and continuous improvement.
Notable Quotes
"You said that we're just starting to scratch the surface of what an agentic society actually looks like. We're approaching this world in which the marginal cost of the good output is approaching zero. We're going to see exponential demand for productivity and output. The way that you scale to that is with agents. When all of that happens, the org chart starts to become the work chart. You just don't need as many layers."
"Because these models are so effective at this point, you want to start to tune them to certain types of outcomes. All of a sudden, these are these living organisms that just get better with the more interactions that happen. I think this is the new IP of every single company, products that think and live and learn."
"And so all of a sudden, products aren't just like these static artifacts that we start to ship. That's not just like, hey, come up with an idea or an insight, go solve a problem, ship it into the world, maybe make it a little bit better and then have a dashboard. All of a sudden, the whole KPI is what is the metabolism of a product team to be able to ingest data and then digest the rewards model and then create some sort of outcome?"
"I think there's things that are kind of more broadly applying to the organization themselves. And then there's things that are applying to the people who are building the AI products too. So more broadly, I think there's a pattern that's starting to emerge for successful companies. Like one is they are embracing AI and everybody becomes AI fluent. So I think everybody's using some sort of copilot or some sort of AI in their day-to-day workflows."
"Planning right now is just crazy. How does anyone plan a roadmap when there's just like, OK, GPT-5's out? We think about it as what season are we in? Season one might have been prototyping of AI and then it was all around models and reasoning models. And now it's the advent of agents."
""I think it's all about the loop, not the lane here. And so I think that whatever function you are, you have to be obsessed with trying to understand like the efficiency or the cost of the product, the actual rewards or system design that you're going after, the actual UI UX, how that actually manifests for agents or people. You have to start to get really good at that really quickly. I like this phrase. You just use the loop and not the lane. Can you say more about that?""
""And obviously in the coding space, you mentioned cursor, GitHub has very similar features that we're using kind of as an ensemble of models that have been fine tuned across, you know, 30 different countries. All of the languages to actually then go iterate in a loop for the next set of suggestions or code completions and things like that.""
""We saw a massive difference from when we used synthetic fine tuning to when we annotated 600,000 patient, physician interactions by experts and actually fed that into the model and continuously optimized it to then produce, like, you know, I think we're sitting between 30 and 60 character acceptance rate, depending on the run. And then we're going to do something like 83%.""
""Like we use email today to collaborate with each other. We use docs like everybody uses Word and PowerPoint. You know, there's a billion people living in places of artifacts that I think can become really important composable pieces of the picture. And I think they should be.""
""The way Nick described it is we're in the MS-DOS era of ChatGPT, which is interesting. It's like the reverse of what you're saying. So it's like maybe if you start as that and then you have to move to GUI and then maybe it'll go back. But he said there's going to be like a Windows version where it's much easier to understand what the hell is going on.""
"So everybody on the team is expected to code. But like, you know, sometimes just chatting in and like talking in real words actually gets you to a prototype that's more interesting and like more expressive and reflective of your creativity. So we use that. I mean, I think everybody's using AI to write. Everybody's using AI to kind of find ways to have efficiencies and like coming up with documentation and things like that. And so I think it's everywhere, which is cool."
"You would think if some AI had all of the information you had about where the market's going, your metrics, your product today, it would be so good at developing a strategy for you. Many people think that's the one thing AI will be really not good at for a long time because that's where we need all this human judgment stuff."
"I think docs themselves, like for every idea, for every, you know, need will just start to kind of fade into, you know, applications and different artifacts in the productivity suite, which, you know, is just a different way of working."
"When you said there's 15,000 agents, what does that mean? Is that 15,000 types of agents you can use? Or is it like that's how many processes are there? So that's, you know, customers, 15,000. 15,000 customers who have produced agents. I think the number of agents is actually like millions. 15,000 customers that are building a specific kind of agent on your platform. And they're running. And the number of agents is in the millions just running. Yes. In the cloud."
"The number one learning that I had was look like WhatsApp didn't win because it had stickers or stories or dark mode. In fact, I don't even think it had all of those things when it won. It won on a few premises because one was the phone book. Like you knew that when you use WhatsApp, you could reach every single person because you had their phone number. And those are the people that you care about when you're using messaging. It was the reliability and how fast it was. Like I could text my grandmother in India and know that she would get my text message all the time. And then it was the privacy. Like when you are sending 200 messages a day to the four people you care about most, you want to make sure no one else can read the messages. And so the end to end encryption really mattered. And so it wasn't the hundreds of features. It was all in kind of the infrastructure and the platform."