Lenny's Podcast: Product | Growth | Career

Inside Devin: The world’s first autonomous AI engineer that's set to write 50% of its company’s code by end of year | Scott Wu (CEO and co-founder of Cognition)

May 4, 2025
Listen Now

Summary

The podcast episode features Scott Wu, CEO and co-founder of Cognition, discussing Devin, the world’s first autonomous AI engineer designed to write a substantial portion of a company’s code. Devin is fully integrated into Cognition’s 15-person engineering team and interacts asynchronously through tools like Slack, Linear, and GitHub, emulating a junior engineer that developers can collaborate with and manage remotely. Currently, Devin contributes about 25% of Cognition’s pull requests, with ambitions to exceed 50% by the end of the year. The evolution of Devin has transitioned from a high school-level programmer to a junior engineering role through the application of reinforcement learning and agentic autonomous workflows. This transformation marks a fundamental shift in software development, wherein engineers transition from “bricklayers” of code to architects overseeing AI-driven coding agents. Scott emphasizes that AI augmentation will increase, not decrease, hiring by enabling engineers to focus on high-level problem-solving while outsourcing routine code to AI. Cognition’s journey involved significant product pivots, including building deep integrations with Slack, GitHub, and Linear to facilitate seamless human-AI collaboration and reduce friction in workflows. The design choice of personifying Devin with a junior buddy personality enhances adoption and interaction, enabling engineers to relate to the AI as a team member. The episode covers technical innovations such as incremental codebase indexing for Devin to understand large, complex legacy systems and discusses significant cultural shifts required for effective AI engineer adoption. Scott also contextualizes AI as the biggest technological revolution of our time, surpassing previous shifts that depended heavily on hardware distribution, highlighting AI's unprecedented scalability and impact on industry workflows. Finally, the episode reflects on the future of software engineering as increasingly collaborative between humans and multiple autonomous AI agents operating asynchronously, requiring new skillsets and organizational adaptations.

Key Takeaways

  • 1Devin, Cognition's autonomous AI engineer, significantly boosts engineering productivity by autonomously producing 25% of the company’s pull requests, with projections to exceed 50% by year-end.
  • 2Devin operates as a fully autonomous junior engineer, integrating asynchronously with existing engineering tools to manage tasks end-to-end, including issue tagging, code writing, and PR submission.
  • 3AI is poised to be the most significant technological shift in history, distinct from past hardware-driven revolutions due to its rapid, hardware-independent scalability.
  • 4Contrary to fears of displacement, AI tools like Devin will increase demand for engineering talent by automating routine tasks and elevating human roles to higher-level architecture and design.
  • 5Successful deployment of AI engineers requires significant cultural shifts, deep tooling integrations, and reimagined workflows to facilitate smooth handoffs between humans and AI.
  • 6Devin’s personification as a ‘junior buddy’ enhances user engagement by making the AI relatable and framing it as a team member rather than a faceless tool.
  • 7Reinforcement learning (RL) and agentic loops are foundational to Devin’s evolution, enabling autonomous decision-making, iterative improvement, and complex problem-solving beyond imitation learning models.
  • 8Devin augments team knowledge by accumulating insights across engineering sessions, enabling continuous improvement and shared understanding within teams.
  • 9The role of human engineers is evolving to supervisory and architectural functions, focusing on task scoping, reviewing AI output, and defining problems rather than writing boilerplate code.

Notable Quotes

"Our whole team is only like 15 engineers. We use a ton of Devin when we're building Devin. Most folks on the team are definitely working with up to five Devins at once. Devin merges like several hundred pull requests into production in the Devin codebases every month. It's in the neighborhood of a quarter or so of the pull requests now. Honestly, we expect it to be a decent bit more than half by the end of the year."

"AI is going to be the biggest technology shift of our lives. So most of the big tech revolutions that we've had over the last 50 years, like personal computer and the internet and mobile phone, they all had this big hardware component that was a big part of the distribution. One of the things which is already, I'd say, different in AI is just how explosive the technology can be. There's no weight on hardware distribution. It means that the space is just growing so exponentially."

"When Devin launched about a year ago, it was very much a junior engineer. Over the past year, they've made a lot of progress, and Devin is now being used by tons of companies in production. We chat about how their engineering team of 15 uses Devins to build Devin, including how every engineer uses about five Devins each to help them code and move faster. How a quarter of their pull requests today are committed by Devins, and that they expect this to be over 50% by the end of the year."

"There's no weight on hardware distribution. It means that the space is just growing so exponentially. Folks who were building for those industries kind of saw their market grow and grow and grow basically steadily year over year as the number of people with mobile phones increased, right? As the number of people connected to the internet increased."

"Just like really building out like how all of us should be working and interacting with agents as part of our flow. And so a lot of the things that we built, I mean, it's, you know, there was no Slack. There was no GitHub integration. There was no linear. There was no interactive planning phase working back and forth. There was no way to touch up Devin's code. And so a lot of the features that we built on the product side since then have really been about basically, yeah, figuring out how to make working with Devin and handing off tasks to Devin as smooth of an experience as possible."

"But throughout, it's all been basically, yeah, being that junior buddy of yours, that makes you go faster and really multiplies you, I would say. You know, I think it can multiply you as an engineer, obviously, by just letting you work with your own team of Devin's instead of having to be kind of fully synchronous on a single task. And then it's also kind of like multiplying your team and multiplying your team's knowledge base, because Devin really accumulates a lot of the knowledge from working with every member of your team and is able to bring that into each new session."

"And one was that reinforcement learning was really working and was going to be the next big paradigm shift in capabilities. You know, back then it was, you know, the initial ChatGPT launch in 2022. And those models were to first order what we would call imitation learning in AI, right, which is basically, you know, you have the model read all the texts that you can find on the Internet and then train it to talk like somebody on the Internet would talk. But that's kind of the first order pass of what was really done. And it was amazing, right? I mean, it passed the Turing test. It was able to respond and to have encyclopedic knowledge about a lot of things."

"Right. But I think that the big shift that we really felt we would see is moving from kind of this text to text model to an actual autonomous system that can make decisions, that can interact with the real world, that can take in feedback, that can iterate and take multiple steps to solve problems. And, you know, now we call that agents, but that was what we were really excited about at the time."

"I want to come back to an area you started us down and then I took us away from, which is impact on software engineering and how software engineering is going to change. So there's kind of two parts to this, just like when people are using Devin today, say like this year, how is the act of being an engineer and building changing for those companies? What does that look like?"

"Right. And so every engineer on our team, you know, we use a ton of Devin when we're building Devin. And so Devin merges like several hundred pull requests into production in the Devin code bases every month, you know, which is I mean, our whole team is only like 15 engineers. And so it's a pretty sizable fraction of all the code that we write."

"So today about a quarter of your PRs are Devon's. Where do you think this will be at the end of the year? I think by the end of this year, we expect it to be more than half. And I mean, as time goes on, you know, one of the things that we've seen is just you're able to do more and more and more work asynchronously. Right. And you're able to hand off more and more."

"I think the soul of programming, the soul of software engineering has really been about through all the areas, you know, not just now, but, you know, even when it was assembly, right? And even when it was Pascal and even when it was punch cards or whatever. I think the soul of it has really been basically just about defining the problem that you're facing and really thinking through exactly what is the solution that you want to build."

"You know, folks will use assembly, for example, if they're really performance optimizing a piece of code. But also, you know, it's in order to build good systems and to understand these things, you certainly want to understand these abstractions of, you know, how does networking work? Like, what is TCP IP like exactly? Or what happens with this Python code when it gets interpreted or all of these details?"

"So Jevon's paradox just says that as the price of something goes down, it can still be the case that the total spend on it actually goes up. And so, you know, you can think about this with money. You can think about this with time or resources. But the direct version here is I think as it becomes easier and easier to program and as programming becomes more and more effective, I think we're going to have a lot more programmers."

"But most folks on the team are definitely, yeah, are often working with up to five Devons at once, I would say. And it's a nice flow where it's, you know, you think through, all right, what are the five things that we want to get done today? You have Devon one do number one. You have Devon two do number two. Devon three. And the thing about it is a lot of it is, and for what it's worth, you know, I think it's taken us some time to really kind of like adjust to it and get to the point where it's really intuitive for us. But I think it's, yeah, it's definitely a different experience where you're handing off most things asynchronously."

Episode questions

How does Devin operate within Cognition's engineering team and what is its current level of contribution?

Devin acts as an autonomous AI software engineer integrated with Slack, Linear, and GitHub to handle issues and produce code autonomously through pull requests. Each human engineer works with roughly five Devin agents, facilitating asynchronous task execution and speeding development. Currently, Devin contributes approximately 25% of all pull requests merged into production, with expectations to surpass 50% by the end of the year, demonstrating significant AI productivity in real-world software development.

What makes AI a fundamentally different technological shift compared to historic revolutions like mobile phones or personal computers?

AI does not rely on hardware distribution, unlike previous technological revolutions which required physical devices to proliferate for adoption. This absence of hardware constraints allows AI to scale exponentially and much faster through software and cloud-native means. Consequently, AI technologies spread more rapidly and broadly, leading to a more explosive and transformative impact on industries and workflows than earlier tech revolutions.

How has Devin evolved since its initial launch and what are the implications of this evolution?

Devin has progressed from performing like a high school CS student at launch to now functioning as a junior engineer with practical skills used in several companies' production environments. This evolution shows that AI coding agents can improve continuously, increasing their utility and reliability over time. The implication is that autonomous AI engineers will gradually handle more complex coding tasks, reshaping human roles toward oversight and system architecture.

Why is the cultural and workflow integration as important as the technical capabilities when deploying AI engineers like Devin?

Deploying autonomous AI engineers requires aligning AI workflows with human engineering processes to avoid friction and inefficiencies. Cognition learned through multiple pivots that smooth collaboration, communication integration (via Slack, Linear, GitHub), and easy handoffs are essential for real use. Effective human-AI collaboration design reduces resistance, increases trust, and enhances productivity, highlighting that organizational culture and tooling maturity are crucial success factors alongside AI technology.