The home team talks with Replit CEO and founder Amjad Masad about the democratization of coding tools; why Replit prioritized a mobile coding app; and the technical challenges of a real-time, multi-user IDE. This is part one of our conversation.
Replit is a browser-based IDE (integrated development environment). See what they’re up to on their blog or just start coding.
RIP Google Wave, one of the greats.
Find Amjad on LinkedIn, Twitter, GitHub, or via his website.
[intro music plays]
Ben Popper Hello, everybody. Welcome back to the Stack Overflow Podcast: a place to talk all things software and technology. I am your host, Ben Popper, Director of Content here at Stack Overflow, joined by the wonderful folks on my content team: our writers, editors, and thinkers, Ryan Donovan and Eira May. How's it going, y'all?
Ryan Donovan Good, how are you doing, Ben?
BP I'm full of pep today. Our guest today is none other than Amjad Masad, the CEO over at Replit, a company you may have been hearing a lot about during the generative AI boom, and I'm sure you've heard a lot about it if you're a software developer who listens to this podcast. As you're listening to this episode, if you enjoy it, we are going to be airing Part 2 afterwards. So if you're listening today on Friday, tune in next Tuesday for the second part. So Amjad, without further ado, welcome to the Stack Overflow Podcast.
Amjad Masad Thank you. It's my pleasure to be here.
BP So for folks who have been living under a rock or don't know your full backstory, let's give them the 10,000 foot view. Just really quickly, how'd you get into the world of software development and what was your path to becoming CEO at a company?
AM I started messing around with computers really young. I was six years old when my father bought a computer, and where I grew up it was actually very rare to see a computer. I grew up in Amman, Jordan and that was in 1993, and it is one of my earliest memories looking at this machine and saying, “What is this alien species doing here?” And one of my earliest experiences was sitting on a DOS terminal and actually watching my father finger-type and trying to find his way around the keyboard, and I got fascinated. It really gripped me almost instantly. And I wouldn't say I had a real coding experience until my early teens when I got a programming environment, because initially it was just MS-DOS and batch scripts and things like that. And then I built my first business using VB4, I built this LAN gaming cafe. I don't know if you have this trend in the US, but you would go somewhere and play Counter-Strike with friends when the internet was actually not that great.
BP Oh yeah, absolutely. I was playing Quake and Warcraft II locally only. Local Area Network party. Love that stuff.
AM Right, because dial-up was not all that good for these kinds of games, although it kind of worked, surprisingly. So I built this management software for them and that was my early experience in actually making something useful for other people and getting money for it. And in college I experienced this problem which turned out to be more universal than I thought at the time, which is that when you're trying to do homework or trying to learn programming, setting up the programming environment is actually one of the most daunting things you can do. It's actually very boring and very laborious and you have all these problems when it works on a machine, you send some code to a friend or to the professor, and they can't run it because they're missing a dependency. And I used Google Docs at the time in 2008/2009 and I was like, “Man, I can just share these things easily. Why can't I share code? I mean, that's what the browser is made of. I should be able to share code.” And you can share a GitHub gist or something like that, but you can’t share something you could run. So there wasn't anything out there at the time. There were some JavaScript playgrounds, but there was nothing for Python or anything like that. So I was one of the first people to do that and build an open source project where you can run a lot of different languages. And at the time we used Emscripten, which became Wasm. What people know as Wasm today started as a research project from Mozilla. So I used it to kind of compile Python to JavaScript and Lua, a bunch of languages, and put out this playground. It went viral on the internet. And it was kind of surreal, because here I am a kid in Jordan and I see the inventor of JavaScript tweeting about my open source project, and that got me a job in the US at Codecademy. I was a founding engineer there. I came to the US in 2012, I was 24 years old. After that, I left for a stint at Facebook. I worked at open source projects like React and React Native at Facebook. And then in 2016 I'm like, “This online IDE problem is still not solved, and not solved in the kind of way that I want to solve it,” which is to make it multiplayer by default and shareable. And that's when actually my partner, my wife and my co-founder now, and I took the first step to starting this company, and it's been somewhat of a long journey since then and really started taking off in 2020/2021.
Eira May Yeah, I was so interested in a piece of writing on your website where you were drawing a connection between people who are programming today and people who were literate prior to the invention of the printing press and that kind of kicking off more widespread literacy. Just a very macro question but I was interested, how do you see Replit’s role democratizing access to coding technology in a similar way? Could you talk a little bit more about that?
AM Yeah, so that analogy is not a perfect analogy, obviously, but the idea is that before the printing press, reading and writing was something that the priesthood did and no one else did. And in a lot of ways it was kind of constrained and purposefully kept from the general population. It was seen as something that only elites could do, and when we actually had the decentralization of reading and writing, and now we have universal literacy, it changed the world in fundamental ways. I think it created scientific revolutions and the industrial revolution and democracy and a lot of other innovation, how we live and how we do things. And you can't see it happening without the invention of the printing press and the democratization of reading and writing. I think Alan Kay, who is a hero in a lot of programming circles, said that the computing revolution hasn't happened yet. And what he meant by that is computers went from these machines that you have to learn to program in order to use them to machines that were mostly made for consumption. In a lot of ways, computers today are more like TVs than actually computers that were conceived in the mid-last century as thinking partners, as things that really augment our intelligence. And so when you think about why it is that only a few people can actually use the computers to their full power, which includes programming, one of the answers is that the tools are actually inscrutable and hard to use. I wouldn't go so far as to say it is intentional that we make our tools hard, but there's a part of it. Actually some of the criticism we get at Replit is that we're making things too easy and somehow being hard is important.
BP You can't let everyone into the priesthood, okay? It wouldn't be special. It wouldn't feel special anymore.
AM So our goal from the start is to make the tools easy, and that meant opening a web browser and starting to code. And since then, we found that everything you try to do with software is harder than it should be. Putting a website up is really harder than it should be. Why can't you code on your phone? A large percentage of the population using computers today are just on phones and so we made a mobile app– I think one of the best mobile apps for coding. And so it's just really a constant evolution of how we can make this more accessible, more learnable, and more fun, frankly, to use.
RD Well, to follow Eira's high-minded question with more of a nitty-gritty, I want to talk about the challenges of doing a real time multi-user browser. I'm a big fan of Google Docs here. It's such a nice experience, but I remember our former Chief Product Officer, Teresa Dietrich, telling a story about when she was at AOL and introducing the three dots in an AOL chat and having that crash the system with all the pingbacks. So what are the technical challenges with having everyone editing at once?
AM That's a very important question that I get rarely asked, because it took us perhaps five years to actually get it to work well. So Google Wave –I'm probably dating myself here– but Google Wave was this project that ended up dying, as most Google projects do.
BP R.I.P. one of the greats. Gone too soon.
AM Yeah. It's actually hard to describe what it was, but in some capacity it was kind of a real-time social network plus some tools on top of that. And they invented this format called Operational Transforms. So Operational Transforms allows you to do efficient text manipulation in real time and in a multi-party system. So when I type a word in my text editor or letter, instead of sending the entire content of the text editor, you send just the diff. You just say, “At location XYZ, I'm making the letter change or a delete or an insert,” or whatever. The added benefit of that is you can do reconciliation between different editors. So all programmers have experience with diffs because we use them in GitHub. Diffs is one sort of asynchronous way of handling conflicts, and it's actually very laborious to handle conflicts in Git. But in real time updates, this protocol actually has built-in heuristics to handle conflicts. Now, it turns out that our problem is a lot harder than Google Docs, because in Google Docs you have a single server that is the source of truth, and you have clients connected to that server, so a client-server architecture. With Replit, you have clients but you also have the computer itself that could potentially do edits. So I can write a program, and that program is editing a file. Humans can be editing a file, but also the program could be editing a file, and now AI can be editing a file, so you have a multi-agent system. And the interesting thing about it is that you can write a bug and the computer might crash because you're actually writing code in that computer that's also handling the marriage conflicts and all of that stuff. And so it becomes a distributed system challenge. You have multiple sources of truth: you have the file system, you have the editor, you have multiple kinds of editors, and so it made the Operational Transform problem way, way harder. Now there are innovations like CRDT. Figma famously did CRDTs and a few others. There are a lot of things that make it easier, but we had to go the hard path and we ended up in our own domain, really improving and making Operational Transform a lot more robust and easier to handle, and it took a while. For a while it's been buggy and all of that, but we nailed down all the edge cases and now it works super well.
BP Sweet. So from Eira's question I got this idea of wanting to further democratize access to computers and the ability to code and create with them, not just watch videos and play games, and when Ryan asked his question, luckily you said the magic word and brought up AI agents that might be working alongside you. So I think Replit has been at the forefront of offering up coding assistance. Tell us a little bit about the ones you've been building, what they do, what you're excited about, and then I would love to explore the pros and cons, because we were talking about how people don't want to make the tools too easy. I can now use natural language and have something whip up code for me, address the errors for me, and create a program for me without having done my CS basics, and maybe not really operating from first principles very well. So tell us about what you've been building at Replit, and then how you see the opportunities and challenges in that space.
AM So I first started thinking about, call it statistical approaches to code synthesis, since my Codecademy days, because we had to do a lot of work on correcting code for people, helping people code, and all of that. I always thought that the classical approach of creating a parse tree, traversing that parse tree, and having some heuristics and logic around that was too laborious and maybe there's a learned type of system that could do that. I played around with different language modeling techniques before transformers. One technique that actually had some interesting capabilities is Ngrams. You could actually model code using Ngrams. There was a paper in 2012 that is now a classic paper, saying, “On the naturalness of software.” It has this one of those iconic titles, and it talks about how we can use natural language processing techniques to model code. So it's always been on my mind. It never really worked super well. I think around GPT-2, I just started seeing glimpses that this thing actually can generate coherent code. It was still not that great, but GPT-3 showed the potential. Actually, the system we ended up building at Replit looks more like GPT-2. It is a 3 billion parameter code model that helps you do autocomplete, edits code, corrects code, and things like that. We still also use providers such as OpenAI for some of the larger model needs that we have. But today we have the autocomplete feature as your typing code. Maybe you're familiar with Copilot or the Gmail autocomplete shows this ghost text. This is basically the AI predicting what you might want to type. And I think about it as less an AI, more like a typing aid in a way, because you're still driving. They call it Copilot because you're still the driver. It is something that is maybe looking over your shoulder and giving you suggestions about what you might want to do next. But then there's the other more interesting interaction which is generating entire files, making large edits on files, even building entire features and writing entire unit tests, and this is where a lot of people are using chat models. And the good thing about chat models is that you can talk to them purely in natural language as opposed to the more Copilot interaction where you're typing code and it’s completing the code. You can actually give it instructions like you are talking to a coworker. And so right now we have two modalities there. One is that you can highlight code and right click and edit or right click generate. You can do contextual actions on the code editor itself, or you can talk to a chat model or you can go back and forth between these two things because it has context. And so these are the two modalities that we have today. We think the more agentic type behaviors where you can ask the AI to build an entire feature are coming down the line. We're building the infrastructure for it. I think the models need to be a little more powerful for that to work. By next year I would expect to see a lot of products coming out with more coding agents. In terms of your question about the trade offs, that's a very important question that doesn't get enough airtime. What does it do to the student's ability to learn how to code? And then the other related question is, how do you actually know for sure that the code is doing what it's supposed to be doing, and this is similar to the open source problem like NPM installing left-pad and then left-pad getting taken over by hackers.
BP Right. I mean, people had already outsourced a lot of this to NPM or to copy/paste from Stack Overflow, to your point, so you always want to be in the loop, trust and verify. But I think if a year from now one engineer or product manager is asking for a feature and getting it the next day, what does that mean for the students who want a job as a junior engineer?
AM Yeah, so I don't think we're going to get to a point where you can disintermediate software engineers. And for a product manager or designer to completely rely on agents, at least I don't see it for a decade. It's really hard to see that happening. The reliability, the hallucination problem, all this stuff is really unsolved and no one has any idea how to solve it. That's the reality of the situation. And that's the fundamental problem with neural networks, we don't know how they work. One way to think about neural networks is that it is an algorithm that searches for other algorithms. So GPT-4 as a network is a set of algorithms that are totally opaque to us. We don't know actually what they're doing, and therefore we can't trust them and you need to verify. And so there will always need to be a software engineer that is actually verifying and looking at the code. And if you're a junior engineer, I would definitely learn the basics. I would probably code without an AI for a little bit. I would probably just learn the computer science basics as much as possible, but I would really learn how to use AIs and how to leverage them to become more productive. That's going to be the expectation when you enter the job market. There's going to be pressure on you being a lot more productive than the previous generation of engineers.
BP Totally. I think it's going to be like how if you didn't know Git coming in at this point, you'd be in a lot of trouble. I feel like that's going to be the same for AI.
AM That’s right.
EM I wanted to ask a little bit about the manifesto and what the purpose was of putting together a Replit AI manifesto. I mean, obviously we've talked a little bit about how practically you're thinking about it, but why did you feel like it was important to articulate that stance with– is it Michele, your VP of AI? How did you guys come to write that and come to put that together?
AM Yeah, so the larger context is what we're seeing in other players in the market and how they're talking about AI. OpenAI, Anthropic and others are on the record saying they want to build AGI. They want to reach human-level performance and they want to automate away a lot of jobs and things like that, which is okay, great. I mean, that's a great mission to go after. At Replit, in our DNA, we care more about intelligence augmentation than artificial intelligence in a way. We want people to just be more productive. The reason we built Replit is because we want people to learn programming so that it can affect their life in a positive way. We're less interested in full automation. We're more interested in helping people get the work they want to get done. So in that manifesto, we talk about ADIs, so we coined a new term– Artificial Developer Intelligence. So instead of artificial general intelligence, what we're saying is that we want to create an AI that is tuned to the needs of the developer. And we thought that was a novel thing that no one is really talking about, and most of the talk is about how do you automate software engineers, and instead we're saying, no, we want to empower software engineers. And it was part to put out a flag and just say that if you're interested in more augmentation, you should come work for us, and second is just to start a conversation around that, around how we can do that, how other players can be involved, how open source folks can also be involved in this. And we've gotten really great reception. We've gotten a lot of people that are really more mission-oriented apply and want to work for us.
BP They're on team human, so they want to stick with you.
AM Yeah. I don't want to draw black and white us versus them. I do think that ultimately even those big labs really care about humans obviously and human flourishing, but I do think that maybe inadvertently they don't really understand the full extent of what they're talking about. But I think the more concrete goal of any company should always be how do you help your customers become more productive and serve them better, rather than automate them.
[music plays]
BP All right. So this time of the show we usually shout out a Lifeboat Badge winner, but there isn't a new Lifeboat today, so thanks to macxpat for answering the question, “How do I install a Linux package in Replit?”
AM Oh, nice.
BP We've helped over 300 people, so thanks, Max. We'll put it in the show notes for anybody who is listening and is curious. As always, I am Ben Popper. I'm the Director of Content here at Stack Overflow. You can find me on X if you want to shoot me a DM. You can hit us up, podcast@stackoverflow.com, with questions or suggestions for the show. And if you enjoyed the program, leave us a rating and a review, because it really helps.
RD I'm Ryan Donovan. I edit the blog here at Stack Overflow– that's at stackoverflow.blog. And if you want to get at me in my DMs on X, the DMX, I'm @RThorDonovan.
EM My name is Eira. I'm also on the Editorial Team at Stack Overflow, write for the blog, and write the podcast show notes. And I am on social media most places @EiraMaybe.
AM My name is Amjad Masad. I'm CEO and co-founder of Replit. Go to Replit.com to check it out. It's very easy to sign up and start coding. I’m also in most places @AMasad. GitHub, X– it’s so strange to say X, but at least there's a lot of opportunities for puns here. But on X/Twitter.
BP I like it. All right, everybody. Hope you enjoyed the episode. As a reminder, Tuesday of next week we will have Part 2, so if you enjoyed the conversation, please share it out on social media and then come back and join us next week for the second half. Thanks for listening, and we'll talk to you soon.
[outro music plays]