On this episode, Ryan chats with Vish Abrams, chief architect at Heroku, about all the work that needs to be done after you’ve vibe-coded your dream app. They discuss the security and scaling work needed, why getting the foundation is as important as the business logic, and what skills a vibe coder needs after prompt engineering.
Heroku is a platform-as-a-service (PaaS) for deploying, scaling, and managing apps.
Connect with Vish on X and LinkedIn.
Congrats to Populist badge winner AmaDaden for their answer to How to generate a legend with colors in PlantUML?.
[Intro music]
RYAN DONOVAN: Hello everyone and welcome to the Stack Overflow podcast, a place to talk all things software and technology. I am your host, Ryan Donovan, and I am here to talk about vibe coding and its discontents. This is a special episode sponsored by Heroku, and I'm joined today by Vish Abrams, Chief Architect at Heroku. So welcome to the program.
VISH ABRAMS: Thank you. Thanks for having me.
RYAN DONOVAN: Tell us a little bit about yourself. How did you get into software and technology?
VISH ABRAMS: Sure. I think the most relevant portion to talk about is when I first moved out to the Bay Area to move from small business software development into big tech software development, and I came out to the Bay Area to work at NASA and we ended up creating a private cloud for NASA that ended up resulting in the OpenStack project, which many people may be familiar with if you're around in software 10 years ago. It was kind of a big deal.
It ended up being superseded later by interest in Kubernetes and some of the more modern container-based deployment scenarios. But back then, everybody wanted to build a cloud to compete with Amazon, and that's what OpenStack helped you to do.
RYAN DONOVAN: Cool and now you are at Heroku, which is a Salesforce company. Yes?
VISH ABRAMS: Yeah. I've been at Heroku now for about six and a half years, and most people go, well, is that before it was acquired? Because memories are short. But actually, Heroku was acquired by Salesforce way back in late 2010. So it's been many years. I was not there pre-acquisition, but I've been leading architecture at Heroku in various forms since I joined and now Chief Architect.
RYAN DONOVAN: So you've been a programmer for a long time. One of the newest things we hear about programming is the rise of vibe coding and hear a lot of people going into their ID terminal of choice, type in some words and get the whole program. So they've got their vibe coded application. What now? What are they missing? Why can't they put that on the web?
VISH ABRAMS: First of all, it's a really fascinating time in the industry, right? There's been this idea of low code for so long. There's been tools that have been trying to do it, and now suddenly there's a level of reality and power to that type of experience that hasn't ever existed.
I hope everybody's at least had an opportunity to experiment with a little bit. You can just go in and say, write me an application that does X, and in many cases, an LM will spit out right away a fully featured application, and if you have some of the agentic abilities that are there– things like cursor, et cetera.
You can even guide it a little bit and tell it directions to go and do iterative coding and things like that. And I think what's really appeared is this, first of all, a new skill set for developers, a new type of developer. I mean, my wife does some vibe coding. She has no software engineering background.
So if it's possible for just intelligent people in other areas surrounding technology are suddenly able to build these things. We're in a different place. In my mind, as you ask, the part that's missing is, well, what do you do next? You wrote your code, and agents are very good at that part of the process, much better than we expected them to be I think anyone in the industry this quickly.
But when it comes to maintaining that software over time, deploying it, figuring out how to update it, working on it with the team, doing code reviews, making sure that it's not doing silly things, dealing with security vulnerabilities. There's just a whole laundry list of things that software engineers have to do. That vibe coding is not really well positioned to do yet.
RYAN DONOVAN: Yeah. When people think of the applications, they think of the business logic around it. Software engineers think about, well, there's this whole super structure that goes around it. Like you want a pool, I have to build out the plumbing, get the supplies, figure out the chlorine situation, right?
VISH ABRAMS: I think there's actually two different directions to think about this from. There is what do software engineers that are traditional software engineers, what do they no longer need to worry about? And then the new breed of software engineer, these people coming into the industry who don't have a background in software engineering, what things do they now need to think about that they're not familiar with thinking about?
So it's like we have two groups of people kind of on a collision course coming closer together and needing to learn new skill sets and figure out where they meet. And that's pretty fascinating to think about, like the skillset that you need as a non-engineer to deploy and manage software over time and the skillset that you need as an engineer to interact with this natural language type of coding style.
RYAN DONOVAN: Yeah. So let's assume that they have the sort of idea of the program pretty well in hand. We ran a post a while back that was saying the hard part isn't coding, it's requirements. So let's say they have the requirements down. They have a piece of software that does what they want it to do. What do they need to think about after that? What's the next thing that they need to start thinking?
VISH ABRAMS: The first step is getting the foundation right, and many people in the industry are familiar with the Twelve-Factor App manifesto that was written by Heroku years and years ago. It was sort of a cheat sheet for a newer developer that wasn't used to building a web application on the principles that they needed to build in order to make a scalable and deployable web application. For example, keep your configuration separate from your code. Put your code in version tracking and things like that.
So sort of the basics of what it means to be a software engineer. And if you kind of go back a decade and you look at the industry in that time, I mean, what was the big change that was happening? It was like rails, Ruby on Rails was the big new thing. You went from engineers having to think about, okay, well how do I manage a virtual private server, and how do I keep my kernel updated, and how do I deploy, and all of the skillset, which is suddenly, no, don't worry about any of that. You know, you can scaffold up an application and you can just build something right away.
And so it's like a microcosm of the change we're going through now of a new set of developers that had to worry about different things and had an easy path to deploying applications. And what was needed at that time was an understanding of what are the core principles that you need to build applications in the right way for them to be scalable.
We're in a similar place with these new applications, but I think the twist on it is we need to be able to figure out how to communicate to the LLMs how to build the application in the right way. So if you consider your agent a junior developer, it's like what does the agent need to know in terms of guardrails about building the application that's going to make it scalable over time, right?
So you can imagine a Twelve- Factor manifesto, four LMs. Like what are the prompts that you need to give it– DRY, there's classic programming principles that to seasoned engineers are kind of common knowledge. And the same thing is true about deployment. You want to build your application in a way that the one I said earlier, configuration and code are separate, right?
You can tell the LM to build your application that way, or you can just say, build me a snake game and it'll do whatever it wants to. Maybe not maintainable at all.
RYAN DONOVAN: Do you think that those Twelve-Factor principles can those be built into LMs? Like can you just have an agent that checks things for the configuration as code, that checks the dependencies, all that sort of thing?
VISH ABRAMS: I think you can. The way that most people are doing these kinds of things right now is prompt engineering, right? If you're working in Cursor or something like that, you have a Cursor rules file that kind of gives the LM hints about how you should be writing the code. And I think the next step beyond that, which is just starting to happen, is how do you sort of put the guardrails in place after the code is generated to check it and then maybe do another iteration? Like, did we actually do this–
RYAN DONOVAN: Right?
VISH ABRAMS: And we see this in the debugging process with LMs today, where you have it spit out some code and sometimes it doesn't work and you get a compilation error. And what the agentic platforms do is sort of put that in a loop and say, okay, go fix the error. Go fix the error. And then when you finally stop getting the error, you've got your final thing.
And so I believe we could have similar types of guidelines around some of these principles. We're just at a place in the industry where we haven't quite gotten there yet. We're still figuring all this out. I know people are experimenting with different rules and different ways of interacting with the LM. I mean, if I'm gonna put on my future-looking hat here a little bit and make a guess. I think a lot of the direction around what's happening with tools exposed via model context protocol servers is really gonna be the thing that starts bringing those powers in a consistent way.
The biggest challenge that we have with LMs today, especially when we go into production, is non-determinism. So their power is I take fuzzy ideas or fuzzy logic, and I can turn it into something concrete and it's super good at that. Just a natural human language. Do something. It'll figure out sort of the best version of that and maybe even iterate it on a little bit and come out with something really good.
But the problem is if you really need it to do something deterministically, maybe it works 60% of the time. You know, 60% of the time you get what you want and the other 40% you get something different.
And so constraining it to the point where you actually get something that's deterministic is hard. And my take on this is that the tooling layer, the function calling layer, or now the MCP tools layer is where you put the deterministic code to accomplish the goal, and then you let the agent live in the sort of non-deterministic part. You shrink the scope of the non-deterministic part and increase the scope of deterministic part over time.
And so you have the agent doing sort of the control loop, the non-deterministic part, but then you have verification validation tools, you have data retrieval tools, you have all of the actual real work that you want the agent to do, kind of gets pushed down to that layer, and that makes it more reliable.
RYAN DONOVAN: Yeah, because that non-determinism is the power of LLMs in a lot of ways, right? That it does dream a little bit for you. You talked about MCP. One of the things I've sort of been talking about with people is the lack of standards in the Wild West for agents at LLMs, and it looks like MCP, I think Google has Agent2Agent. There's some other ones coming up. Do you think those sort of things will all sort of join together and there'll be an HTTP for the LLM industry?
VISH ABRAMS: I think we'll definitely get there. I mean, a lot of it's happening. I mentioned OpenStack at the beginning, and there was a powerful moment in OpenStack that I think is comparable to the moment that we're having with MCP right now, which is when we started marketing the idea of OpenStack, the reason it grew incredibly fast, we went from like eight developers to over 500 developers across a dozen companies in like six months, and the reason it exploded was because it was needed. There was this idea that everybody believed in of like, oh man, we need to compete with AWS, we need a cloud story. Let's go build on this thing.
And the reality was the initial set of code was written by a few of us in a weekend, and it barely functioned. And so it was like what it actually did versus what was needed were dramatically different, but there was so much excitement around what it could be that it got a ton of attention and MCP is in exactly the same space.
Like the reality of how it works and what it does is pretty janky in a lot of ways. It's needed, right? We need this, and the best protocol doesn't win. The one that everybody decides to use is the one that wins. So like we'll get there.
It needs to go through some iterations and improvements off around MCP servers right now is terrible, but we'll figure it out because everybody has jumped on the bandwagon now and we're all sort of moving in the same direction, and that's the most important thing.
RYAN DONOVAN: It does seem like it's getting a lot of support, keep getting people talking about the MCP. Do you think folks are contributing to the MCP? It is an open-source project, right?
VISH ABRAMS: It's open-source. Yeah. Came out of Anthropic. Same story as what happened with OpenStack. It was originally run by Rackspace, and people were like, well, it needs to be in a foundation so we can really commit to it.
So I think there is a push there, but mainly the first thing is just getting things, working with it. And I mean, right now it works very well for local LLMs, local agents, and running over Standard I/O when you're trying to deploy it remotely. There's still a lot of gotchas there–it's hard to deploy, there's authentication issues, et cetera.
And who knows, maybe it'll be a thing where Agent2Agent is the one that overtakes it eventually because it's a little bit more well thought out, or it's, you know, sometimes the second version is the one where you remove all the kinks.
RYAN DONOVAN: (laughs)
VISH ABRAMS: The exciting thing is because of the attention that it's getting across companies and industries, we're gonna have a lot to experiment with.
I mean, there's still this big open question about whether it's going to be a hosted agent somewhere that is the main thing, or everybody's gonna have a local agent on their computer, or there's gonna be a set of agents and they're going to do interactions and communications and handoffs, which is sort of the A2A approach.
I don't think any of us really know where we'll end up because it's moving so quickly. Everybody's trying to make a bet and do the best they can. And I mean, we're right in there with everyone else building out MCP servers and trying to make it easy to build and deploy MCP servers on Heroku because the excitement's there and want to support where the industry's going.
RYAN DONOVAN: Yeah. Another sort of thing I've been talking about with people is every SaaS company's gonna be an API and I think the way that sort of Agent2Agent MCP stuff works is gonna be the one that preserves or takes business models.
VISH ABRAMS: Yeah. I mean, it's fascinating and it means I think that people are going to have to think about how they charge for their products differently, because when you controlled the sort of presentation layer for everything, there were a lot more options in terms of hiding how the actual work gets done and gets paid for.
If you're exposing your entire software suite as APIs, and it might be a local agent calling those APIs, you sort of need to be charging on the API request layer, which I think a lot of companies just haven't done. In many ways, I feel like a decade ago with companies like Apigee and some of these big API management companies were all the rage. Everybody was in this API everything, API everything. And we never really got it over the line.
RYAN DONOVAN: Yeah.
VISH ABRAMS: And now with agents, it's suddenly like that is back in– everybody's right in the headlights, you know? Here we are. Oh gosh. We actually have to follow through with this API thing. Yeah.
RYAN DONOVAN: I want to go back to some of the vibe coding stuff.So one of the big things for every application now, you don't download the application, you have it hosted on the cloud. That infrastructure piece feels like it's gonna be a hard jump for vibe coders. Do you agree?
VISH ABRAMS: So first of all, where are your application– because when you write something, I mean the natural impulse is like, okay, I wanna share this with other people. And yeah, sure, you can hand your laptop or your phone to someone and say, look at this cool thing I built.
But these days, social media, et cetera, the first thing you're wanna do is send it to a friend remotely somewhere. And that means suddenly you're in a web deployment distributed system world. And probably swimming way beyond your depth in terms of if you're a new developer and you have never thought about these things before, I can't imagine the breadth of security issues and vulnerabilities that are gonna start up as a result of this. Right? And so I think the main thing that people need, I. As a bridge is like a good set of defaults around deployment that generally would mean using a platform and probably a platform with a bunch of sort of guardrails and default ways of doing things. When I look at this, I'd say this is a perfect fit for Heroku because Heroku has been helping, you know, new developers get stuff online since the beginning, like that's what we do, right?
So it seems like a very natural fit for people to come there. And we've been thinking about that. How do we make it easier to be in a coding tool? Like why not give the agent a MCP server that allows it to deploy directly Heroku for you? So you could just ask the agent to do it. You don't even have to run any commands or log in on a website.
It's just like, yeah, okay, I wrote this thing. Okay, now put on Heroku for me. Great. Now the next step beyond that. I think there's a lot more past that that we're still figuring out, but that first step seems pretty straightforward.
RYAN DONOVAN: Just having the platform that builds everything in, these seem like the next evolution of the hobbyist where they're getting into trouble is they're trying to be entrepreneurs about it too.
VISH ABRAMS: There's already stories of people sharing their vibe coded thing on Twitter and then immediately getting DDoSed or attacked or someone finds their private information. So it's– these are things that are going to happen and as we build up the guardrails– I mean, this is such a new space that no one has fully figured out all of the things we need to protect against in this new world. But we're starting to get some ideas, I think.
RYAN DONOVAN: Yeah, definitely. You know, we talk about the platform for the, I'm gonna say janky, but it, I don't mean a janky vibe coded application. This protects it. How do you think about building that platform?
VISH ABRAMS: I think first of all, a lot of the original principles are in place, which is like, okay, let's make sure that the application is secured via SSL, right? There's some very basic principles that are built into most platforms now, so I wouldn't call those particularly innovative or interesting or creative, but getting all the basics right I think is the first thing.
But the second thing that I'm thinking about is what are the next pieces that you need in the lifecycle of an application that make it easy for a non-expert coder to then take the next steps and become a non-expert operator.
I don't think we're quite at the place of just saying, telling the agent, just give it infinite ability to scale my application up to a thousand. I mean, imagine costs going out of control where agent says, well, I definitely need a thousand copies of this application now because I'm hitting a load spike or something like that.
There needs to be some controls on that part of it, but I think at least the very basics with the right set of tools given to the agent, you should be able to have the agent help with a lot of those parts of the process, especially in the early parts of development, you know, before you need a team of people looking at it and there's people doing poll requests and things like that.
In those early stages, I think a lot of it is about finding the right data to give to the agent. So locally, when you're building and deploying something, your agent can help you find errors in the logs, can run your application for you, find errors, help you iterate through the cycle. But think about, as all engineers know, works on my machine is like the classic problem, right?
So you've got it building locally, you have it pushed to Heroku or some other platform and suddenly it's not working there and you need to debug that. What do you need to debug that? You need to get the logs out of your remote server as well, and you need to feed those back into the agent. So those kinds of tools like give me the logs and metrics from the application that's running are going to be very, very powerful in giving the agent that next layer of capability.
And those are the types of things I think, initially that we're gonna be thinking about mostly and that we're starting to build. Beyond that, it's like, okay, how do you get it into a place where you have collaborators involved? How do you put it into a pipeline where you have dev, staging, production, you know, like the standard stages you go through as a more and more mature application. We need to make all of those accessible and understandable to this new set of coders that isn't used to thinking about things that way.
RYAN DONOVAN: Yeah. Talk about the logs and metrics, and that usually requires some amount of instrumentation, and I imagine there's a lot of things up the chain of maturity that require more fundamental changes to the application. Is there a way that we can sort of talk to vibe coders and be like, do this first, get this as your foundation?
VISH ABRAMS: I don't think we're quite there yet as an industry, but I imagine that there's going to be a set of default rules that are a good starting point, like templates. I think a great example is our platform supports automated telemetry coming out of the application, but in terms of what we can instrument from the outside, so CPU usage, memory usage. For certain languages, we have a little bit of instrumentation around, like if you're a job application, you can see the heap size and things like that.
But there's– some of those instrumentation points need to be in the application code itself. So I'm thinking like traces, open telemetry spans and traces, right? And so if you have a template that essentially tells the LLM when it's building the application, “Hey, by the way, include open telemetry spans for all of the stuff that you do.” Then sort of automatically you get all of that extra telemetry and start doing really powerful things.
And so I think where we're going to land is, and probably they're gonna be curated templates. You know, someone's gonna have, this is my ultimate set of Cursor rules that work really well with this platform, and here's what I recommend you do. And everybody's gonna start there.
And then someone's gonna, why can't it be an open-source project? Why can't we PR, pull request against it and say, no, actually this rule works better. But to get there, we need not only a place to share the things, but we also need some sort of testing to figure out which ones are the best, right?
Because there's such an explosion of different parameters going in. There's different base LMS that you're using that have been fine tuned in different ways. So at this point, we're still in the experiment and figure out stage, I think, as an industry rather than the, yes, these are the five things that you need to do, but we'll end up with some opinionated takes and I think it'll smooth the path considerably once we have those opinionated takes.
RYAN DONOVAN: Yeah. I also get the impression that a lot of vibe coders want to use AI on their application, and then of course, brings in the agentic stuff like we talked about earlier, but also inferencing, which from what I hear, is going to be the thing that's gonna eat up most of the GPU cycles now.
VISH ABRAMS: Yeah, so we've actually pivoted ourselves internally in Heroku in terms of what we're offering in terms of AI offerings on Heroku. I think a couple of years ago it was like, okay, the thing that we need to do is expose GPU execution. We call them dynos, similar to an executing container, running container, we call it a dyno in the Heroku parlance. So we were thinking, okay, we need dynos that have accelerators and GPU so that people can do all these custom machine learning models on top of Heroku.
And suddenly within, I would say, maybe the last six months, suddenly nobody cares about that anymore. Now it's all about like, I'm gonna use an LLM. I'm just gonna have access to, you know, inference. There's so many companies out there that are building the next new version of it. I just want to be able to pick the best one and then I'm going to use that to drive almost all of my AI workloads.
We've really leaned into that. We just actually GA-ed our managed inference and agents system, which essentially just gives you super easy access in the same way that Heroku provides. We call them add-ons or resources. So you can provision a Postgres database, have it connected right up to your application, just start using it with a single configuration variable.
Same idea. You hit a provision button, you're connected to an inferencing service. And it can even do some pretty powerful things like code execution and there's some built-in MCP servers in that platform that are interesting. But I think it's going to enable a lot of people to build AI into their application, as you're saying, in a much easier way without having to think about all the complexity and hand gluing stuff together. So that's another area that we're focusing on a lot.
RYAN DONOVAN: Right. Of the forgotten tasks of the vibe coder, what do you think is the hardest to build into a platform?
VISH ABRAMS: I’m going to take the angle on this one of thinking about the software engineering vibe coders to sort of tease out the necessary skillset, right? Because not everybody who tries vibe coding goes, oh, this is amazing. I love how this works.
So a lot of seasoned software engineers are a little skeptical that there's even anything there. And I think the challenge is the skillset is different. And so the part that's hard to build in the platform is reskilling that a person needs to do when they start going down this vibe coding rabbit hole.
I guess, I mean, first of all, I think you said it earlier, you have to think about the requirements. You have to almost pretend like your agent is an offshore engineering team that you don't have direct access to, that you have to give a very clear set of instructions to, and then you get feedback when they're done and then you go, oh, wow, that's not what I wanted at all.
Now, the advantage is that happens in a cycle of like a few seconds instead of. A couple of weeks. So the cost of getting it wrong is not as high, but you still need the skillset of building out a clear set of ideas of what you're building and then building it. So that's an example.
I think another really key one is debugging. At least where we are right now, there are times when the agent gets itself into a hole that it really can't get out of, and that's when a human needs to step in and sort of direct it in the right direction. And that skillset is a more challenging one to build up. And it doesn't necessarily mean that you need to understand the code, but it sure helps, right?
But you do have to be good at figuring out how to break down problems and solve them. That's a skillset that I don't know that we can solve at the platform layer. I mean, I think we can grease the wheels a little bit and make certain parts of that process easier, but there's still going to be some parts of the skillset that really rely on human knowledge and skill to build up.
RYAN DONOVAN: Yeah, you could highlight that X agent has been running for some amount of time, and if you're like, that's been going for five days, probably shouldn't be doing that.
VISH ABRAMS: (laughs) Well, I mean, when we're talking about building agents too, I'm sure another big thing is just the cost concern in general. Like you have to have some limits, especially if you're going start doing things like giving your agent access to buy things. People are talking about personal shopper agents, right? It's like, well, what's the limit on the credit card that you hit on that agent?
RYAN DONOVAN: Yeah. I mean, I think when I talk to folks who have AI based tools that go to change code, they come up with a PR and let somebody commit it.
VISH ABRAMS: It's amazing to see the scope of areas that AI is changing happening so quickly because yes, you want a human looking at the PR, but then you probably also have an agent with that human helping them review that might help it point out, oh, there's some issues there. So it's like there's this amazing potential for productivity increases.
I get a little concerned sometimes about the expectation.Because we see speedups in some areas and we imagine that, oh, well, as an industry we're just gonna massively accelerate, but if you've ever done any business process modeling, you know that just speeding up part of the process doesn't necessarily give you better throughput at the end.
And in fact, sometimes it can make it worse because it introduces contention and bottlenecks in other parts of the system that you didn't expect. And I wouldn't be surprised if we see some of that. It's like, yeah, it's really easy to build something for the first time and suddenly now it's really hard to validate if the thing that you build is actually valuable because now suddenly you have a thousand versions to review of a particular solution. And which one do you pick? We haven't had that problem before because before we could only build three.
RYAN DONOVAN: (laughs) Well, it's that time of the show where we shout out somebody who came onto Stack Overflow, dropped some knowledge, shared a little curiosity and earned a badge. Today we're shouting out the winner of a populous badge. Somebody who came onto a question on Stack Overflow, dropped an answer that was so good it outscored the accepted answer.
Today we're shouting out AmaDaden for answering “How to generate a legend with colors in PlantUML.” We'll put it in the show notes and you could check it out yourself. My name is Ryan Donovan. I edit the blog, host the podcast here at Stack Overflow. If you have questions, concerns, ideas to cover, email me at podcast@stackoverflow.com and if you wanna reach out to me directly, you can find me on LinkedIn.
VISH ABRAMS: Thanks so much for having me, Ryan. This has been great. Once again, I'm Vish Abrams, Chief Architect of Heroku. Feel free to find me on Twitter or X as they call it now I'm @Vish, V-I-S-H or find me on LinkedIn and look forward to hearing any ideas anyone else have.
RYAN DONOVAN: All right, well thank you for listening today and we'll talk to you next time.
[Outro music]