The Stack Overflow Podcast

Cleaning up build systems and gathering computer history with Adam Gordon Bell

Episode Summary

This week, we chat with Adam Gordon Bell, director of developer relations at Earthly and host of the CoRecursive podcast. We discuss Earthly's efforts to design better build scripts and why engineers enjoy telling stories about mistaken assumptions.

Episode Notes

As promised, here is the grass hat.

You can find out more about Earthly here.

We spend a little time talking about Nix OS the operating system you can roll back if you don't like a patch.

Raise your hand if you remember learning computer science with Turbo Pascal.

Maybe you didn't know, but discs aren't as slow as people think. Adam's recent episode is about upending common assumptions on IO performance. 

Shoutout to our Lifeboat badge winner of the week, Josh Smift, for answering the question: How to delete *.web files only if they exist.


Episode Transcription

Adam Gordon BellYou want me to describe what containers are? So the thing is--


Paul Ford No, don't do that. [Paul chuckles] We've done that about 30 times. We're never gonna get it right. So let's just move on.




Ben Popper Are you struggling to deploy cloud native applications to a hybrid cloud? Do you want to become familiar with Kubernetes and Istio? IBM cloud has a set of free, hands on training, ebooks, and an always on free tier of services to help you learn. Visit To learn more, that's Overflow. 


BP Hello, good morning, everybody. And welcome to the Stack Overflow podcast. It's me Ben Popper, Director of Content here at Stack Overflow with my wonderful co hosts, Paul and Sara.


PF Oh my goodness! Good morning!


SC Good morning! Good morning. 


BP Good morning. Sara spent a good chunk of his morning making fun of my new grass hat. I guess I'll have to put that in the show notes. Like I said--


PF Grass hat! Grass is not like an obvious term.


SC No, yeah no one knows what that is. [Sara & Ben laugh]


BP Sara saw it on Instagram, she was like, ''What was that thing?'' Well, like I said, I moved from the city to the country as part of my sort of like change in the pandemic. And whenever we go shopping, if we go to Target or Lowe's or something like that, my wife buys me the most rural camo hunting influenced piece of clothing she can find, so she bought me like a like a beanie that looks meant to look like grass, I guess, for when you're out stalking. [Ben laughs] I don't know why it has to look like that. 


SC It can definitely be used by hunters or murderers. 


BP I'm more of a nature Dryad Druid. It's what I wear to my Wiccans ceremonies.


SC That makes sense. 


BP No, people tuned in because we have a great guest today, Adam Gordon Bell, as someone who's trying to learn about programming, just sort of get my bearings, I came across the CoRecursive Podcast, which he hosts. And I listened to an episode with Matt Godbolt, called To The Assembly and just found it fascinating. Lots of stuff in there that I had never known before, kind of gave me a great overview of computer history, as well as understanding of binaries, and things like that. So, Adam, welcome to the podcast. Why don't you say hello, introduce yourself?


AGB Yeah. So, I'm Adam, I think you did a great job of introducing me. Yeah, I have a podcast about software development, we kind of try to take kind of a narrative approach, like people sharing stories about, like a problem they encountered and how they overcame it. Yeah. And I also work on the open source build tool that's called Earthly.


SC What does Earthly do? I'd like to know that.


AGB Build systems are really complex, like, they're just like, a messy area where like, developers build code, and then somehow something happens. And then it like, gets into production. And it's always just this messy area. And there's like one guy who knows how it works. And so we're just trying to make a tool that makes it a bit easier to to handle all that make an understandable way to describe like, how do you get from your, you know, your source code into some working software? How do you run tests against it, etc. You know, like, like Google internally has tools they use for for doing builds, in a way that really works for them. Twitter--


PF What's the name of their big build tool? Basil? Basil. 


AGB Yeah, Basil. 


PF Okay. And Twitter, you were saying?


AGB Yeah, Twitter has, I think theirs is Pants.


SC Pants?! 


AGB But these things are complex.


PF Yeah, yeah, of course, Twitter has Pants. Or it's called like Stock Market, it's one of those. I want to completely derail based on something you just said, because I started looking at Earthly as we're talking, tell us about it a little bit, because it's a container based build tool, and then that I will derail.


ABG Your build steps in Earthly will take place inside of a container, which means that you get some sort of isolation. And it means that you can run the same build process that might happen on the build server, also on your local machine, which is important, because a lot of times just something goes wrong, and I have no way to debug it. So we kind of are using containers as a way to isolate what's happening inside of the build.


PF So here's what I was updating, I turned on a laptop I hadn't in a while and then it was time to update Ubuntu on that laptop.


SC And then the laptop and wen about your day.


BP Walk slowly backwards.


PF It's only it's been six, maybe six months since I turned it on. But that's that's enough time for a lot of damage to happen, right? And as I'm doing it, and I'm thinking about our container as well, like, why am I not just adding layers to this thing, instead of constantly just ripping out the guts of it? and reinstalling it, like it feels like we're due for that. Like, Is there ever going to be a moment when we have fully containerized operating systems and all our all of our to root jails, like bear fruit? Or are we just going to stay in this world of like cross your fingers? I hope I did a back up.


SC When you say containerized, do you mean that instead of having to reinstall your entire operating system, you want to be able to switch back and forth?


PF Yeah, I really do. And it's like you have to reboot to do it. But I mean, just sort of like, why if I'm going to have kernel 5.0 and then I'm going to get that slightly new kernel, it's just like, isn't isn't? Aren't VMs just supposed to do this at this point? Like, can't I be done running disk upgrade and just go get the new Ubuntu?


ABG Have you ever looked at this NixOS?


PF No, tell me about NixOS!


ABG So, it's not containers, but it gets at the idea. It's like, the entirety of your operating system is like a file where it says like, what you're running, right? And then you can you can like roll forward, a kernel version and rollback.


SC Ohhh!


PF That's what I'm talking about! Immutable operating systems. That's what I want.


ABG I used it for the while the devils in the details, it's a bit a bit tricky.


PF Listen, I have nothing better to do. Let's be honest.


SC Do any of us? [Paul & Sara laugh]


PF I mean, it's just how many I run dist upgrade just to kind of relax, right? Like, you just want to see those packages come over. So yeah, this looks good. This, this looks like it has a ton of console output. That's what I'm about.


SC My strategy is to avoid it. Can you not avoid it on Unix? I am on OS 10. And I just avoid it as long as possible. And not think about it.


PF If you're running Ubuntu, and not doing upgrades, then why are you running Ubuntu? 


SC Yeah, that's why I don't!


PF These are soothing things like I have a command line email backer upper. That I wrote partially myself. 


SC Is it not Gmails backup servers?


PF Well, it kind of is, but it's, you know, it goes in certain places and runs a series of filters, and then it mirrors back to the iMac server that Google provides. Right. And so like, running the command, get mail, you know, I probably do it 20, 30 times a day, because it's actually like a little just like it's soothing. It's a little fidget toy, huh? Yeah. And you see the the Python output and you're like, ''Okay, I got all my mail. Look at me.''


SC Adam, do you have any like manual things that you like doing in your life?


ABG Do I have manual things that I like? I like checking Twitter. Does I count?


SC Yeah, do you have a script for checking Twitter. 


PF But see, that's the inconsistency of the experience is what's so tricky there. Because you could be like, you don't want something where you're scratching an itch, where you're like, okay, now I really feel that the world is about to fall apart.


ABG Like for me, like if I haven't kind of written some sort of code in a while, like, it can really bother me. It's like, I need to get rid. I need to do something. I feel like, I have some idealized idea in my head that if I'm not like creating something that something's wrong, I need to scratch that itch.


BP How did you get into this world? Adam wack? How did you start learning this stuff? What was your sort of entree to the world of coding school job? Where do you want to start? 


ABG Oh you want to do the whole life story? Well, I when I was in grade 10, we had a class and it was in turbo Pascal. There wasn't a lot of teaching involved. Like the teacher was like build a Yahtzee game. Here's a book on turbo Pascal, split up into teams. But it was super fun. Just like we built this like ASCII graphic like Yahtzee game where you like roll dice. And then you know, you have to like score things. And it was amazing. And I was just like, this is great. I want to do this. That's what I did. Here I am.


BP Now you're a game developer. 


ABG I don't know the game stuff got left behind. I'm not a game developer at all. But it was just fun, just to you know, type some stuff in see something changed. Like it's a pretty addictive little process, right?


PF Yeah, I wonder if that's one of the reasons why the web took off is that just reloading constantly is very stimulating. You're like, I changed it, I changed it. I always used to feel that good web engineers would have a callous index finger from constantly reloading from the keyboard. But I don't think that's true anymore. I think that things auto update in ways that they didn't use to it's no longer quite so manual.


SC I was trying to figure out what my version of therapy coding is. And it's like, designing a website that I like, like a layout that I like, for like, only a 13 inch MacBook Air and nothing else. And then walking away. You know, where it's like, looks nice on my machine. That's it. That's the end.


ABG I think like side side projects, like, especially if they're throwaway, like they're very soothing, because they don't have any constraints. Like it's like the messy real world that makes you like, okay, now I have to do this. Now it has to support whatever, yeah, different layouts. I have to authenticate against them. Oh, that's not fun. But if I just have to write a little Python script to do whatever, or just the real world makes things less fun.


PF Because you know, there's that Python script that solves a problem. And then there's the one that's packaged up where you've put like, a good command line interface and and you've document and it just, it's so easy for things to become no fun. I remember I brought a piece of code into work as an experiment, like a little labs project, and I've done it on the weekends and I had rotated the hamburger menu 45 degrees and the outrage and for me, it was just like, yeah, the hell with the hamburger menu. This is all Nonsense anyway. And people saw it. And we're like, I am so disgusted and horrified by this, like, how could you let this happen? And I realized I'd done a context shift. Like, for me, I was like, no one's ever gonna use this, who cares? And then it's like, it's gonna get released. And people are like, you know, that's anti design and just the content context change. It's not that they--they were right. Like, it made no sense. It was really bad. But it's definitely more fun when things are really bad and sloppy and messy.


SC So, was that the end of your Pascal? Or did you take it further?


ABG So what did I do? So I, I went to university for computer science, and I don't know we did on Java programming there, but also a lot of theoretical stuff. And at the time, I didn't really care for it. I was like, I want to build my Yahtzee games or, or whatever. But it's funny, because now, you know, with my podcast I and just like being later in my career, like a lot of the more theoretical computer science stuff is pretty interesting to me. So maybe, maybe I've come full circle now where like all I'll interview somebody who is like, a professor of computer science. Whereas before, I'd be like, let's just crank out code.


PF Is it valuable in your work? Or do you just want to know how the machine works?


ABG I mean, it depends on the area, right? So what I like is, is these like stories, I guess, were somebody can tell me something. So I interviewed this guy, Daniel Amir. He's like a French Canadian computer scientists. Right. And he was telling, like, the thing that got me hooked on him is he said, like, people always told him that discs were slow, right? So he was like, reading, parsing something from disk. And like, the parsing logic didn't matter much, because disks are slow, right? But he, he didn't think that was right. And he just did some performance checks on it. And he's like, turns out disks are really fast now, like, it's no longer the case, everybody kind of has this idea that like, iO is slow. And computers are fast. And then so he he ended up like digging in and building parsing libraries that were able to, you know, work at the speed of like, SSDs. I really like these stories where somebody's like, here's this problem I had. And like, here's the steps I did. 


PF I mean, I always noticed with serious computer scientists, especially the ones who got the industry there, they get into physics, right? They're like, okay, here's what the hard drive is doing at that moment. And now we're gonna need to scrape that data onto the drive for the real time, financial blah blah blah. You're just like, Whoa, there's an actual machine in there. Like there's there are things that are happening inside of the box, that have to do with little things turning on and off very quickly. And I'm like you and then it's like, you know, maybe I backed in through more pragmatic reasons. But as time goes on, I'm just like, well, there's science in here. And physics, and it's pretty cool. 


BP Adam, it's interesting what you said, I noticed this from our blog, when when people were trying to decide what to assign, or we're working with a client who's doing like a sponsored story, the thing that the audience seems to respond to most, as you said, is kind of that narrative process piece. And especially, I thought something was right, but it turned out to be wrong, or I built something this way. And then I realized I should have gone the other way. For some reason that seems to have a lot of appeal to the engineering mindset. Those are usually the pieces that perform best on our blog, maybe as well on your podcast.


ABG I have a long story about this, I guess. Right? So like, I've been working here in my home office for like, a long time, like nine years or something like that.


PF Where are you by the way?


ABG So I am in Peterborough, Ontario, Canada.


PF Nice. Okay, so you're, you're there. It's you've been there for nine years, I cut cut off your early story with excitement about Peterborough, Ontario.


ABG So the thing that that the thing that I miss, like with working at home is Yeah, like people sharing these things. Like, like Ben was saying, like these problems, right? Like, I remember, when I did work for a software company in town, you know, we would like, go outside for lunch? And then, you know, like, somebody would have a tale to tell right about something that went wrong. You know, something went wrong with this new rollout feature. And then you know, they'd kind of they kind of tell you, like, you know, the server was down, you're like, oh, did you check x? And they're like, Oh, I checked x, it wasn't x. Did you check Y? It wasn't Y? And then like, finally, you're like, was it Z and like, oh it was Z! And it's like--


SC Ohhh, yeah, that makes sense! Yeah. 


ABG And then yeah, you're kind of like, you go through this process in your head. And it seems like just wasting time, but I think of it in a way. It's actually kind of like training, right? They're like taking you through this like thought process, like their thought process. And kind of sharing this like folklore of like, here's how you solve this problem, right? And it's so much better than if you were like, oh, let's put a Google Doc together. And we can put like, the five things to check if x goes wrong, like you're not gonna remember that.


PF It doesn't, it doesn't work. It's a storytelling culture, right? Like you and you try to capture the storytelling, you try to get the folklore down. It doesn't actually function that way. I've tried, people don't fill out the Google Doc. 


SC Yeah. It feels like brut force.


ABG Or people don't read it.


PF And no one reads it. No one goes back to like, unless an editor puts it together is like interesting crisis reports. 


SC Yeah, you don't remember there's a Google Doc either.


PF Yeah, there's no incentive structure. Whereas if people are having coffee--the other thing too is you kind of want to keep your failures quiet, for the most part in your career. And so like it doesn't, it doesn't percolate outwards if people are talking like, ''Ah man, ah, Sally saw that like six months ago when she tried to up, you know, she tried to update Postgres to 12. And it turns out that all the nodes were screwed up, or just whatever it is, right? And you find, it turns out that everybody has those stories. 


ABG And they stick with you. That's the thing. I noticed, because like I've been doing, like, I've been doing my own podcast for a long time. And then when I thought back about it, the things that stuck out to me were the guests that kind of shared one of these experiences. So I try to double down on it, because it's like, then when you have a problem, you're like, ''Oh, yeah, remember, Richard was late for his kid's birthday. And there was blah, blah, blah. And it was the caching headers'' You're like, ''Ah, the caching headers.''


PF It's always course. Just, it's course. 


ABG DNS, it can be DNS a lot. 


PF Yeah, it's gonna be DNS, too. That's true.


BP Yeah. I mean, this reminds me of what Clive Thompson said when he came on, you know, he would be going to do these interviews for his book on coders. And sometimes, it's hard to get him to warm up, maybe the person wasn't, you know, sort of clicking with him in the interview. So he just say, ''What's, what's the worst bug you've ever seen? Like, talk me through that, tell me that story'' you know, and then immediately, the words would just start flowing, they start going. And by the end of that story, you know, they were kind of ready to talk about anything. I was gonna say this is making me miss lunch at Stack Overflow. That's what this is what it was all about. 


SC I don't know about y'all. But when I'm in that situation, I was like, I'm just kind of like brute forcing a solution, right? Like, I'm just try that, you know, when I'm in that world of trying everything--that happened to me yesterday. I was, I was playing with Arduino. And I couldn't get it to work. And I was really frustrated. And I tried everything. And with hardware, you know, you're switching, you're not just playing with different ways of approaching the software, you're also thinking about how you're connecting the hardware? And is it this this USB cable? And should I switch it out with this one? What about this hub and that kind of thing. And then when I finally figured it out, it was like three hours later, I downloaded the wrong library. But when you're like in that world, you're just kind of you kind of feel like you're just sometimes it can feel like you're just throwing spaghetti at a wall to feel it, figure out what's going wrong. So when you fix it, it's really nice to tell that story.


PF I mean, it's ironic that we're doing this from on the Stack Overflow podcast, right? You know, there's a definite like, we're all touching different parts of this elephant. That is a folklore gathering. Right? And but even so like, it's super valuable. But there's a some sort of tissue and just listening to, because you hear the sound of their voice, and you hear the frustration and the confusion, and you kind of get a human story connected to the work, as opposed to the answer. This is very tricky, right? Because when people advocate for remote work, they often advocate for a lot of these things like we're you know, or we can write it down, we're going to have these tools. And that completely works like but that remaining 10% is hard to find. Alright, so starting a podcast, listening to stories, trying to maybe get some of that oral culture back into your very written culture life as a remote employee.


SC Adam, you're saying you work with a lot of--or you talking to beginners? Or you know, and one thing I've realized, and one thing I try to tell all the beginners that I interact with is something that's a really senior skill is the confidence and ability to say when you don't know something, and I think, you know, just I don't know, you know, what is that thing? I don't know. Can you tell me what that is? I think another part of that senior skill is being able to when you hit that problem, being able to quickly and effectively communicate what the problem is, in a format that is, ''Here's what I'm seeing, here's what I've tried, here's what I am going to try after this. Here's all my system configurations. What do you think?'' I always think that's really interesting. 


BP And to what Paul is saying about this being, you know, a Stack Overflow podcast, I mean, it is interesting. Stack Overflow itself is like sort of very circumspect. There's not a lot of storytelling, but across the Stack Exchanges, like if you go to the software Stack Exchange, there is a lot of that, you know, well, let me sit back. And let me tell you about how I did it wrong five times. And then maybe we can work through your problem, which is nice.


ABG Yeah, I want that I want I want the narrative version of it. I guess. That's like, I was late for x. And I was wearing this shirt. And then I figured out it was a DNS issue. I don't know. It makes it stick in my head.


SC That's great. Well, how do you find your guests? What do you look for, what kind of people do talk to you?


ABG It's hard to find--I don't have a good answer. But I would tell if you're like, Hey, I had this really hard problem. And here's what it was. And I found out a way to solve it. And it evolved, like computers and code. And then on the other side of it, I was a totally different person. That's who I want to talk to you. Right? Those people are hard to find, I guess. But like, you know, Ben, you were talking about the Matt Godbold story, but I think at the center of it was this idea that, like he wanted to know if some new c++ stuff was going to screw up his high performance testing. And he had this whole background of like writing assembly code. So he was like, why don't we just read the assembly? And then the kind of like, change things because people were like, ''Oh, yeah, you could just look at the assembly and see what it's doing.'' Like we're so far from the place where people think about reading assembly that when he said that, like that was like a revelation. And he's become kind of famous for being like, ''Oh, let's look at the assembly!''


SC I could just see that meeting. I'd be like, ''Ah, that sounds great. I've got a I've got an appointment. I'll be back in like an hour. Y'all want to get started?'' [Sara & Ben chuckle]


PF There was a friend who is helping me understand a lot of stuff. And I was like, ''What is a recursive function?'' He was like ''Really?'' And he's like, ''Really?!' He's like, ''Alright, let me show you how spcl compiler produces assembly'' and he showed it to me. And I was like, ''Oh, it's just a function that calls itself over and over again, with arguments.'' And he's like, ''Yeah, I mean, it looks like that in assembly too.'' It's wild, right? Like, it's that output is right there. It's not actually is, it's not mysterious. It is a lot. It's there if you reach for it, but it's actually it's almost like this, like secret in our culture. Like we kind of keep hiding, for good reasons that you can do that, that you know, and it's, it's not like you're, you're in your web browser, and you're gonna like suddenly see your webpages as they are in memory in assembly, but you kind of can, if you  learn every little hex. I don't know, it's good for you. It's good for your soul.


BP Yeah, like Adam, thinking about the two stories you tell one about the, you know, reading to disk and this other one about assembly. And they're both kind of about things that you may have forgotten that there's kind of like a received wisdom now. And then if you go back and look a little closer, it may not be true. It's almost like the like Malcolm Gladwell approach, right? Where it's like, what I'm really doing here is just taking you to the 'aha moment' you know, like, there's like some there's a there's a lot of, you know, winding paths here and like, you think you were going one way, but like the payoff is yeah, that 'aha moment' which is like you were saying, that's like the fun part of narrative.


ABG Yeah, I mean, you've probably given me too much credit to equate me to Malcolm Gladwell. Because I'm just finding people with interesting computing stories and having them listen.


PF This is an extremely Canadian response, right? 


SC Yeah!


BP Malcom Gladwell, also Canadian.


ABG That's true. Yeah. Yeah, I don't know. I love these stories. I so I interviewed this, Brian Kernighan, who, you know, was involved in the early days of Unix. And, like, I had this idea, like, hey, tell me these stories of when you create Unix, because it's gotta be like, I was like, you know, maybe he'll tell me something that's just like, their secret to being so like, impactful, you know, this group of people who build such important stuff. And he was kind of like, Hey, we didn't really think we were doing important things. In fact, like, he started off working on like theoretical computer science research. But then the group kind of spent a lot of time just like mucking around with computers, building operating systems. And they didn't think it was important. They just thought like, hey, computers are kind of cool, but they're hard to work with, how can we make it better? So that was like a big aha moment for me. It was like these people. I'm like, ''Oh, my God, what was your secret? How did you decide to build this really impactful thing?'' And they were like, ''Oh, we were just kind of messing around. We had a bunch of smart people.''


PF Well, then they went, they went like shopping for applications, right? They started working on at one point, it was typesetting, the patent applications or something like that, just because they're inside the giant phone company. And they had to, they had to create an excuse for why they should continue to build Unix, which is now the foundation of like, almost all non Windows computing that goes on in world. 


BP How do you drum up more resources inside big cut? Well, I know you're having trouble with these patent applications, they're stacking up over here. So lend us some money and some machines, we'll figure that out for you. 


ABG So that's totally, it's a super interesting story, because like Bell Labs had wasted a bunch of money and time on this operating system stuff in a previous project. And so they were like, nobody's worked on operating systems.


PF It's waste of time!


ABG Yeah, they had to make up that patent story. Like it was basically a lie to say like, ''Oh, yeah, if you let us work on operating systems and buy us a new computer, then we can make these patents stuff work.'' And they did, but it was it was just a bluff. Right? They were like, ''Give us an excuse to to work on this.''


PF Well, they didn't build patents from 2000. Right? They build Unix that had a typesetting and markup--


SC Really inefficient way. Yeah, exactly. Yeah. Yeah. 


PF But frankly, isn't this how every nerd sells problems, it's just like it, you know, I'm gonna pick a couple levels of abstraction down because this will be so much more effective. If we have a general typesetting solution for everything. And then we can apply it to patents. And then like, like, I remember this Donald Knuth with with tech, the typesetting system. At one point, early days was like I gave her I gave us a goal of nine months to put one letter on a page. And it was just like, yeah, that actually was about right. Like, it took nine months to get one letter on one page. And you know, but then from there, you could go on and sort of change scientific typesetting.


SC Yeah, my engineering manager, goes into high gear when I hear that because, you know, you're it's kind of what you think about when you're working with a team because I think it's really easy for anyone to get too deep into rebuilding a system. So when you start to hear, you know, I wonder what would happen if we went down to assembly you're just like, oh, wait a minute. Do we need to do that?


PF This is what colleges are for.


BP Yeah, Sara's not falling for this. This is an easy Jedi mind trick. She's not falling for it.


No, but like college, college projects, university projects, master degrees. Because frankly, if the as long as there is some outcome, and they can write a paper or a thesis about it, it's been at least a qualified success, right? Like it actually working software doesn't, that can be globally distributed and used, especially via a web browser is not the goal of the master's thesis. And so they can get away with it. And I think that's really important. Because I mean, you know, I'm in the same boat, which is, the projects I work with, the projects that you know, I sell it at my company fulfills are entirely based on them existing at some future state. And that really limits the range of choices you can make when you're when you're starting to explore and build something.


ABG I mean, I don't recommend this, but like, I pulled this trick before, basically their trick, which is, you know, Hey, I just learned about new technology X, and then somebody's like, hey, how do we build Y? And you just say, like, X is the best way to build Y for sure. Right? [Sara & Ben laugh] 


SC Yeah, I've seen that before.


PF ''This new web browser will be based on ZFS.'' Yeah, yeah. Just I love it when it's like the wrong combination, where it's like, ''Oh, yeah, absolutely. We want to use low, low level socket interfaces to create a new shoe shopping site.''


SC Yeah, And that, my friends, is why many fortune 500 companies are still trying to figure out why they have eight JavaScript front end libraries. [Sara laughs]


PF Well, that's, you can see that urge, right? That collective urge to go that one level of abstraction and kind of encode all your knowledge, because I'm gonna make this so much easier for everybody else. 


SC I'm gonna build this one page in Angular, and you guys will see, it's gonna be really awesome.


PF Spelt is clearly the one that's going to come in now and just make every, it's gonna be so bad, because it's just right on that edge of like, wow, I just had it. When someone says they had a conversion experience. You're like yeah, alright, we'll be maintaining that for 12 years? 


BP What did Teresa say when she's on the podcast? If somebody is trying to commit to this, she says, that's fine. As long as you deprecate something else we were doing right, so it doesn't keep stacking. That's the trick for letting somebody else try something new.


PF Back to react.




BP Alright, everybody, it is that time of the episode, I'm going to shout out a lifeboat or somebody who earned a badge for answering a question that had a score of negative three, and got it up to a score of 20 or more, creating a nice little knowledge artifact on the web. So today, it is: 'How to delete dot web files, but only if they exist' asked by Miki four years ago, and answered by Josh Smift. So thank you very much, this lifeboat badge was awarded yesterday. I'm Ben Popper, Director of Content here at Stack Overflow. You can always find me @BenPopper on Twitter, and you can email us


SC And I'm Sara Chipps, Director of Community here at Stack Overflow. And you can find me at @SaraJo on GitHub.


PF Sara, this is a question for you. Adam, you are not allowed to answer this. Why is Peterborough, Ontario called the electric city?


SC I don't--should I guess something? It's where Canadians invented electricity.


PF Very close. It was the first town in Canada to use electric streetlights. That's from the Wikipedia page.


SC Fascinating! Wow!


PF So now, good internet. I'm Paul Ford. And you can check out my company I'm a friend of StackOverflow. And Adam, how do people reach you? What do you want them to know? Where did they find your podcast? Tell us!


ABG Okay, so yeah, I'm Adam Gordon Bell. You can find me on Twitter @AdamGordonBell. The podcast is is CoRecursive. You can find it wherever you find podcasts. And yeah, check out Earthly, the build tool. It's open source. You can find it at That's all I got.