The Stack Overflow Podcast

It Ain't Real Till You Break Prod

Episode Summary

This week we converse as a quartet, with Cassidy Williams joining the regular crew. We talk about how to make fun of yourself, breaking prod your first month on the job, and the best way to convert digital gold into cold hard cash.

Episode Notes

Cassidy helps to write The Overflow newsletter and is two months into a new gig as a Principal Developer Experience Engineer at Netlify. That's where she broke Prod, but it turned out ok.

We chat about Hey what it means for software engineers when prominent coders are arguing with big mobile platforms about the fees that the owners of the OS collect. What's old is new again. 

Bot armies are farming gold in World of Warcraft, which takes us down a wandering path of wondering how often people have access to powerful computers, but limited access to money they can spend on essentials.

Last but not least, we try to dissect a great question from our Software Engineering Stack Exchange: ways to explain code when told it doesn't make sense.

Shout out to our lifeboat badge winner of the week, "wizard", who answered the following question: is there an equivalent method to C's scanf in Java.

Episode Transcription

Cassidy Williams And as I was freaking out about it, it was kind of great because my coworkers were just like, no, it's okay. [Ben laughs] It's, it's not a real tech job unless you've broken prod.

Ben Popper Right, right, right.


BP Hey, everybody. Welcome back to the Stack Overflow podcast. I'm Ben Popper, director of content here at Stack Overflow. And I brought three wonderful people with me today. Want to say hi everybody?

Paul Ford Hiii everybody. 

Sara Chipps Hi everybody. 

CW Hello. 

BP Hi Sara. Hi Paul. 

PF Hey Sara. Hey Cassidy. Hey Ben.

BP We have Cassidy Williams with us today. She writes the newsletter in part for Stack Overflow, along with myself and my colleague, Ryan Donovan. And we did the dev surveys we do every year. We interviewed, you know, 65,000 developers. And what we learned was that the newsletter is one of the things that is making people feel most welcome in the Stack Overflow community. So I want to bring Cassidy on [woohoo] and say, awesome good job. First of all, [thanks] and talk a little about like your process. Yeah. Cause like how do you do it?

PF We have a chance here because we are also on the welcoming matrix, but let's be frank. We're a number of notches. This podcast is a number of notches [yeah] below in terms of its welcoming-ness.

CW Yeah, keep up! [Ben laughs]

PF Yeah, exactly. So, so I really, I want to throw it back to Cassidy. You know, I think this is a great conversation because I don't think people ever have this conversation in public. What do you do to be welcoming, especially cause we're in a community that is famous for being prickly.

CW It's hard to say, but the conclusion I've come to because people have asked me that when I'm like emceeing an event and stuff as well, is that I'm willing to be the butt of the joke, even if I'm the one telling the joke. And I, and I think that that is a very big factor because a lot of people in the development community try to be very cool for school and when you're willing to not be cool for school and you're willing to be the one that's making the terrible joke just to make people at least laugh like, Oh, come on. [right] I think that is a really powerful tool that not enough people take advantage of, of just being stupid.


PF You know, I'll tell you before I had to get on stage in my life and talk to people whenever anybody went, okay, let me hear it: ''I can't hear you!'' I would lose my mind. [Ben laughs] I was like, that is the worst thing. I just, why are you doing it? And now that I've been on the other side, I'm up there going ''I can't hear you!'' [right] Could you just, the audience has to be given a little bit of work to just get on your side.

CW Yeah. And, and that's, that's something with, I have a personal newsletter. I try to do it this with this one, I try to include some jokes in there that sometimes are subtle. Sometimes it's just like, Hey, this is a very, very low effort pun, but something in there where people might breathe out of their nose just a little bit faster, it'd be like, okay, I'll bite.

PF But not, not clever. Not too clever. No. [ahhh]

CW I'm never clever. 

BP Yeah. I mean, there's two options and you want to get the audience involved. You either go full. What was that? Microsoft's guy's name after? You either go full, full... 

SC Ballmer. 

CW Developers, developers, developers!

BP You either go full Ballmer and bring the energy.

PF Ballmer, yeaaah.

BP Full Ballmer or yeah, you're a little bit self deprecating and people respond to that and they're like, Hey, you're vulnerable a little bit. I can laugh with you and at you. I can relax. I think that's right.

SC So your tip is like make fun of yourselves a little more. 

CW Yeah. And I'm not saying like completely do self deprecating jokes to the point where people feel bad for you. [Ben laughs] I'm just saying like, be willing, be willing to just laugh a little bit.

PF This is real. If you are vulnerable in public, it's one of the hardest things to be. And you actually have to be in a really like specific place in your life before you can, because most people actually can't afford to be vulnerable in public. But once you get to that place and you have a little bit of power, it's one of the most important things you can do because then people, you will get email after email, after email message, after message, email. Cause I'm 40, 45. Other people probably are able to communicate through more modern means of communication. But people will be like, yeah, it, you know, it's not just me. Oh my God. They're so shocked. The regular message of marketing and thought leadership is I've got it figured out. And I will tell you what to do. 


BP Yeah. One of the most popular pieces we ever ran on the blog was by Ellen Spertus. And it's my most embarrassing mistakes as a programmer so far and just yeah. Huge reaction. And it's like a moment of, of community where people came together in the comments and were like, Oh yeah, I did that. Or let me tell you my story, you know, like kind of everybody can let their guard down a little bit.

CW Right. Even so I'm, I'm at my current job at Netlify. I've been here for wow. Almost two months now. That's amazing time flies. But I broke prod like two weeks ago. [yeaaah] It was, it was, ugh, a terrible feeling. I was freaking out trying to fix everything. 

PF Wait, wait, you broke like Netlify prod? Just like the Metta prod? Like the prod? [Ben laughs]

CW I broke prod. Yeah. It was not great.

PF Like breaking, breaking, prod on the service where everyone else is breaking prod when they're using the service is like that's. Wow! Okay.

CW Yeah. Luckily, luckily it was in an area where not a lot of people were at that time. So I was able to, one of the features of Netlify, you can do an instant rollback. And so it's like a click of a button to just go back to old things. [yeah]

SC Oh, so great!

CW It's, it's super useful. So luckily anybody who did get affected by anything, it didn't affect them for more than two minutes, but it was still terrifying.

BP Did you set a new record doing it within the first month or they were like, this happens to everybody within the first month.

CW I chose not to ask for my own self worth.

PF No, but what you find with cultures like that, where the focus is on pushing, like it always happens in the first couple of months. Like there's, you're just cause you're out without a net right away. 

BP Right. Right. Yeah. This is like that great question we ran through. The question was how to effectively work with teammates whose fixes to bugs cause more bugs. Actually this question is now been closed, but I thought it was a great question.


CW It's a real question. Cause that happens constantly. 

SC Yeah. And there's a lot of passive aggression involved, usually in those relationships.

BP Ugh I can't believe they closed this question is off topic after it got a great answer and was accepted.

PF Really? You can't believe that about the Stack?

SC Yeah, really. [Sara laughs]

BP I can believe it. I just, you know, come on, come on. [Paul laughs]

PF Ahhh.


BP Cassidy, I heard you been using Hey, which is the hot new email client. Can you tell us a little about what it is [yes!] and what you like and don't like about it?

CW Yes. So it is, it is a new take on email. We're disrupting the email industry, people. [Ben laughs] So it's Hey, H-E-Y, And you, it's an invite only thing currently, I think in the next like week and a half, they're going to open it up to people. And it's an email service. You sign up for a brand new email and they have a bunch of very opinionated features. For example, the inbox is now an imbox with an M because it's an important box now. [Ben laughs]

PF Ohhh.

BP Ohhh.

CW So the emails, the emails come in and if it's someone who hasn't ever sent you an email before, there's a screener. And so you can say yes or no, I want to receive emails from this person. 

SC I like that.

CW And if it's a no, then yeah. If there's a no, then you never get emails from them again. If it's a, if it's a yes, then it goes into your imbox and then once you're there, there are some, some cool little features in there. And then you can reply to emails, et cetera. You can mark an email as reply later.

SC Oh I like that. 

CW You can sticky an email. So it goes to another section of emails that are sticky.

BP Yon can umbox for unimportant things?

CW Well, and here here's the thing that I don't like about this system is there's no such thing as in imbox, imbox zero in it. So once you've, once you've addressed an email or applied to it or whatever, it just goes into a previously seen section and that previously seen section doesn't go away. 

SC Ohhhh.


CW There's always those, those emails that live there. And so I'm not sure how I feel about that. One other thing that's cool though, is there's a attachment aggregation section. And so every single time you receive an email attachment, it goes to the attachment section. And so you can search through your attachments really easily. And I think that part is great. Like if every email could have some kind of attachment aggregation, I think life would be so much easier for me.

PF Like a file system for your email, like you can see all the bits that people have sent you.

CW Right. Yeah. And so I can say, Oh, well I know that there is an invoice here that someone sent me, you can search by like date, by the person that sent to you.

SC Oh that's great.

BP Yeah. I do think that like, yeah, I'm often searching around being like, I want to resend that PDF. I want to resend that, you know, little audio and then it would be great if I was just in a dropdown on the right. So that's all hosted in their cloud or?

SC Good question.

CW Yeah. So that's, that is where it starts to get like, Oh, this is, this is something. You can't import your old email into it. [mmmm] 

SC That's rough.

CW And so you start a brand new email account on this platform, which is fun. 

SC God, how would you even do that? 

BP You gotta be dedicated, Sara. You gotta be dedicated.

CW That's the thing like for, for myself, I access old emails all the time and it would be a very long time before Hey could be valuable to me because I go back to my Gmail. I would have to so often, [yeah] and I even emailed them. I was just like, so is this just how it is? Because I could really...

SC Yeah, are they planning on doing it?

CW And they literally said, well, your old emails still exists. Just go back when you need them. And I was like...

SC I don't like that.

CW I will need them a lot. So that, that is, that is my hesitation.

BP I thought that was like an open, I thought that was like an open thing, like, like an RSS, but it's called like PMP or something where like, you just can map one to the other. Isn't that like...

SC SMTP feed.

CW Yeah. SMTP.

SC Yeah. I bet that just didn't open it. Is that what you think Cassidy? They're just like, not allowing for that?

CW I don't know, in their FAQ, they just said, no, this isn't a thing. [Ben laughs] So I'm hoping that that is just like a V1 thing. 


BP Right.

CW And that it'll come because I do think that it could be useful if I could have my old emails in it. 

BP Right.

PF Look this is, this is opinionated product culture, right? Like we're going to take, we're going to draw this incredibly firm line in the sand, see how people use it and what they need. And then we'll, we'll adapt accordingly. It's not a bad strategy because we're on here talking about Hey and trying to figure out what they want instead of them giving us what we want. So, yeah. So what you're talking about, Ben, I mean, this is, what's really interesting here to me is that they're, they're kind of breaking away with a lot of email tradition. Like even Gmail has, uses the classic email protocols underneath. Like I mirror all of my email to a local drive, make it searchable, then I can access it in searchably in my text editor, because I'm a specific kind of human being and I accepted that about myself [Ben laughs]. And so what they're saying ''eh, to hell with it, let's go web first.'' And there's been other efforts like this there's IMF, which has an old way of sharing out all your folders, right. To a client. Right. And then there's also JMap, which is the sort of JavaScript web based version of that protocol for faster web client development. But that can kind of also talk and it feels like they're saying, Hey, you know, we're all using this platform. Obviously it can send them emails and send them out and, and they support enough of the protocol to do that. But you're saying now we're going to go very webby. I, it looks a lot when I look at this screenshot, it just it's, you know, it's, it's the same people who brought you basecamp. So it's got that kind of very specific aesthetic.

BP We've got three, three talented programmers on this podcast. And one other guy, do you think you could just write a script, Paul, since you're already mirroring, like Cassidy, you were saying, can you just send yourself every Gmail you've ever sent and received? And then it would all just be like, just send it to, Hey, and then it would  be there in inbox, somehow?


BP No?

PF That's bad. Don't do that.

CW Sounds terrible.

SC You think you could do it, but it would be a mess. 

CW Yeah. I think it is possible. But I don't want to touch that with a ten foot pole. 

BP Right.

CW Email, email is hard.

PF Yeah. In a perfect world where it was two, just two computers talking that you controlled, you could absolutely do that. It's the least efficient way to do it. It's through, by actually resending the email, but you can totally do it. But the entire world of email now is set up so that you can't do things like that. Cause it is indistinguishable from spam. 

BP Right, right, right. Speaking of spammers, I want to just do one great bot story here that I saw this morning. There's always been bots or whatever. And back in the day, there were World of Warcraft bots that used to farm gold. [mmmm] And sometimes those would be bots, those that actually be like real people like click farms will have real people, but apparently bots are terrorizing the World of Warcraft servers, stealing precious resources, monopolizing rare monsters and inflating the virtual economy with truckloads of illicitly earned gold! Ugh! Bots.

CW No!

PF Wait. Okay. So wait World of Warcraft is still going?

BP Strong. Still going strong.

SC Yeah. Oh yeah. It's going strong.

BP Super strong. 

PF So bots have gone wild in Warcraft.

BP Yeah. And it's interesting that like your, well, I guess like right, you write a script, it creates an account with a bunch of other ones. And then they go in there and they act sort of like real people, but they're just very dedicated to doing certain things and they never sleep. But then right, the, the, the rub is...

PF That's indistinguishable from a World of Warcraft. 

SC Yeah, yeah, yeah.

BP From me and my buddies when we were 12. Exactly. But the rub is that yeah. This stuff can be converted into, back into real money. You can trade the digital goods for real money [ohhhhh] to people who are big time gamers. So that's why a hundred gold goes for $4.79.

SC Wow. It's not a bad conversion rate.

PF We're increasingly entering into this very strange world that we're not prepared for. Where people have access to computers in the internet, but are hungry. Like, I mean, there's like we never expected that we always expected that like with the computers would come prosperity of a certain kind. And it actually turns out that like electronics and telecommunication can coexist in a world where basic like basic human needs aren't getting met. And so then you enter the global economy and things like gold farming. And it's, it's not quite cyberpunk dystopia because it's actually like, you know, it's people, people are being creative and weird too. It's not just this like pure sort of this pure form of exploitation, but it is definitely a weird scene. And like, we're just kind of tredding along, like we're, we're, we're currently sharing probably like a hundred megabits of bandwidth just chatting. 


CW Yeah.

PF I'm I'm making, we're making gigs of files. We're doing good. And then we're going to go have our coffee and then it's just, it's just like such a big world out there. 

BP Yeah. We're just chatting for free with a level of internet data that would have cost $5,000 in 1999. 

PF Yeah.

CW Yeah.


BP Today's episode of the podcast is brought to you by Stack Overflow for Teams, the best way to organize and share knowledge across your company. It's used by small startups and some of the biggest companies in the world today it's free on the basic tier until June 30th with no credit card head on over to Check it out.


SC Uh, back to, Hey, should we talk about how DHH is losing it? [Cassidy laughs] On Twitter.

PF Okay. So wait, we, we need to, we need to back this out. So this is a, Hey comes from a company called 37 Signals and I'm assuming it's still... hey're still calling it that?

BP I think they call it Basecamp, right? 

CW I think they're all still Basecamp. Yeah. 

PF Okay. Basecamp. They renamed it Basecamp. Okay. So Basecamp is, are the people producing, Hey, and this is a friend that's been around for a while. And the person who created it was sort of the founder of Ruby on Rails is a guy who goes by DHH, which is David Heinemeier Hansson. Is that correct? 

SC Yeah!

PF Now they have submitted an app to the Apple app store. That's what I know. I just want to give our listeners some context before we go acronym wild. And Apple said no way, people are signing up for the service. You're going to need to pay us, you know, five hugs, head of wine and mead, which is what Apple always demand. Then what happened?


SC At first it was a little eye rolly, I think because, well, I, so this is, this is a real rollercoaster for me because my first reaction was welcome to the world of app development. I'm sorry no one ever told you that Apple takes a cut. Cause I was like, at first it seemed like, it seemed like he was upset about Apples 30%. [yeah] I have a text group with a bunch of other developers. So this is, this was our initial take. And then someone did the work of actually digging in after all the snark. And it looks like what happened was they didn't put in-app purchase and their app. So they, what they did for Hey is they have the app, but they made it so you can purchase it online so they don't have to give that 30%. But then Apple was like, sorry, you have to put in an in-app purchase or will not put it in the app store.

BP Right.

PF Well that's... mmm...

BP And I'm confused about that because I thought other people already did this. Like there are certain things with Amazon and Kindle or Netflix or Spotify where they don't offer the in-app purchase option. Cause they they're like fine. You know, we'll make it harder on customers to buy things like they can't buy directly through the app, but we're also not going to pay your tax. I'm confused about why Hey is getting singled out here.

CW Because they're making lots of noise probably.

SC Yeah, that's probably right.

BP Because Apple is letting some companies bypass the app store cut like Amazon prime video. So if you're a big dog and you have an ecosystem that Apple also relies on, you can get, you can get this special treatment?

PF You go in and you have meetings and you figure out a plan and people don't really talk about what the real deal is. I mean, who knows, it's also opaque. And then everybody yells about it online. And I mean, at some level, this has to get resolved at an antitrust legislation level because they have so much power. So does Google and they control the platforms and the access we have. And, and like if the cut is too great for companies to thrive, that is pretty rough. That means that we're not building a good ecosystem. And then Apple is going to have a very strong case bolstered by seven or 800,000 lawyers that everything they're doing is completely fine.

SC That seems like they're going after the little guy here. It seems like DHH might be in the right, getting upset.

BP Hard to look good when you're going up against a pretty small company like Basecamp and you're, you're Apple. I think it's hard to, to come out looking good.


PF I think the argument is that for-pay services tend to be like their corporate, like, you know, like Gmail would be a good example. Like you pay for your corporate Gmail account and then you get the Gmail app on. And it also, you know, and, and that, that allows you to access that. And they're not going to get in the way of that. Whereas this is like a, a slightly different deal. I don't know. It's so exhausting and confusing and Apple goes one way and Google goes the other, and then people get very emotional and reactive about it on Twitter. And I feel that this will never resolve and we will all deal with this for the next 40 years of our life.

SC Yeah, that's right. Yeah.

CW Alright. Cassidy, I have a question for you. 

CW Ooh okay.

BP Asked yesterday: ways to explain my code when told it doesn't make sense. This is something we all experience as animals trying to communicate with our voices or back when I was a journalist and I wrote something and didn't make sense. What do you three, think about ways to explain my code when told it doesn't make sense?

CW Well, my code is perfect, so I've never had this problem.

PF I will say I had an engineer take me aside not too long ago. Cause I wrote a little something as an experimental project and they went, you know, that wasn't bad. That was actually like you used reduce, like you did a few things. I was really, I was really surprised by.

SC Wow!

PF I know. You know, as a boss.

SC I thought he was going to roast you.

PF Oh, they would have, they would've. No and then I was like, look, maybe I should, maybe I should think about coming into engineering a little more deeply. And they're like, well, you know, we are looking for juniors. So they got it in there. 

CW Aw!

PF It's fine. You know, you gotta get, at least one knife twist is important.

CW I do think that there there's one answer in there. I did a very quick skim. There's one answer in there that I think is very true. Get people to look at your code sooner. Uh, so you don't want to be the person that is just like, I'm going to make a PR. And then it's like a 6,000 line PR and people have to scroll through tons and tons of stuff. It's I think it's very key. If someone doesn't understand your code, break it up a lot more and introduce certain smaller concepts at a time, document it properly. And then also just be willing to justify why you do certain things, be ready to go to a whiteboard and draw something or something, to be able to explain what you're doing.


BP I was going to say that like, you know, as a writer and an editor, that makes so much sense often if I'm working with somebody and just like, we keep struggling to get to a place where like the first draft or the second draft is something good. You say, alright, let's, let's go back. Write me like a structure, like write me the bullet point version. And then we'll get through that and then write me like the one page version. And then we'll get through that. And then we can write the five page version. Right.

PF Cassidy makes a really good point. And Ben you're telling me, you're, you're talking about editing, right? Like there is storytelling around technology is a deeply undervalued skill, right? Like here are some mental models that we can apply. You know, here's some ways to like being able to stand up and be like, let me tell you how the variable moves through the system and where the functions operate. And if you look at it and everyone listening to this might be like, well, it's not that hard. But if you look at what the masters have tried to do, like Donald Knuth with literate programming, walking through every single bit and telling a story around every piece of code. It's still really hard, like getting the abstract model of what the code is doing in your head and then narrating it is actually like one of the great unsolved problems in technology. And when you, when you go down to like the debugging level and you say, Hey, no, we're going to know everything that the computer is doing. So we can really understand what's happening here. It's completely overwhelming. It's just this nuclear level of information. Cause there's 2 billion things happening per second. And so this is hard. Like that storytelling is hard.

SC Yeah. Look at reading this question. It's really interesting. It seems like this person runs into this a lot, which says that if you, if your code is often hard to understand, I think that's like a bit of a smell, but what's interesting is that when he asked people what parts specifically seems to be confusing, they say the whole thing. So this seems like, I think you're right. Cassidy of, you know, maybe he's just doing these or they're just doing these huge pull requests and people are like, I don't know how to parse all this. Or maybe there's writing things in a confusing manner. 


CW There's also people who try to be really clever and sometimes...

SC Yeah that happens to me too.

CW You just have to be more boring. Like there was one time where I was just like, dang, this piece of code is ugly, but look how cool it is. And it was like a nested ternary statement that had all of these extra things happening at each level. And yes, you can do it. And it's technically one line of code, but why would you do that when you could just make it a tiny bit more maintainable by breaking it up into five?

PF You know, this has, this has always been the Python point of view, which is like, yeah, we gave you your Lambdas. You have them. But how about a nice for-loop instead? Like how about you just like, cause everybody knows what those are. They're even indented. And it's the great struggle in languages, which is I'm going to give you this efficiency so that you can show off versus here's what the poor person who follows you is going to have the easiest time understanding,

CW Right.

SC Yeah. Yeah. But they're showing off part is fun. 

CW The showing off part is very fun. It's, it's fun to be just like, look what I can do. Don't worry, I'm not committing this. And it's just like kind of human communication too. I can use really big words at you and explain something in a very convoluted way and it'll be really clever or I could make it more relatable to you. So that way you understand it more. And I think it's the exact same thing with code.

PF So I'm learning more about climate models related to some of the work we're doing in our firm. And these are multi like, they were like, it's like a million lines of Fortran, and each one and there's like 30 big ones and it is a world. And you go in and you read some of the Fortran and it's literally like vegetation equals 0.7. Like it's just like this little model of the world. And it's some of the worst programming you'll ever see. It's kind of amazing. And it all bundles up into this giant climate model, but it's weirdly also completely apparent because the scientists are just trying to get their science into the code, as opposed to showing how much they know about, you know, matrix transformations. So it's surprisingly readable, just like it's just very blunt force.



BP Alright y'all it's that time on every episode we shot a lifeboat winner. A lifeboat badge goes to somebody who got an answer score of 20 or more on a question that had a score of negative three or less. So thanks for saving the knowledge. Today we're giving it to Wizard. This is awarded yesterday. Is there an equivalent method to Cs scanf in Java? Do I say that scanf or scanF? 

PF ScanF. 

CW I like the name scamf a lot.

PF Scamf! It's like a, it's like a little gremlin guy who kind of comes along. It's me, scamf! 

CW I'm scamf!

BP Is there an equivalent to scamf in Java and there's some answers in here. So we'll put that in the show notes and everybody can check it out. Thanks so much everybody for listening. If you have something you want to share with us, you can hit us up My name is Ben popper. I'm the director of content here, and you can find me on Twitter at @BenPopper. 

SC Awesome. I'm Sarah Chipps. You can find me at SarahJo on GitHub. 

PF I'm Paul Ford. I'm the co founder of Postlight, a strategy design and engineering firm. Find us at

CW I'm Cassidy Williams. I'm a principal developer experience engineer at Netlify, and you can find me at Cassidoo all over the internet. 

BP Wonderful.