On this week's episode, we discuss where CI/CD breaks down and how Paul and Sara approach it.
Today's conversation was inspired by a great blog post from Charity Majors.
We also discuss the Chrome team's decision to migrate Puppeteer to Typescript, and the way in which large tech organizations are increasingly interconnected by a set of open source tools and platforms.
Lastly, we discuss the impact expanded funding for community colleges could have on the pipeline of software engineers entering the job market.
Today's lifeboat badge winner is Abdul Saboor, who answered the question: How do you convert negative data into positive data in SQL Server?
Paul Ford Free community college would be really good at enabling the tech economy, you'd get, you'd get paid back.
Sara Chipps Mmm, that's a good argument that I haven't heard.
PF I just made it up right here. But I mean, you'd get a huge return on that investment, because more people would be able to be more productive, just flat out.
[INTRO MUSIC]
BP Hello! Good morning everybody!
SC Good morning!
PF Alrighty!
BP Hi Paul. Hi Sara.
PF Hi!
BP Welcome to the Stack Overflow Podcast.
SC Welcome! Everyone!
BP So I have a couple of stories here. We could chit chat about that I pulled off the old webski. Migrating puppeteer to TypeScript. Sara, we're a dotnet shop. And I know we've talked a bunch about TypeScript before. Is it normal for you know, big companies like Google or Microsoft, Amazon or Facebook, to pick up each other's tools and languages like this when they're really good? And does that mean something to have them like, you know, blend together like that, or it's just, you know, it's fine to take somebody else's tool and run with it when it's working?
PF Wait, who's using TypeScript?
BP The Chrome Dev Tools engineering blog, told me this morning that they're migrating puppeteer to TypeScript. They love it so much. I know TypeScript we talked about a lot.
SC Yeah, I think it's when those tools become best practice in the industry, it's kind of hard to avoid it. And I don't even know that you people do avoid it. Because it's open source, right? If it was proprietary, then it would be a lot harder.
PF Go is a good example there. Right? Like there's Go all over the place.
SC Yeah, that's a great example.
PF React is another one, like people have a lot of both sort of personal reactions to Facebook and a lot of competitors. But everybody uses React, it is seen as slightly outside. And frankly, if Facebook said we're never going to support React again, it would be fine.
SC Yeah, people would step up.
PF So I think it's that, you're in a, you're in a position where it's like, okay, here's TypeScript, we have, we have all the code. And if Microsoft had suddenly said, out of the blue, we never would have touched this thing again, you'd still be able to get the benefits and update the codebase. So it's a calculated risk.
BP That must be nice for like software engineers, because then you can move between any of the big companies. Like you said, there are these universal tools that are the, like you said, the best practice and so they're used across all of them.
SC Yes, yeah.
PF This is a complicated thing about open source, right, which is that, you know, there are people who are working on new licenses, that maybe would make it impossible for the military to use your code and, and sort of thinking those things through. But in general, once it's in the commons, anyone can pick it up and manipulate it. And so there's huge advantages to that kind of cultural flexibility, including cooperation between enormous organizations that normally have absolutely no place or way to cooperate. And it sort of supports and you know, like Microsoft doing TypeScript is overall a net good for our industry. And ideally, we'll bring more people towards, you know, better clear code, regardless of your opinions and thoughts on Microsoft, and so like that. But the regardless part is because it's open source, you actually can, you can kind of forget Microsoft and focus on TypeScript.
SC Yeah, it's kind of like, it's really neat, because of the open source nature of it. There's a lot of natural trust. And you can make it through a lot of the procurement process, because everything is open.
BP Oh, that's interesting. I have another story I wanted to bring up. It was written by Charity Majors, and we put it up on the blog this week, it did really well. So the premise here is everyone talks about CI/CD all the time. But everybody is really just talking about CI and not really focusing on CD, even though in the end, that's, you know, something, you need to have that clear path, or you're gonna end up in a lot of trouble. Yeah. What do you think about that premise? Is that something you've experienced?
PF Let's, let's define what those things are.
BP It's a process and methodology designed to make sure all the code you merge to main is deployable at the time by testing and deploying it automatically after every diff.
Pf What does CI stand for? What a CD stands for?
SC So we're talking about continuous integration and continuous delivery.
PF And they're saying that, so the integration is I put it into GitHub?
SC Yeah, I put it into GitHub, I merge it with the thing.
PF And the delivery is I ship the app.
SC Yeah, yeah. And I think one does lead to the other. But I often see people be purposeful about the first one and less purposeful about the second one, because it's a tough problem to solve. Unless you're focusing on them both together. The first one definitely does get prioritized.
PF Well, you need a product manager who is focused on driving the product to the customers in collaboration with the engineering team at that same pace, right? And that it's actually like an engineering culture where everybody's coordinating on Slack, there's kind of no other dependencies aside from we're gonna make the product better. We're gonna we're gonna meet our goals. Maybe we have a process here that helps us do that. They're chatting and they go, okay, I'm gonna push it and someone goes in the process is literally looks good to me. They give it a thumbs up and now you have continuously integrated, it goes in, the tooling is great. Yhat next button where it goes out with development notes or it goes to, you know, a lot of larger orgs still have QA and testing where before things had customers, they want, you know, classic scripts to be run, so on and so forth. That product manager role which can come from inside of engineering, I feel that that is kind of a great Undiscovered Country in most very large orgs where, you know, they need to get stuff into production, but there are these certain barriers that show up and that person who that that level of process and that simplicity of like, looks good to me and out it goes, isn't quite the same as it.
SC Yeah, as an engineer my motivation--I mean, not to oversimplify things--but there's a big motivation for me to want to integrate my code. Like I definitely want to ship at some point, right? Like, I want to ship but do I care about shipping as much as I'm integrating with code? Do I care about how often--
PF Well that's not your job, right? Your job is to make a really good, your job is to make the product good, but not necessarily get it in front of the users. And that's products job, and again, can be technical product, it doesn't have to be like some magical product or program manager coming in. But it's like, so you know, I mean, one way to solve this is like once is to give engineering full responsibility for quality and to say, right, and to automatically deploy. A lot of the sites, a lot of the new tools like Netlify, or some of the Vercel tools kind of do that for you. Once they see that the CI is done, they Auto Deploy off into a branch. If you're in a branch and out it goes into the world, and then once you merge with the main branch, it goes out fully into production. And so what you have is his auto deployed branches as the equivalent of the staging and testing server. And then, everyone in the org--frankly, this is how we do it at work--everyone in the org has that responsibility. We don't do a lot of QA.
SC Yeah, that was my question.
PF Yeah. No, it's owned by the team. You want a well architected platform that's highly testable, automated, but frankly, product engineering and design need to be in there looking and taking full ownership.
SC Clicking around?
PF Yeah, and there's a million people who will tell us that's wrong. But I would actually not.
SC I'm interested in having that conversation. Do you find that the onus comes on product and design to do a lot of the QA in that case?
PF No, no, engineering takes responsibility. That's it. Yeah, it's not even. These are big teams you're using, if you have the concept of an app is 36,000, separate interconnected components, each one of which must be tested before deployment, you've introduced unbelievable friction, right? Like a little common sense. Like, here's what we're deploying, we need to make sure it works. We didn't touch any of this over here means that you know, somebody's running a script for three days, two days after deployment after staging goes live is not, isn't necessary. It just isn't. Yeah, this is to me one of those things like, okay, I'm trying to think of a good example, but like, you know, the old school, QA culture is a reaction to how engineering used to work. And the cost of change and repair is very low. You know, I mean, this isn't, we're not building space shuttle software, here there is you need to make sure, you know, database integrity is it can be assured in a lot of ways. And certainly data integrity, to me would be the one thing that you want to be absolutely paranoid about, you can't mess up user records or have privacy challenges, and so on. And so you're going to take a lot more time and care. But ideally, you're not changing your schema very much, you might be moving a widget, updating a design, so on and so forth. And they're very understandable frameworks for saying, I did a good job or not. And for people looking over your shoulder and taking responsibility. And utterly you just have to take responsibility, like a, you know, a director of engineering, see something that isn't good and says, I don't want that out. You can't do that here. Let's fix it. And then you incentivize that, you know, the people who say I made a good, improve it are the ones who get the rewards in the organization. And it's a pretty good culture.
SC I found that developers, well intentioned or not, aren't always the best at finding issues with their especially on front end, do you have a lot of front end testing? Or do you feel like you have a, you've worked on a culture that that level of responsibility has driven developers to become a little bit of better at their own QA?
PF You can ask people to take responsibility for their own work, you really can. I don't like that engineering culture where it's like, well, I did the code. Nuh uh. I don't usually go into that, that mode when I'm talking about my work as a boss, but it's like, no, it's good. It will be good, you must make it good. And if you didn't make it good, why bother? Like go program somewhere else. And so it actually like the organization is set up, I don't hire engineers, our engineering team does, but the organization set up for that filter. We actually, were one of the places we do a paid code sample when people apply. And it's a way for us to see the care that people have.
SC I like that.
PF There's nothing perfect in hiring, there is no perfect solution, but paid code sample is at least like we respect your time, you got through filter one, let's make sure that this is meaningful. We try to keep the challenges, you know, somewhat interesting for people who like to work.
BP Paul, what you said about the automation in the branch makes a lot of sense, because in this story, it was saying, most deploys, you know, run by hand, at some intermediate interval after the code was written in, then you're kind of divorced from it. And it gets worse because a lot of people's work goes in, it's all batched together. And so, you know, there might be changes in there that had nothing to do with that what you said, you know, I wrote it well, the code is good. So it'll work in production. But you know, you have to bring, I think you said like together as a team and own it front to back right?
PF You know, in as I'm saying all this, I know that our developers engineering leads standing up in their chairs saying "You go to hell, buddy, you don't know anything!"
BP That's the point of having a podcast. Throw out strong opinions.
PF Emails, right? Like, I mean, you know, "I did a Salesforce integration five years ago" like I get it, like I actually get it. But I swear, I swear to you on the deity that you prefer to think about, that you can have high quality software with the ownership of the team, being the main driver of quality.
SC Ella Emhoff's inauguration coat is my new deity that I swear on.
PF That was a good one. It's beautiful. So for people who don't know, and I'm gonna assume this really applies to Sara and it also applies to me and my cohort. Ella Emhoff is the second gentleman's daughter from his first marriage. And she lives in Bushwick, Brooklyn, which is kind of where cool kids go. And yeah, she's cool. She's crunchy, and she knits and she made her own coat. Sara, wow would you describe the coat?
SC It's outstanding. It's like a--it's not a houndstooth, it's like a plaid. There's, I'm sure there's a word for this type of plaid. It's like, it has gems throughout the shoulders and the top of it and just like an absolutely fabulous color.
It was bedazzling.
PF No one is cool in politics. And it was just like this moment of actual sort of crunchy coolness that you just never see. Although I was immediately thrown back to the TV show Veep. The daughter has some aspects like that. But regardless, she did a cool eyebrow, waggle. And it's just, you know, I'm happy to have some knitting back in our lives.
SC I never thought I'd say that.
PF Let's, you know, let's knit more, we need it.
BP It's also fun when, yeah, the kids in the White House are a little bit embarrassed about it. Like they feel awkward. I enjoy that relationship. It's kind of like, "Dad, I get it, you're the president. I get it, Dad, I get it, Mom."
PF I mean, if you're uncool lawyer, can you imagine how exhausting it is? Because you'd be like, well, I, you know, I really do want to go see, you know, whatever band. And they're like, "Well, let me talk about the parameters." And you're just like, "Oh God"
SC Yeah, if you want to go on a date, and the Secret Service is behind you.
PF And you're cool, like you're actually cool. You just want to go to a warehouse party. See, see like a noise ban play. The one that I had the most respect for is--I grew up in Pennsylvania, I've been, southeastern PA, I've been Joe Biden, proximate literally my entire life. There's never been a time in my life when I wasn't fully aware of Joe Biden. And I have heard a lot of those stories, you know, probably 36,000 times. And I look at Jill Biden, Jill, who is able to sit there and hear him like talk about driving a car for the 400,000 times. That is the stability and firmness that our country truly needs. [Sara & Ben laugh] Do you know how many times? And I wonder what she thinks about like, I think she's grading papers in her brain.
SC Yeah. Oh, that's nice. Yeah. And teachers are always very patient people I find.
PF Yeah, she is. She's a big fan of community colleges, which actually, let's bring it back to our world, right. Like, that is a place where lots of people are learning to code, along with with boot camps and stuff. Like these are really integral parts of getting technologists into the economy. Community College, Boot Camp, there's and org I love called Vets Who Code, that I love.
SC Mmm yeah great one!
PF I would love to see more emphasis on that like, that is how you--you don't have power this economy, the tech economy, necessarily by releasing lots of new software, you know, at a high level, or partnering with Microsoft. You do it by getting people to learn JavaScript.
SC Yeah, more developers.
BP That was sort of like the the angry bottle meme is like, why don't you learn to code? It's like, well, I'm can't just afford to go, you know, stop my job and get educated. But that would be something kind of amazing if they did going forward, you know, some sort of incentives for areas like that, where there are a lot of unfilled jobs. Like there's, there's more open positions then there are people.
PF And you could really argue that what our industry needs, because there is a tremendous need for new developers. And there's actually just sort of needs for more people with development skills and a lot of roles, right? Like not just programmers, but people who can do good work with AirTable and just light data management.
[MUSIC]
BP Alright, everybody, it's that time of the I'm going to read a lifeboat badge winner. Convert negative data into positive data in a SQL Server. Thank you to Abdul Saboor awarded January 18th. We'll throw that one in the shot. So thank you Abdul. I am Ben Popper, Director of Content here at Stack Overflow. You can always find me on Twitter @BenPopper and email us podcast@stackoverflow.com. If you listen to the show, and you like it, please do go leave a rating and a little review on all the platforms where you listen to podcasts. It helps out a lot.
SC Like and subscribe. I'm Sara Chipps, our Director of Community here at Stack Overflow. And you can find me at @SaraJo on GitHub.
PF Hey, I'm Paul Ford, friend of Stack Overflow, check out my company Postlight. And wow, are we hiring, if I didn't completely alienate you with my opinions on QA. [Sara laughs] If you want to take some ownership over quality and your role as an engineer. front end, back end. Please, please check us out Postlight.com/careers.
BP That's right, go get that paid interview. Take it as many times as you want.
PF Well, uh, no.
BP Forget that part. [Paul laughs]
PF Alright!
[OUTRO MUSIC]