In 2019, as the original co-founders and hosts moved on, the Stack Overflow podcast was rebooted with a new cast of hosts. On today’s episode Ben Popper, Cassidy Williams, and Ryan Donovan sit down to discuss how much has changed in the five years they have been collaborating on Stack Overflow’s blog, newsletter, and podcast. If you you are sick of AI talk today, remember how bad the crypto craze was just two years ago!
Cassidy reflect on her time as a CTO of a startup and how the shifting environment for funding has created new pressures and incentives for founders, developers, and venture capitalists.
Ben tries to get a bead on a new Moore’s law for the GenAI era: when will we start to see diminishing returns and fewer step factor jumps?
Ben and Cassidy remember the time they made a viral joke of a keyboard!
Ryan sees how things goes in cycles. A Stack Overflow job board is back! And what do we make of the trend of AI assisted job interviews where cover letters and even technical interviews have a bot in the background helping out.
Congrats to Erwin Brandstetter for winning a lifeboat badge with an answer to this question: How do I convert a simple select query like select * from customers into a stored procedure / function in pg?
[intro music plays]
Ben Popper Don't start building your AI app from scratch– save time and effort by visiting intel.com/edgeAI. Get open source code snippets and sample apps for a head start on development so you can reach your seamless deployment faster. Go to intel.com/edgeAI.
BP Hello, everybody. Welcome back to the Stack Overflow Podcast, a place to talk all things software and technology, five-year anniversary edition.
Cassidy Williams Woo!
BP I am Ben Popper, I'm the Director of Content here at Stack Overflow. I get to go on a fun little sabbatical this year, a Stack Overflow perk for having attended as an employee of five years. And we brought two other great co-hosts today: my colleague Ryan, and Cassidy Williams, our longest and strongest editorial contributor/collaborator since 2019.
CW I don't know if I'm strongest, I'm actually pretty weak, but I'll take it.
BP Why am I throwing out superlatives at the start of the show? Anyway, the three of us have been working together since 2019 on the Stack Overflow Blog and Podcast and newsletter, and we wanted to get together and just reflect a little bit on what's changed in that time, what we're seeing today in the industry, and a little bit of what we're looking forward to in the year to come. So let me step back for a second. Cassidy, I'll have you start, but I was connected to you through Sara Chipps, who was our Director of Community at the time, and she said you were great. I don't know how you two met, but you had been at events and on panels and speaking gigs together and you came and visited New York back when we had an office pre-pandemic, pre-remote everything, and did a little recording session about GraphQL and then you wrote a blog post for us about a similar topic. So five years ago to now, what's on your mind?
CW Wow. So much has happened, throw in a pandemic with everything else. It's been so long and it's wild. So Sara and I had known each other just from the women in tech scene in New York –there's dozens of us– but then we kind of stayed connected and she liked my personal newsletter which I've been writing now for seven years, and she was just like, “Hey, Stack Overflow needs a newsletter. You want to help us out?” And so that's kind of how we got connected, and it's been a time. Back then I was working, let's see, I think I was working at CodePen back then, and I was transitioning into a full-time coding teaching role at that point, which I loved. It was a team called React Training and they're the folks behind Remix and React Router. I think we've had them on the show a couple times since then. Loved doing that, but then that all got shut down when the pandemic happened, and thus, new job changes happened and new cities happened. I ended up moving from Seattle to Chicago. It's been a wild ride, but Stack Overflow, the podcast and the newsletter has been my constant.
Ryan Donovan I feel like since we've been here five years, you've had a whole bunch of different jobs and a whole bunch of big changes. It's felt mostly the same for me except for the tech around us.
CW It's been similar but different where it feels like I've jumped around, but really it's just kind of slow transitions over time. And it's been cool. I think we've all had children in that time, too. Man, it's been wild how much life is different.
BP Cassidy, you've changed jobs the most. I probably lived in the most– I've lived in five different houses since the pandemic started.
CW Oh my gosh.
BP Not all for a long time, but months here, months there, years here, years there. I moved around a lot as I was trying to figure out how to get resettled. And for me, there has been the constant of Stack Overflow, but it just feels like so much has changed. When I started back then, talent was our biggest line of business and sort of the forefront of what Stack Overflow was doing. We stopped doing that completely for a while. We recently announced we're getting back into the talent business in a certain way with an Indeed partnership. But generative AI and code completion was not really something on our radar. Now we have a whole line of business on the API licensing side that is predicated on the value of the data created by the Stack Overflow community and whether or not folks can train on that. If they do train on that, where should the value flow and how do we create sort of a positive feedback loop between the humans who are curating knowledge on Stack Overflow and the AI that's training on it, and then very often providing answers to people as well.
CW I was so excited when I saw that the job board came back. There are job boards out there, but Stack Overflow is just where so many developers live and stuff and so it's cool to see that back and to have a new hub to explore different roles.
RD That was something a lot of people missed, but I think for our side, it wasn't as sustainable a business, so I'm glad we have a good partner for that.
CW Yeah, that worked out too.
BP I think another thing just thinking about the constancy and change of Stack Overflow is that when I started, we had long acknowledged our ongoing mission to figure out how to make the community more welcoming and to make sure that it was a place where anybody could come and ask questions. We sort of know that anecdotally from doing the student ambassador program and others that most everybody uses Stack Overflow, but very few people sign up for an account or go on to ask their own questions or provide answers, which is not unlike all social media, like your Twitter or something, your Facebook. Most people consume, a few people comment, a small group of people create the most content. One thing that I've noticed recently which is interesting– when we make small changes to low hanging fruit like changing the signup flow for new users which has been around since 2008, now we're signing up more users than we ever have. And I don't think that's necessarily a reflection of a change in demand. It's hard to say what role generative AI and its ability to answer coding questions has in the macro picture, but all of a sudden, we're coming up against constraints as a marketing team where it's like, “Well, we literally can't email this number of people. It’s too many people now.” And so there are some things that we've had hanging around where if and when we have the resources and bandwidth to focus and modernize them, it can have a big [impact].
RD I think we should take our bit of credit since we required comments in the blog to have a Stack Overflow account. I'm sure all the folks trying to spam us have signed up for Stack Overflow accounts.
CW Everybody came through the blog. You did it, Ryan.
BP That’s right. This is all haters. We’ve hit a new record.
CW Oh, man. We even had the key, the Stack Overflow little macro pad that we did for April Fool's. That was 2021, I think.
BP That was the most fun I ever had doing marketing anything, for sure. Fun to go viral and bring a meme to life.
RD You’ve got to take a joke too far. That's the way to do it.
CW That's what it's all about, pushing it to its limits.
BP Cassidy, you're the one who's gone through the most career transitions, and now you've had close to a year maybe at a CTO role. Do you want to talk a little bit about that and what you've learned?
CW I'm at like a year and a half, going on two years now, which is wild. It's been different. I really love my team, which I think makes all the difference. When you have a good team, you can get through a lot. It's been really interesting to be in this wave of the generative AI and then also just economy weirdness and everything. We're kind of weathering the storm and figuring out what our next move might be. We don't know. We've gone through pivots, we've gone through team changes, we've gone through investors being very happy to being very angry and back and forth a lot, and that's been a really interesting challenge where there are definitely times where I'm like, “Hmm. Stability would be nice.” But at the same time, again, it's been a really cool team to work with.
RD Do you think some of the non-tech changes have affected the investors’ happiness?
CW We're in a different economy than we were in 2021, where in 2021, you could breathe an idea at a VC and you'd get $2 million. It was very much just like they were saying, “Yeah, go, gold rush. All ideas are great!” And then last year, I think all of a sudden they were just like, “Pump the brakes, calm down, revenue plan.” There was even an episode in Silicon Valley where it was just like, “Revenue, no, no, no, no. If you have revenue, then your revenue will never be enough. You need to be pre-revenue. Then you make a lot more money.” It was suddenly opposite of that last year. And this year it's been a weird mix of both where I think more startups are getting funding, but also more startups are shutting down and more startups are just wrappers around a large language model, and so it'll be interesting to see what happens.
BP We get feedback from listeners and so that's one way for me to sort of take the temperature of what's really happening, but I think the hardest one for me is to know what the reality is for the job market and what the reality of the impact of generative AI is. I think we hear it both ways. Junior engineers are going to be extinct; well, that doesn't actually mean there won't be juniors– they will just start at a different level because AI will be handling the more basic stuff. Or companies are actually reducing headcount or moving from a mentality of always increasing headcount for engineers year-after-year to leveling off or trimming down. And again, it's hard to separate out what part of this is technology and what part of this is economy. Change in interest rates, change in venture capital, and change in expectations for technology companies. Is it about growth? Is it about monthly active users? No, it's about revenue, it's about profitability. Those are very different archetypes for how you build a business.
CW It's been interesting to see that mentality shift because some people are just like, “This is completely different. What do I do?” And then some are just like, “This is probably how it should have been the whole time.” It'll be interesting to see how not only the startup landscape, but just the tech landscape in general changes. I'm not worried, knock on wood and all of that, but there have been periods of, “Oh no, tech is dying,” so many times in my career, and tech isn't dying, it's just changing.
RD The gen AI thing is the hotness right now, but as people bang on it, people are realizing it's not saving the world. It's doing some cool stuff, but it can't do everything. We had a podcast out and the headline was deliberately provocative. Would you fly in a plane that was safety tested by generative AI? And everybody's like, “No, absolutely not. Why would you ask that question?”
CW What was it, two years ago where we felt like all of our episodes were about NFTs and Web3 and stuff? There's waves for everything. Some waves last longer than others. And I worked in AI back in 2015 and there seemed to be a wave then, then it kind of died down and now the wave is back again. There's always the pendulum shifts and waves until the next big thing.
BP I memory holed our ‘too much crypto talk’ time. I forgot all about that, but it did happen. I want to push back a little and say that things happen where we discuss them as if they're progress, and then they're no longer really moving the needle for consumers or business use cases, they're just kind of stepping up in stats. So I would say for mobile for phones and tablets, there was a thinner, lighter era of, “This is significant,” and now there's like, “How many more microns are we going to shave off? Does the consumer really care?” Or a sort of mentality during the crypto period that everything will be rare and collectible to some degree and you just have to be in on the game and understand that others are blind to what's really happening here. With AI, I don't feel like we've passed the point of diminishing returns in a lot of the space for generative AI. I read an article in The Economist, which is a pretty sanguine place that doesn't necessarily get caught up in the hype, about how much generative AI is changing drug discovery. Medical trials that can be arduous over the course of many years now can be done in simulation with protein-folding AI, yada, yada, yada. And so I think things like that is where I'm most excited about the impact– creating and progressing scientific discovery for things that are super meaningful to us: fighting climate change and fighting disease. But then on the opposite side, call it the most frivolous or the most creative, the improvements made to image generation, video generation, song and sound generation continue to just blow me away. They haven't stopped impressing me. It feels like doubling on themselves in terms of their proficiency over the last year or two. I don't know. Are y'all less on the hype train than I am?
CW I both am still on that hype train but I also kind of have one foot off a little bit, because I think that AI is a really good tool for humans to do things and it can do a lot of really, really useful things for humans, but I don't think we're at a point where humans can be fully out of the loop quite yet. You mentioned movies and image generation and stuff. I'll do just an example. There's that movie Civil War that came out relatively recently. Their posters were all made with AI and it was very clear. A bunch of people in Chicago were just kind of like, “Have you guys ever been to this city?” because their poster of some Chicago buildings on fire were, first of all, those buildings are not across from each other on the river. They're literally next door to each other. The geography is wrong. Also that building can't be on fire like that. It's a concrete building. That wouldn't happen. There's just so many things that were wrong with such a little poster where it doesn't make that much of a difference, but an artist who might have taken a photo and then tweaked it might have done better.
RD When you're talking to movie nerds, every little detail counts. They're going to be like, “That doesn't burn like that. It's not shaped like that.”
BP I mean, you want to bring the continuity editors into this and also the commenters for sure. You could get lazy and be like, “Look, we could spend budget on this actress, or we could spend budget on this promotion, or we could spend budget on this marketing,” and they're like, “Look, I'll just whip up an AI Chicago, and you want it to be on fire? I got you. Here, I'll pump it out.” And at that point, somebody should stop and be like, “All right, does this actually look like Chicago? If not, can we fix this in Photoshop?” The AI will give you a lot to work with and then you've got to take a little bit of care. One of the things that I love to return to is that it's almost like we're using these tools wrong. The point of generative AI is to be a dream machine that comes up with new stuff that we haven't seen before that guesses and invents new things, and we're always like, “We need you to be a really precise search engine that returns to me exactly what I'm looking for in an accurate format.” You're trying to fit a square peg in a round hole or whatever.
CW There's all of these headlines right now about search results not being accurate because they were sourcing from Reddit and forums and stuff. One that I saw recently was like, “Yes, you can put glue on pizza.”
RD Right. How many rocks should you eat in a day?
BP It helps keep the cheese on.
CW Right, how many rocks should you eat? So many things like that where we've still got some time. I think there's a lot of potential here. I'm still on the hype train because it has saved me a significant amount of time, especially things like GitHub Copilot, for example. But once again, I think it's great as a tool for humans to use to get from A to B faster, or a little bit better, but we still need some humans.
RD I still like the hype train, but I worry that there will be a future where they're trying to eliminate the humans from it. Just trying to automate and everybody who is a human is just doing mechanical Turk hits just verifying that this is in fact a dog or whatever it is. I worry for that future.
BP The test of ‘Are you human?’ that they use as a captcha are just becoming increasingly ridiculous where it's like, “God, it's so obvious that this is AI training. I really don't want to participate in this exercise and feeding the beast.” It's like, “Click on the box that shows exactly three of this animal.” And this is not because it's hard for a robot, this is you trying to distinguish the stoplight from the red light. This is just for a car.
CW One that I saw recently was, “Which object in this picture would be something you'd buy in a store?” And I was like, “Oh, no.”
BP To your point about how deeply involved are humans in the search, in the past, search would have taken you to this result of, “Yes, you can use glue on your pizza,” you just would have been like, “This is a silly answer that I found on a social media forum. I don't have to take this person seriously.” Whereas if they're like, “Well, this is the top result,” then you're like, “Well, this is authoritative and now I will trust it.” That's really the razor's edge that we're trying to find here at Stack Overflow and just in general I think for the health of the internet and people's information. How do you make sure that people understand this knowledge is still based on something humans created? How do you acknowledge the human in the loop? And then how do you share the value there? If I do a search now and I get a great result that's like, “How do I tie this fly fishing knot?” and I get the result back that’s a complete paragraph and answers my question perfectly, and then I never visit the site that had the answer, that is a change to my internet behavior that has an impact on that content creator and so that's got to be acknowledged and I think we have to build in some way that shares the value, hopefully.
CW I was actually talking to a friend of mine at OpenAI, and he was talking about the challenges of interviewing people for jobs there because so many people use GPT obviously for the job interview. And he's like, “But it's also not a bad thing that they do because we want people to use GPT, but also they're using it for the interview.” He was just kind of going back and forth trying to figure out how bad that was, or if it wasn't at all.
RD You want to interview that person, and if they're using GPT, you're not interviewing that person. You're interviewing their AI assistant. The best AI comeback I saw was when somebody was doing something with AI and check it out. And somebody's like, “Why would I bother to read something that no one bothered to write?” We still want that human connection. In the future, it'll just be cold emails written by GPT that we respond with our GPT autoresponder.
CW It's just full conversations and then eventually you look at the thread and you're like, “I didn't say any of this!”
RD I bought how much fish?
BP One thing that I think about in that context is, if I had to set myself up in the past to be like, “What part of the work I'm doing is me versus something else?” the one that started getting to me initially was the autocomplete in email where I realized about 50% of the emails I send are just open, autocomplete, send. And that was kind of like, “This feels odd. This feels like I'm not as involved. What would I be communicating if this wasn't stepping in to communicate for me?” And I guess the other side of it is fair use where I've been watching some videos, my son is really into making beats, and it's like, “Here's a really famous sample from a 1970s soul song. It would cost you X dollars to sample this song, or this artist doesn't license their music.” So you feed that in and you say, “Can you make me a song that sounds pretty much like this but it's new and the voice is a little more gravely?” And therefore it's not that artist, it's just a take on that artist, and where you draw the line there of, “Well, this artist deserves compensation versus not.” We're going to have to redefine fair use in the courts in a whole new way and see how that plays out. And again, for me, it's about balancing new value for startups and corporations that are building new useful things, tools that creators and consumers feel more powerful and get more done with, and that last bit that we still have to respect the humans in the loop, the original creators, and share the value so it’s equitable so that we don't, to Ryan's point, end up in a situation where there aren't any human left. Humans need to be able to survive in this ecosystem, hopefully.
RD I think they've already defined fair use pretty narrowly. There was a court case back in the late-80s/early-90s about a hip hop song, ‘100 miles and Runnin,’ that had a second and a half of a distorted funkadelic sample and they lost on it. It sort of destroyed the layered sampling of the 80s and 90s. So now you hear hip hop songs and they're like, “It's one hook. It's one song that they've sampled.”
CW I think as long as folks who aren't historically underpaid– I don't want them to get more underpaid. I think that's the danger of all of it because using it as a tool as a software engineer, yay, it’s great. It's a tool that I can use to help me write code. With an artist, if someone rips off their art using AI and then they end up getting paid even less for the work that they're using, it feels worse and different and it's kind of hard to figure out where to draw the line there.
RD The more people monetize their AI products, the worse it feels, because you are taking an artist’s job. Are you a solo developer sort of making this happen on a shoestring budget? That's kind of cool. But are you a big company cutting costs, cutting people? That doesn't feel so great.
CW We'll see what happens for the next five years with AI and all of these tools.
BP Virtual Ben, Cassidy and Ryan, we'll see you in five.
RD All AI podcast.
CW We’ll all be holograms.
[music plays]
BP All right, everybody. Well, as always, thanks so much for listening. We appreciate having you here with us for however many years you've been listening. As always, we want to show a little appreciation for someone in the community. Awarded May 25th to Erwin Brandstetter, a Lifeboat Badge for coming on and giving a great answer to a question that had a score of three or less. Now that question has a score of three more and the answer has a score of 20 or more. Erwin answered, “How do I get a simple PostgreSQL function to return rows?” So this was asked 11 years ago. 45,000 people have gotten in here and wondered, and we've got some great answers for it. As always, you can find me on X @BenPopper. Email us with questions or suggestions for the show, podcast@stackoverflow.com. Ryan and I are doing a chat later today with a listener who wrote in who's got some great ideas about how to sequence your sprints for product development. If you like the show, leave us a rating and a review. It really helps.
RD I'm Ryan Donovan. I edit the blog here at Stack Overflow. You can find it at stackoverflow.blog. And if you want to reach out to me, I am actually on X @RThorDonovan.
CW And I've been Cassidy Williams. You can find me @Cassidoo on most things.
BP All right, everybody. Thanks so much for listening. If you've been a long time listener and you want to let us know how things have changed for you, shoot us a note. We're happy to chat about it or share it with everybody else. Thanks for listening, and we'll talk to you soon.
CW Bye!
[outro music plays]