Kathryn Murphy, SVP of Product and Design at Twilio, chats with Stack Overflow CTO Jody Bailey about walking the line between product design and engineering early in their careers, lessons learned at tech juggernauts like Salesforce and Amazon, and their respective roadmaps for integrating generative AI into their products. Plus, advice for people trying to decide between product design and engineering (spoiler: you can do both).
Twilio is a customer engagement platform whose communication APIs for voice, text, chat, email, and video are used by millions of developers. See what’s happening on their blog, dig into their docs, or check out their Stack Overflow Collective.
This summer, Twilio announced CustomerAI, which applies the power of LLMs to the rich troves of customer data that flows through Twilio’s platform. Learn more here.
ICYMI: From the stage of WeAreDevelopers, Stack Overflow announced a roadmap for integrating GenAI into our public platform and paid offerings. Check out Stack Overflow Labs to see what we’re working on.
Also ICYMI: Listen to our conversation with Jody about his path from physics to sales to programming and what drew him to working at Stack Overflow.
Register for SIGNAL 2023, Twilio’s customer and developer conference, happening virtually and for free on August 23, 2023. Attendees can expect a deep dive into AI and how it’s revolutionizing customer experience technology.
Connect with Kathryn on LinkedIn or the social network formerly known as Twitter.
Connect with Jody on LinkedIn.
Three cheers for Lifeboat badge winner blackgreen, who swooped in to save How can I write a generic function that accepts any numerical type? from the howling void of ignorance.
[intro music plays]
Ben Popper Hello, everybody. Welcome back to the Stack Overflow Podcast, a place to talk all things software and technology. I am your host, Ben Popper, Director of Content here at Stack Overflow, and today I have two very special guests. We are going to be talking about product management, we're going to be talking about what it means to work inside of an organization that has a lot of irons in the fire, and how you work with big teams of developers and designers and all the other folks who get their hands on product to take it from an idea to something that's actually in production and being touched by millions of users. So today from Twilio, we have Kathryn Murphy joining us. She's the SVP of Product and Design at Twilio. And we have our own Jody Bailey, who is the CTO here at Stack Overflow. Hello to both of you.
Jody Bailey Hello!
Kathryn Murphy Hello! It's great to be here.
BP So Kathryn, Jody got the chance to come on the podcast already, we'll put it in the show notes and people can dive into his background. Why don't you give us, for yourself, a quick bio. What would you say are the important things to know about how you ended up in the role you're at today?
KM I started my career as an engineer. My little fun fact there is that actually my father taught me to code when I was 18. I started a part-time job in college coding without realizing the significance of that, as a woman in the engineering field. But then I would describe most of my career as sort of a wandering between product and engineering. And when I found myself at a large company– Salesforce –I was finally forced to choose, “Am I a product person? Am I an engineering person? Am I an infrastructure person?” Because I just kind of had the luxury of being able to do a lot of those things and kind of wander. And my litmus test was, “What am I doing with my free time?” and the truth that I found in myself was, “Oh, I'm researching customer problems and the market and the competitors. I'm not researching better ways to do CI/CD.”
BP Right, you're not coding up a side project on the weekend.
KM Exactly. And I was like, “Dang it. I think I'm a product person.” But anyway, I've embraced that. But I guess I would also say that I've loved technical products and I've always loved products that are engineering-focused or oriented and I've always loved the developers that are part of my customer community. So I am now at Twilio which kind of makes sense with that little background. And at Twilio I lead products, our customer data platform, so multichannel marketing products, and then our customer AI, which is both our predictive and generative AI initiatives for Twilio.
BP Yeah, Twilio is certainly a brand known for having lots of developers in the community and as customers, so you can kind of bring those two worlds back together. That's great.
KM Yes, exactly. I mean, this is my attempt anyway.
BP Right, exactly. You can have your cake and eat it too. Jody, I said I wouldn't give you a big chance to give us the overview, but you're the CTO now so that's a certain kind of role, but for you, what was your introduction to product management and is that still part of your day-to-day here at Stack Overflow?
JB Yeah, so I mean, it's pretty difficult to work in software development and not interact with product management. So my introduction really was through the engineering aspect of things. And like Kathryn, I was drawn to business problems and understanding what customers need and how to provide solutions. And I've had the opportunity to work with some pretty exceptional product leaders which was inspirational, and I've also had the opportunity to work at places like Amazon where everything starts with the customer which I think is a big part of being a product person. What I will say is that in my spare time I tend to be more drawn to some of the technical things. I tend to focus a lot more on the technical product management piece, but also advocating for the product management piece. I'm a big believer in the idea of product design and engineering really leading direction together. So I had the opportunity to manage product management and do that and technical product management. I tend to lean a little more towards the technical product management side of things myself.
BP Yeah, it sounds like you both see there's two sides of the coin here and you can find different ways to express that at different roles. So Kathryn, you mentioned spending weekends thinking about customers and ways to implement solutions for them. When you think about it now, having had a lot of experience in the field, what are some of your guiding practices for product management, some of the golden rules you might tell someone who's coming up in the field or the things you instill in your team that you feel trickle out to the organization and make everybody work better?
KM You'll probably see this as a theme throughout the conversation today, but I have a principle or a methodology that I call “whole product” that I learned from someone that I wished I would've learned way earlier in my career. The concept of whole product is not just about building a feature or getting something out to market, but it's thinking about the whole product pieces of it, meaning how we're going to market it, how we're going to sell it, how we're going to price and package it, how's a customer going to adopt it, how are we going to renew? So it's not enough to just build and ship, which is a really important part. In fact, I have these four levels of product management maturity, and level one is you can ship with accuracy, and that is a big deal. But can you grow beyond that and be whole product thinkers where when we build new things, we actually know how it's going to impact customers and the business and we think about adoption just from the get go.
BP That's very cool. So I guess I heard level four is you've got it soup to nuts. Level one is, “Hey, you shipped your MVP on time.” What are levels two and three? And then I'll pass the hat to Jody for some thoughts.
KM Yeah, so level one– you’re going to ship. Level two– you shift to adoption focus, so you ship but you also start to measure how are customers using this, how is it being adopted, where is there friction? And those are your success metrics versus ship. Level three is actually whole product, but then whole product has a graduating level too. Level four is forecasted outcomes. You've gotten to the point where you actually know kind of end to end how to build things and how to have positive impact and you can forecast outcomes like, “I'm going to predict new ARR for the business, or I'm going to predict an impact on GRR. So that's my level four. And I've worked in some product organizations in my career where we were level four for pieces of it, but most of the time I think I've worked in level one, level two organizations.
BP Not naming any names. No, absolutely not, but to each their own. Jody, do you have levels, do you have a spider web, do you have a different metaphor? How do you think about going from just getting started in this to being a real Jedi of product management?
JB I love the levels that Kathryn's speaking about because certainly shipping is always kind of table stakes. It's like, “Oh, okay. Well we're shipping stuff predictably.” And then the area that I think a lot of us try to focus on are outcomes as opposed to outputs. If you think of delivering something, that's the output, but what is the outcome you're creating? Not just that people are using it, but are they becoming more engaged overall, or maybe they're adopting a feature in exchange for something else and that may or may not create the business outcome you're looking for, which I think is maybe closer to the whole product version that Kathryn's talking about. And like I said, always a big fan of looking at outcomes versus outputs.
KM Yeah. I think one of the other reasons I have these four levels is that I've done the same thing, I've just watched so many product orgs try to go from ship to “Let's do ROI analysis for everything on our roadmap. Let's go try to figure out if this is worth it.” And then you suddenly realize, “I don't know how to do that. I don't have any of the mechanisms in place to track what the adoption is like. Maybe we should have set up separate SKUs and separate selling motions and we didn't do that,” and so it's really hard to just go straight from “I ship on time” to full blown ROI and forecasting capability. So it's not even that I've been so clever to figure out, “Oh, there's these levels.” It's more like I myself attempted to go to ROI and uh-oh, there's other things that happen before we can get there that we need to figure out.
BP That makes sense. You've got to crawl before you can run, and if you build those things into the product ahead of time, or at least are synced up with these other teams, whether that's business or marketing, then maybe some of the tools are in place as opposed to if they're an afterthought you have to tack on which can be pretty difficult. Once the plane is in flight it's harder to add new features.
KM Exactly.
BP So Jody, one thing I wanted you to reflect on which I think is really interesting, is that at the moment there's sort of two very different approaches to product management within Stack Overflow. There's our more traditional approach, and I was chatting with some of the folks who had come to the company recently. They said, “I came from a startup and I was coming to Stack Overflow and it was clear that there were things that were stack ranked in order of priority because customers had told us x, and so we would approach them with these deadlines in mind and sort of tick off the boxes.” But with the advent of gen AI, we've decided that we want to invest a lot of our resources into that, as we've talked about publicly on the blog, and try to move quickly and experiment in public, share things on meta with our users. It’s really interesting to me, it’s almost like a new way of working emerged inside of Stack Overflow as we pursued this goal of releasing OverflowAI. If you haven’t heard about OverflowAI and all the big announcements we made, I’ll put it in the show notes. You can check out the Labs page to sign up to be an alpha tester. But anyway, Jody, I know you were deeply involved in this. Can you tell us a little bit about the way our approach to work, and especially to product management, changed over the last few months?
JB Yeah, that's a great point, Ben. And I think in large part this is driven just by what's going on in the space in the market right now with generative AI and everything else. Everything is moving super fast and we're all trying to figure out what's possible, what's not possible, and then how do we deliver value to our customers and users faster than ever. And for us the way that we have operated historically is fairly traditional. We identify a number of areas that we want to improve in or things that we want to deliver, we prioritize, we do research, do all the experimentation upfront, and then deliver something. And what we've been doing differently this time is we're figuring it out as we go. So we went into the most recent initiative without any preconceived notion of exactly what we were going to deliver. And one of the fun things about the product we build and I'm sure Kathryn experiences some at Twilio, is a lot of our product team and engineers are the customer as well. They're people that use our product and so we have that added benefit, and we just started doing experimentation, so it became very agile, if you will. It started out as a large team of about 40 people, and it's grown across most of the product and engineering team, and we're getting together every week and we're doing demos, we're doing experimentation. We've had to learn how to incorporate customer research in the definition and experimentation process, whereas historically we would do more of that research upfront and then apply it. And now we're having to learn to work together and try things, get the feedback, do the research, all concurrently and then produce results. It's been super fun. Definitely repeated sprints as opposed to a marathon, if you want to use that.
BP But to Kathryn's earlier point about the different levels, it's been really interesting to see folks iterating really quickly on product, as you said, incorporating user and customer feedback, but then also saying some of the fundamental parts that you would need to ship. It's like, “Well, who's going to own this part? Do we have the database side of this set up? Do we have the search side of this set up? Does anybody even own that?” And then people have to raise their hands and sort of figure out how all the pieces connect, which I think has been kind of inspiring to see. Like you said, I think a lot of people have gravitated towards working on this project just because it's moving so fast and there's so much greenfield that it's a really exciting thing to invest some of your energy in.
KM Yeah. Jody, it brings me great comfort to hear you describe this, because I would say it looks very similar at Twilio. And as we went from a lot of experimentation, lots of hackathons, lots of things, to now we have people whose day-in, day-out job is AI, that's when the pressure really was on. People were like, “But there's no research! We don't have PRDs, all the traditional mechanisms.” And so we introduced the concept, the hashtag of #babyship. And it's like you said though, you still do the things but you're doing them in parallel, but you do them smaller and bite sizes. So we're not throwing methodology out the window per se, but we're being open to parallel work streams and we're also being okay with throwing things away that we ship, where that's kind of new. It's funny because many of us have been in startups and we're like, “Oh yeah, we used to work this way,” but it's hard in a big organization because you feel a lot of pressure to deliver results all the time and it's kind of messed with us a little bit and we've had to go back and remind ourselves that innovation is more #babyship, not the big cycles.
JB The other thing, and I'm curious if you've experienced it, Kathryn, is people's roles are evolving and how people interact together is changing and shifting. And one of the things we've had to do is kind of get comfortable with being uncomfortable sometimes, because we're stepping on each other's toes. Not intentionally, but it's learning how to be okay with that and then talk and work our ways through it and kind of reestablish new norms. Is that something you're experiencing as well?
KM For sure. And then add to it other swim lanes or cross-functional things that we would normally bring in at certain times like marketing or privacy and regulation. We are not only uncomfortable in R&D stepping on each other's toes, but we brought a ton more players into that same mix who we are having to teach about how to do #babyship and all be uncomfortable together.
JB And InfoSec and those things is a great example. You can't wait to involve them and you can't give them a heads up. They kind of have to be part of the process and be watching as you go and raise their hand when something comes up.
BP As I mentioned earlier, recently our CEO made a bunch of big announcements about the launch of OverflowAI, exciting new capabilities, want to bring a new kinds of search to Stack Overflow, both the public side and for Teams, an IDE for VS Code powered by Stack Overflow, a bunch of other really cool stuff. I’ll put a link in the show notes to all that and the Labs page where you can sign up to be an alpha tester. But Kathryn, I wanted to offer you the chance to talk a little bit about what Twilio has been doing in the AI area, and specifically how you think about crafting product from the technology perspective and from the customer perspective.
KM I think AI is certainly such a hot topic. Twilio just announced a few weeks ago CustomerAI, and it's a combination of both predictive AI and generative AI. But what I really love about our vision, and we kind of worked pretty hard on this and pushed each other pretty hard, is that it's not just a random set of AI features that we are cranking away on like everybody else, but we really pushed ourselves to say that it's a tenet to the vision that Twilio already had for customer engagement. Specifically, how do we bring all those Twilio channels that developers around the world are familiar with and know and love together with the customer data platform that came from Segment, and then really make it possible for builders, developers, marketers, contact centers, to use AI to do their jobs better, faster. So when you go back to even my own story about how I found myself worrying about the customer problems and researching the customer problems, what I love about our AI strategy is it's all about what do customers need. And so even when we get stressed out because we didn't do the research on something we're building, we say, “Well, what's the customer problem we're trying to solve with this AI capability? Is it a problem we already know about? Are we making it better and faster?” So I'm really excited about a lot of the features we're going to be bringing to market next month at our big SIGNAL Conference.
JB I love that and I think it reinforces something, and that is that we're not here to build new exciting, cool tech, but to solve customer problems. And AI is another tool in the toolbox. When we look at it, we look at it the same way. What's the mission of the company, what are we trying to accomplish, and how does AI allow us to do that better? It sounds like exactly what you're doing.
KM It makes it more exciting. I think also part of our bigger vision is we just want every team to be able to use AI to build. We have a smaller team that's sort of the pioneers, but we want every team, and that's Twilio teams and all the teams that use Twilio services, to just have AI be a natural easy part of how they build. It's almost for me, because I'm old, like when cloud or mobile became just the way you build, and I think that's really how we see it– that its primary purpose is for customer problem solving.
BP Yeah, I really like what you said there. It does feel like AI is a layer that is going to spread and seep through everything. And it's been really interesting to see folks who previously were working on Public Platform or they were working on Teams or they were working on some other product within this show up in Slack and say, “I just figured out this thing with LangChain last night,” or “I found this new open source model on Hugging Face that maybe we should try,” and clearly doing a lot of exploring in their own time to understand how does my role work now if AI is a constant that might touch everything in the software business.
KM We did some fun. We've done several hackathons with our customers– AI hackathons, which has been super cool because the customer is always focused on what problems they need to solve for their business. And so when you bring those developer customers into the mix, it centers you really in a hurry around what things matter to them. And that has been a real big source of inspiration for us around what things we need to deliver to the world.
[music plays]
BP All right, everybody. It is that time of the show. We want to shout out a community member who came on Stack Overflow and helped spread a little knowledge and saved a question from the dustbin of history. A Lifeboat Badge was awarded to blackgreen two days ago, “How can I write a generic function that accepts any numerical type?” Blackgreen, we appreciate your answer, and over the years you've helped nearly 7,000 people, so thanks for spreading a little knowledge on Stack Overflow. I am Ben Popper, I'm the Director of Content here at Stack Overflow. You can always find me on Twitter @BenPopper. Email us with questions or suggestions, podcast@stackoverflow.com. And if you like the show, leave us a rating and a review. It really helps.
KM I'm Kathryn Murphy, the SVP of Product and Design at Twilio. And you can find me on LinkedIn or at Twitter– rather, X– under my name. And just a quick shameless plug if that’s okay, in a few weeks we have our big customer and developer conference, including our famous developer keynote. And you can register for free at signal.twilio.com.
JB I'm Jody Bailey, CTO at Stack Overflow. You can find me on LinkedIn, kind of boring, or stackoverflow.com. Hopefully everybody comes out to Stack Overflow and participates in our community. Thanks.
BP All right, everybody. Thanks for listening, and we will talk to you soon.
[outro music plays]