This week we launched Collectives™ on Stack Overflow. It empowers the organizations connected to certain technologies to more directly support the communities that have grown around various topics, bolstering the quality and health of our content in a way that benefits our users as well. In this episode we chat with our CTO, Teresa Dietrich, and Jascha Drel, a senior product manager, about how and why we built this product.
You can check out all the details about Collectives in our launch post here.
We detailed the user research that allowed our community to help shape this product in a Meta post here.
Teresa is on Twitter here and Jascha is on LinkedIn here.
Jascha Drel This year, we want to have a possibility for users to actually submit long form content to the collective so that it could be displayed there, and would be an additional way for users to contribute to the overall knowledge base of Stack Overflow.
[intro music]
Ben Popper This episode is brought to you by AWS Bug Bust, the world's largest code challenge for Python and Java developers. Fix bugs, reduce technical debt, and win exclusive Bug Bust prizes, including an all expenses paid trip to Reinvent 2021. To learn more, visit aws.amazon.com/bugbust.
BP Hello everybody! Welcome to the Stack Overflow Podcast, a place to talk about all things, software and technology. I am Ben Popper, Director of Content here at Stack Overflow. And today we have two great guests, Teresa Dietrich and Jascha Drel. Good morning, and welcome to both of you.
JD Hey, good morning.
Teresa Dietrich Good morning!
BP So for folks who don't know Teresa is our CTO/CPO and Jascha, I work with you a lot on the advertising product side. What's your official title?
JD I'm a Product Manager in the Reach and Relevance teams. And I currently work on the Collectives initiative that we're going to talk about.
BP Oh, good. Here's a spoiler alert. So yeah, for folks who don't know, we have a new line of business called Reach and Relevance, which kind of combines what we think is some of our unique strengths across talent and advertising. And this week, as you may have seen, we introduced a new product called Collectives on Stack Overflow. So I brought Teresa and Jash on to talk about the how and the why. So let's start big picture, to each of you. What is this new product? What is it? Sort of at a base level? How does it work?
JD Yeah, so I think it's a product that like kind of solves a problem for two sides, right for for organizations that have technology products, and that are already present on Stack Overflow through their tags, and like their conversations going on about their technologies. And for the users that like use Stack Overflow on a daily basis, try to get information about these products, and could use some more in depth information from these organizations that actually built the products.
BP And Teresa, from your perspective, what is this product? And how is it going to function for both clients, community and users?
TD I think it's exciting for me, because when we formed Reach and Relevance, we really wanted to talk about how we can have both awareness and engagement. And I think that awareness is you know, where ads is been there. But this is really about allowing engagement. And I think that as a technologist, you can find lots of answers, and hopefully on Stack Overflow, they're better answers than elsewhere. But I think allowing the builders and the authorities on these technologies to be, weigh in on this, to recommend dancers to answer questions themselves, really gives people a better sense of confidence about whether they're can use this answer, can use this solution successfully. And I think right now, we're all about trying to remove blockers, and help technologists build better faster and innovate. And if you can have a higher degree of confidence that the first solution you try, is the right one, that's only going to help our users.
BP So let's give an example. We have two great clients who signed on to partner with us to launch this Google Cloud and Golang, you know, the open source sort of organization behind Go, how are they going to utilize this? I guess an example would be right, like you have a question about setting something up your new customer using Google Cloud. And you might do a search for a certain tags and keywords on Stack Overflow, you might find three or four questions which are all asking the same question. This happens a lot. And one of them is higher rank than the other, but another one has an accepted answer. So how would Collectives sort of help people to wade through that and figure out know how to find the best solution for their particular problem?
JD So what we're introducing with with Collectives is like the recommendation feature. So organizations such as Go, or Google Cloud, could go into these questions and like, figure out that there are multiple answers. And all of these answers could be good, right? And it could all solve the problem. However, these organizations could go in and say like, okay, from our perspective, this is the best way of solving the problem and then recommending it, which also is like almost an acknowledgement towards the user that put a lot of effort into writing this answer, which is complete and right and all of that stuff. And get this like formal acknowledgement from these organizations like, this is good. This is what we think is the best way of solving the problem.
TD And I think the other way is that through a Collective, a user can be a recognized member of the Collective by the organization. So that could mean they're an employee. Or it also could mean is there an expert within Stack Overflow that's been recognized by that organization. And so their answers on questions will also be highlighted. So in addition to the answer actually being right, you can see answers from folks who are now recognized experts in this area to help increase your confidence on that solution.
BP Yeah, that's very cool. Because obviously, there are lots of people who contribute to Stack Overflow. And over the years, we've built up a lot of reputation and won a lot of badges. And we often can sort of measure that impact. Okay, this person answered 50 questions in his reach, you know, 10 million people through those answers. Now they get sort of an official recognition of sorts through, you know, a company and open source organization. And it kind of fits very well, I feel like with the open source model of like, how do we, you know, recognize contributors? How do we make you know, someone who's a contributor into a maintainer? And you know, how do we sort of keep people involved in our ecosystem in these flexible ways, so that, you know, everybody can benefit.
TD From the beginning, we were really focused on this not just being commercial products, but actually being balanced with open source products. So throughout our user research, we involve folks from both of those. And for launch, we really wanted to make sure we had both a commercial product, right, like the Google Cloud Platform and an open source product that we were launching with with Goland. Because I think we, we say there's value in both of these to our users, because users don't just use one ecosystem, or one platform these days, they're mixing lots of different technologies that come in lots of different, you know, packages and support models.
BP Yeah, that brings up a good point, you were talking about sort of the research and user testing that went on behind the scenes, you know, whenever we release something new, I think it's important that we communicate yeah, to our community, our power meter users, you know, what went into it? So can you tell me a little bit about sort of like, you know, what that process looked like and what some of the things we discovered were that guided how we implemented Collectives?
JD Obviously so I think it's, it's good to highlight that like, over the course of months, we've done like numerous of these sessions with with both the organization's has as the users. And so we've learned a lot there. And I don't think the product would be any similar to what it is now, if we wouldn't have had done that. So like our user research lead, Mithila Fox, she actually posted a very interesting post on meta Stack Overflow, which I would definitely recommend people to read, because she kind of highlights what we've learned there. And for me, particularly, there was one thing that really stands out. And that was, like, naming is hard. It's probably one of the hardest things that we had to do in this. And when we were thinking about like a name and like a symbol for this concept of Recognize Member, one of the things that came up was Verified User, right, it makes sense. It's something that is common in the internet as a whole. And I was lucky enough to join several of those sessions. And all of them instantly, when they saw Verified User with the with the famous tick, they immediately said, don't do this. This is bad. StackOverflow is not a social media platform, don't do this. And at first, I was like, wow, like, is this really so bad? But it was like, time and time again, don't do this. And thinking back at it. It makes sense, right? Stack Overflow is not a social media platform. So we shouldn't like make any reference to that. So I found that really, really interesting. And are several more of those insights we found. So right very interesting process.
BP Yeah, there's, you know, some emotional and psychological and maybe sort of just as dialectical baggage that comes with the idea of a verified user, and who should be or shouldn't be and how it can get taken away. So that's interesting. Teresa, are we allowed now to share some of the names that didn't make it? Do we have any other finalists for Collectives?
TD Oh, gosh, the one that I wanted to use, but didn't work out was Guilds. Because we thought of these as sort of, in a way almost sub communities that within the larger Stack Overflow community, but we didn't want to use communities because it's already tangled enough. And so I really like Guild, from the sort of Spotify model, right? And everything. And so there was there was mixed feedback. Naming is hard period, right? Naming a user, or names of products and everything like that. So we had some outside help on this. And I think Collectives was sort of in that same vein, but it came across really well. And I think it does that. Adding to the user research, I think, he actually says months, it's almost a year's worth of user research that we've got here. Right. And we talked to mods, we talked to high rep users. And you know, we were really focused on mods, specifically, because we didn't want our moderators to feel like we were either taking something away from them, or imposing more work on their plate. And so I'm so appreciative of the specifically the high rep users and the moderators who gave a lot of their time to spend with us to make sure that we got it. Right. And I think we're pretty happy and feel pretty good about where we ended up with this, of that we're adding value for all of our users without putting any more work on anyone else's plate.
BP Yeah, guilds would have been great if we could have had our sort of like a world World of Warcraft, Dungeons and Dragons vibe.
TD Maybe I'll say no for another product sometime Ben. [Teresa laughs]
BP So I guess yeah, you know, one thing you brought up there that I thought was really interesting was, you know, this idea of like, how can we help moderators without giving them more work? And maybe this brings us to the idea of sort of like content, you know, health and community health. So I guess one thing we should probably just put out there and cover is like, does this change the relationship of the users or moderators to the content on Stack Overflow? Like, is Question and Answer going to work in the same way and if not, what's going to change?
TD The q&a content does not belong to the collective. Okay? On this collective sort of homepage landing page, if you will, what we're really doing is just displaying the content, the q&a content that comes from the larger community, and allows them to sort of have, like we said, recommendations or any of the recognize users that the collective has identified, highlight their answers. But you know, it's still under our same license, that q&a content, it's still exists. It's still moderated. It's, you know, and I'll add it review queues and everything has the same. But it's really more about organization, discovery, and then the ability to sort of highlight or recommend specific content that comes from the community.
BP And Jascha, I know, we're introducing a new sort of content type, which is not new entirely. we piloted it in our stack overflow for teams, which is used by organizations to do an internal version of Stack Overflow. Can you just quickly explain to people what article is and how it'll function on a Collective?
JD Yeah, obviously. So I think like articles is like our way of putting long form content on on Stack Overflow. And I think that for years already, even before I joined Stack Overflow, we heard from both users and clients that there is like an appetite to go beyond like traditional q&a format, it doesn't mean that q&a will be replaced or anything, but just going beyond that through throughout the content types. And we've had the documentation project also years ago, and that didn't work out. But we actually looked into that project and tried to find the learnings now that we've gotten there. And we figured that even though it failed back, then there is still this appetite for this type of format. And by making some tweaks to the to the concept, we think that like, within a Collective space, it actually would be beneficial for everybody. So we're, we launched this on Wednesday, for the possibility for the admins and recognize members to post it to make sure that we kind of keep the quality standard high to make sure that the quality standard of Stack Overflow is is maintained. But we don't want to keep it gated, right. So on the on the longer term this year, we want to have a possibility for users to actually submit long form content to the collective so that it could be displayed, there would be an additional way for users to contribute to the overall knowledge base of Stack Overflow.
BP I heard sort of an interesting example of this recently from a client who I'm working with on a case study, and we wanted to sort of put it out there as an example. They were talking about, you know, when they hire somebody, and they have to set up their development environment at home, they're working remotely, they're getting to know all the systems, they have a bunch of Stack Overflow questions, they use the article to kind of organize them into an FAQ. And you know, like, once you get through this question, you're likely, you know, your next step is likely to be you're running into this problem. And then this problem that kind of go in sequence and stuff. So to me, that was really interesting, because we've always had tags on Stack Overflow. But this is a new way of being able, with more precision to organize some of that content and to almost guide people, you know, through a series of events that they're likely to experience instead of having to like, you know, bounce from one question to the next or search again and again, or kind of go through the tag page. So I thought that was a cool application of it. And hopefully, something will see on Collectives.
JD Yeah, sure. I just feel that there's there's more than than just q&a, right for for technical content. And I think articles can just open it up for everybody like How To guides, right, going into depth on how to do something just doesn't lend itself for a q&a format. And with articles, this will be possible.
BP Yeah, sure, you and I were talking before, and we're just saying, like, as an example of this kind of content, you were saying maybe Go's interested in helping beginners. And so there's like a beginner guide to Go and intermediate guide and advanced guide. And those, you know, include certain questions that they've highlighted, which are applicable to like your stage?
JD Exactly, I think Go, like in the process of talking to the people at Go, it was it was really interesting, their desire to like, make sure that the Go users, like the Go new users have have a good experience and like starting up and making sure because it's, because it's difficult to get on boarded, right, it's not an easy thing to learn language from scratch. And Stack Overflow is an important place where people learn, so they want to really improve that experience. And they also are very eager to work together with our, with our community to figure out what the best way of of doing it because we also have a high bar in terms of quantity on Stack Overflow, which is good, because that means that not everybody enjoys coming to stack overflow to solve their software problem. So there is an interesting balance there, which I think if we bring the three parties together ourselves, Go and attend our community, we can figure out ways to make sure that that everybody gets what they need when they get to Stack Overflow.
BP So Teresa, last question, I guess, you know, I'm thinking about this big picture. We you know, recently announced that Prosus, a big investment firm intends to acquire us and you know, the thesis that was laid out in that announcement was a lot about the growth that they expect to see in the edtech space, and then people going online to learn and advance themselves and skills for a certain technology. This kind of fits into that. As Jash was saying, we're trying to put these three parties together, you know, Stack Overflow, though the company, the client, you know, or organization, and then the community to improve content health. So can you just talk to us a little about like, yeah, what the big picture vision is, for kind of the community and content health. I know, you wrote about this recently in your state of the Stack, but we'd like to share with people who didn't get that kind of, what are we thinking, you know, on a long term roadmap about some of these big trends and how we can make an impact there?
TD I think Jascha sort of touched on it, right, which is, I like to think of our q&a on our public site as sort of that just in time learning, it's very bite size, and you go on and read it when you've hit some kind of barrier, right. And so you're going in, and you're trying to get unstuck. And I think what we're trying to do here with the articles, and, you know, starting in Collectives, because we've seen really great use cases on Teams, as you mentioned, right, for a longer form, you know, bringing more content together, that's going to get a little bit, we still don't want it to be giant, full documentation. Right? We want it to be sort of bigger than a bite, you know, maybe it's a tapa. Sort of plate, small plate. But I think, you know, we're thinking about, especially with this process acquisition, right, and a lot of the folks there in their learning space, right, so Codecademy, Udemy, Skillsoft, that's even longer form learning. And so how can we help our users get the right amount, right, the right scope and the right scale of the learning they need? And bring it to them in a way that is, you know, curated, like Stack Overflow has been curated over the years, right, through up votes and down votes. And, you know, really bring that together in a way that's really meaningful and useful, and not just throwing a million things at you. I think articles is that first step, I know, we've tried longer form content on the site before we this is a new approach. And you know, we're being very careful with it like to begin with just the collective recognize members can contribute it, but we're very quickly thinking about how can users submit something? And you know, it's something we've heard from some of our more, you know, long time, you know, most experienced users is there's not a lot of the questions don't have as much value, but they'd really like to share bigger picture, that larger approach and everything to that. And so, I think we have to balance this being, you know, not wanting to become blogs, but how can we continue to provide different scales and scopes of learning for our users, so they don't just come to stack overflow when they're stuck, but also when they're discovering and exploring and advancing, you know, their skills and knowledge.
BP So you're saying, I can't just take the articles and put them on the blog? Because I was gonna, that was gonna make my life a lot easier if I could just carry them over? [Teresa laughs]
TD Sorry Ben.
BP Okay. But yeah, no, I hear what you're saying. And actually, it's been something we've experimented with a little bit even before the Prosus news, we just happened to start working with Codecademy, and they had these live classes that were happening every week. And then yeah, whole curriculum behind it. And what we were able to do was just sort of get them in front of, you know, one of the biggest audiences of developers visiting each day. And like you said, those people had come they landed on a question because they're stuck on something and then they happen. Notice on the right hand rail, oh, creative coding with Python, I might want to you know, check that out. So we've seen some interesting stuff happened there already. Very cool. Alright. Great. Well, I want to say thanks to both of you for coming on. And yeah, for folks who are interested, there's tons of information on the blog and on meta this week about Collectives, you can check out the ones that we spun up for Golang and for Google Cloud, and expect a lot more in this space to come.
TD Awesome. Thanks, Ben, for having us.
[music]
BP Alright, everybody, it's that time of the episode, I'm going to shout out the winner of a lifeboat badge. Awarded yesterday to Sheepy "How to convert an integer to bigdecimal in Java" So if you're stuck on that, we'll have the answer in the show notes. I am Ben Popper, Director of Content here at Stack Overflow. You can always find me on Twitter @BenPopper, you can email us podcast@stackoverflow.com and if you enjoy the show, please do leave a rating and review, really helps. Jascha, who are you and where can you be found on the internet?
JD I'm Jascha, I'm a Product Manager here at Stack Overflow and I can be found on LinkedIn.
TD I'm Teresa and I'm the Chief Product and Technology officer here and I can be found @teresadg on any of the platforms that you might find, Twitter, LinkedIn or whatnot.
BP Okay, verified users all around. Alright everybody, thanks. Talk soon.
[outro music]