Once upon a time, almost all Stack employees were also users of the site, community members who interacted regularly with what they were building. As the company grew, this naturally became less true. Our first annual Community-athon was a chance for every employee to get a taste of what it's like to ask and answer questions on the site. And for veteran employees, it was a chance to relive the experience of being a brand new user on Stack. This episode, we dive into what happened and what we learned.
Our guests this week were two of our employees: Yaakov Ellis and Stephanie Cantor. Yaakov is a Principal Web Developer, Community Advocate on the Public Platform team at Stack Overflow, and Former Team Lead for Internal Development at Stack. Stephanie is the Program Manager for Community Strategy at Stack.
Want to learn more about how the Community-athon worked? Read up on it here. And yes, of course there was a leaderboard and internet points.
Yaakov was undercover as a brand new user, but some of his answers gave him away. Can you spot the tell?
Our very own CEO spent a lot of time asking extremely important and nerdy question on our SciFi Stack Exchange.
We bumped our engagement from employees by more than 100%. Many questions were asked, much knowledge was spread.
Yaakov Ellis So the goal is not just that they will be connect to the product during the event, but also that will--it will have a kind of lasting effect in keeping people more attached to the product in the community and trying it out, even after the events over.
[INTRO MUSIC]
Sara Chipps Hi, everyone, and welcome to the Stack Overflow podcast. I'm Sara Chipps, and I'm here with my two cohosts, Ben and Paul.
Paul Ford Hi Sara Chipps. Hey Ben.
Ben Popper Hiii Saraa.
PF Hey, it looks like we're gonna talk Stack today, which I always like because I'm an outsider and I get to learn about this weird community of Stack.
BP Yes.
SC All the crazy things that happen internally.
BP Not just one weird community. Hundreds of weird communities. [Paul chuckles]
SC Yeaahhh! So great. We have two guests today. Two folks I work with a lot. I'm excited to have them on. Yaakov Ellis, who is both a software engineer on the community team and also works as a Community advocate and Stephanie Cantor, who is a program manager on the community team. They're here to talk Community-a-thon. Welcome Yaakov and Stephanie.
Stephanie Cantor Hiiii!
PF Hey Yaakov! Hey Stephanie! Okay wait, can someone tell me what a Community-a-thon is in this context?
SCantor So a Community-a-thon at Stack is actually a contest that we held for four weeks, where employees were able to actually compete and start engaging with our network sites.
PF How come? How come? Why are you doing this?
SCantor So we did it for a couple reasons. First, we had like, a couple goals were to one increase empathy. So a lot of our staff employees who actually are on the site have had accounts for years, and who don't actually remember what it feels like to be a new user on the site. So that was one big reason. And another is to actually just get our employees to actually engage with the side because that's important, like Stack Overflow and all our Stack Exchange sites, it's crucial for our employees actually know what they're selling, where they're working and our amazing community.
02:12
YE Yeah, if you look back in the history of the company, if you go back, you know, 10 years, then almost everybody in the company was devoted to the public q&a product. And everybody was on the site. And everybody was familiar with the community. They helped build the community. They were early users. As we've expanded, the different products we sell, even though they're all founded on Stack Overflow in the second change network, you know, the ads, talent, teams is different forms for basic and business enterprise, they'll based on that same origin. But the company has grown so large that you know, we found that a lot of the the staff employees just don't have experience on the network, might be intimidated to interact with the community. I did a little bit of some querying before we started this event and found that even though around three quarters of employees had visited a site on the network, in the previous 30 days, less than 25% did anything on the sites, you know, even upvoting, or posting of any sort. So we have the groups of developers and people who are involved with the community who are spending tons of time on the site. And then you could have other groups of employees who might not know anything about the community of the sites and hardly spend any time there at all. So that's also something we're trying to turn around and and and really re-introduce the site, and the community and the different sites in the network to different employees and groups of employees as well.
BP Yeah, I think one thing that Yaakov said is really interesting, which is that, you know, the different products sort of grew out as extensions of what's happening on the network. And advertising obviously, is the most basic version, talent makes a lot of sense if you have developers, you want to show them job opportunities and companies want to reach them. But in some ways, teams is like the most interesting because it is basically giving a company their own internal version of Stack Overflow to utilize. And so knowing how Stack Overflow works is kind of paramount to being able to sell that product, develop that product market that product. So in that context, I think getting people engaged with the communities is key. And like Yaakov said, at the beginning, it was developers who were joining a site that was small, you know, they were with other developers. Now, many of these sites are quite large. They're, you know, very advanced in terms of how many years they've been around over a decade. So it can be intimidating to break in there. So part of it was helping people to, you know, break in and ask that first question or provide that first answer.
04:36
SC So how long did the Community-a-thon run?
SCantor It ran for four weeks and it ended in the second week of July.
SC Amazing. How did--I from the inside, I really appreciated seeing people in different disciplines participating yesterday, got to talk to our sales team and shout out two people on the sales team that made the top 10. What did y'all see that was exciting to you?
SCantor I just loved we had actually an internal Slack room where people were able to ask questions or the leaderboard was posted a couple times a week there to just kind of increase competition and to kind of give people shout outs and things like that. And it was even awesome to just see our CEO Prashanth, actually on the sci-fi site as well, making everyone look bad with his amazing questions.
SC You all set up a mentorship program, too. Can you talk a little bit about that? Were y'all mentors?
YE Yeah, so one of the things that we wanted to help with with new users is to help them get onto the network and to overcome any hesitations they might have. So that is something we have a lot of in house expertise to deal with. We have a lot of experienced users from the network. That's on the one hand, on the other hand, we did want to present users with an organic new user experience. We didn't want to give them complete hand holding and make things too easy for them. We wanted them to be exposed to things that other new users might have. You know, another big goal of this whole event was to try to get a lot of in house feedback over all the different experiences that users would have from new user onboarding to every different aspect of the experience. And we didn't want to taint that and to make it too easy for people. So it was a little bit of a challenge for that we, before the event, we had a survey that went out for all participants to sign up in which people rated their relative experience in the network. And we offered people the option of either for the less experienced users to be paired up with somebody who had mentored them, or for the more experienced users to be paired up with somebody less experienced that they could help to guide them and answer questions. And, you know, the general guidance that we gave was to, you know, help them make sure that they could open an account and answer, you know, give them guidance on the best way to write a question, or writing the answer and how to find a good site and to be there if the user had any questions. There was no co-authoring of material or anything like that. And then the new users were encouraged to go out on their own. So in the end, we had close to 40 different pairings of less experienced and more experienced users. Almost everybody who was paired up, ended up really finding some benefit from that. And it was also very nice that all the pairings were made between two different employees that were coming from different teams in the company. So it was really a good opportunity to help people to meet new faces and to you know, kind of expand their horizons within the company that way as well.
07:25
BP I kind of got to cheat at this competition because as the director of content marketing and sometimes my responsibility to post stuff to meta, and so I get lots of free free points and free interactions even though I'm not digging into a particular community. I have contributed to the English language one, put my English major to good use. We don't have a dance Stack Exchange yet, but when we start that I'll be sure to put my other degree to work.
PF So help me understand--let me ask a basic question, which is--so this is a great event for a company of a certain size right because it people need to use their own product to understand them, and it's amazing how often things go out of phase because people may not be using their stuff every day. So how do you capture the output and turn it into action on the other side of this, right? Like you're doing this Community-a-thon, you've had a month, you've gotten a lot of feedback, where did the feedback go? And how does it become input into the growth and development of Stack overall?
YE Okay, so that we wanted to make it very easy for participants to give feedback. And then we would end up doing the work to aggregate all of it. So we accept the feedback through the Slack channel through you know, we even said if you want things to be anonymous, send it to one of the organizers, anonymously. We had on our internal team Stack Overflow uses Stack Overflow for teams for internal use. So we had a question there where people could add an answer, giving specific points of feedback. And that received a lot of very, very useful points of feedback that were given and we also sent out optional surveys that people could submit as well. So all of those came and they were aggregated by members of our the pm team that deal with a public platform. So now we have a nice long list of different feedback items really touching on a lot of the different areas that we in general are hearing about, through our site satisfaction surveys and on the meta sites, although there is something interesting about a an event like this, that normally our biggest feedback channels come from either the meta sites, which attract a crowd of very experienced users. And the other feedback channels are our site satisfaction surveys, which are ongoing, those are opt in on the on the website. But the group that we have the most trouble getting feedback from, in general is a new user who goes for the first time to the site and gets frustrated and leaves. They might even be selected for to be offered the survey and they're probably not going to click in and see what frustrates them. So you know an event like this where we have some brand new users coming in who are trying things for the first time, or experienced users who many of our experienced users, myself included, created sock puppet accounts, created anonymous accounts to relive the experience of going on the site for the first time. And you know, I'm you, I'm used to when I'm on the site. Normally, I am automatically an admin on every single site, a moderator and every site because I'm a developer. So you know, I created a new account as well. And to get the feedback from users relating to that new user experience is something that is pretty valuable for us because we don't we often get that as well.
10:35
PF What's it like to be a new user these days? Yeah.
BP Did that work? Hello, fellow teens. I'm here.
YE Yeah, I was I was able to go undercover. There were a couple of users who actually some experienced users on the site who guessed who I was because of one of my answers, and maybe a few more when they hear this thing I guess you'd be able to figure out as well. But but for the most part, it was it was very interesting to go in, you know, have a different identity and to go in and just feel what it's like to be a new user and having to earn privileges that you take for granted.
SC I don't know that everyone listening to this podcast knows what we mean when we say, the sci-fi site and other sites like that. But what we're talking about is the Stack Exchange network, which is huge, 170 sites, 172 sites. And so with new users getting started in the company, we got to see a lot of people's interests that we didn't know before.
PF Who has favorites? I am increasingly in love with the workplace, which might be the most fascinating. I'm in love with Workplace.
BP It's like an episode of The Office. It just never fails to generate ridiculously inappropriate questions, but nothing but hilarious answers here.
PF Here it is. This is my favorite. This is a quote from their manager: ''I'm not telling you to work nights and weekends, but I am expecting that to happen.'' [Ben & Sara laugh] What does this mean? What does this mean? And it's just like, ''Oh, noooo I can tell you what that means.''
11:57
BP Love how brutally honest Workplace is.
PF What are the other great Exchange sites?
SC I love, I loooove, I was gonna say sci-fi but no world building is so good.
PF Yeah, God, that is a classic.
SC That one's always my favorite. World building is all about if people are creating worlds because they're writing a book or they're like writing a screenplay about fantasy or sci-fi, asking like important physics questions, like, ''could a bunch of elephants last on Mars as long as they had oxygen tanks? Would they freeze to death?'' Like stuff like that. And where people who are astrophysicists come on and answer these really cool questions.
PF It's like the anti Twitter 'cause it's just things you don't have to care about and you can't feel bad about like, ah, I don't have to process that I don't have to, you know, no one ever asked for money on world building.
SC No, it's true.
PF So far.
YE Just talking about sites I can tell you the the top other than Stack Overflow I can give you the top five sites during the Community-a-thon event. So the top one was Parenting.
SCantor That was mine.
PF Ohhhhh.
SC Stephanie! Yeah, your a big parenting.
PF Yeah, especially during a pandemic, yeah.
SCantor It wasn't as interesting as a workplace, it was more just like ''my annoying child won't be potty trained, like, what can I do?'' But it was amazing.
PF Yeah, I didn't even know there was a parenting one. See, that's 172 of them. Okay, I'm joining right now.
BP Steph, the answer is jelly beans. It's jelly beans. That's how you get--
SCantor I use chocolate covered cashews.
BP Yeah, you sit on the potty and if you do your business, you got the Jelly Bean.
YE Stephanie finished third overall in the entire company in the Community-a-thon.
SC So great!
YE And and Stephanie, you are a new user. You hadn't you basically hadn't posted so much before.
SCantor I was just a lurker. Yeah.
PF Stephanie, how much work did you have to do to get that ranking?
SCantor So I would do at least one activity a day. So I would either answer a question, ask a question or just like vote a couple times on a couple answers. That's all just like at least one activity a day.
13:55
BP Yaakov I have a question which is always interested me. I mean, one of the reasons I took this job was actually because of the breadth of the subjects on the network. And so I felt like, I would never get bored kind of like being a journalist, like I could always find something different to talk about if I was allowed to dip into Stack Exchange when it came to building the blog, or the social media or anything like that. But what do you think about the fact that the rules are so wildly different, like on Stack Overflow, it's all about objectivity. It's all about showing your code, even in meta, everything needs to be, you know, sort of like in the form of an objective question. Whereas on world building it's, like, ''help me use my imagination'', you know, like, it's completely subjective. Does each little Stack Exchange like sort of build its own set of rules and norms?
YE Yeah, each Stack Exchange site definitely creates its own identity. And they make they define what is on topic and off topic for their own site and you definitely have some subjects where objectivity is required. And subjectivity is, you know, hopefully, politely will but will be discouraged because they're looking for creating the best reliable repository of knowledge on that subject. When you come to subjects like parenting, or the workplace, which was mentioned, then it's basically everything is subjective. There are some objective questions, but there there are few and far between. And every site has an introductory page where they define kind of the philosophy of the site, and what's in bounds and what's out of bounds. You don't even have rules about if you know, I know on the workplace or for the interpersonal site. Also, they say you are allowed to post subjective opinions, but you need to back it up. If you're just giving something completely unsubstantiated, it's frowned upon. But if you're drawing on a personal experience of yours to give some advice to somebody else, and you can back up your reasoning, then that is something that can be helpful for others. So it is definitely something that is a it makes each site unique. Even on Stack Overflow, it's something that's there's always been this kind of tension between having between creating the best knowledge repository as possible, which means never having any duplicates and being very strict about closing questions if they're off topic, because that will end up diluting the quality of the site versus allowing new users to come in and feel more comfortable asking, you know, it's been a challenge and it continues to be a challenge to find to strike the right balance there. Maintaining the quality and abiding by the standards have been set by the community and by the experienced users, while still creating a place that newer users who might not be as familiar with norms of the site can come and ask questions as well.
16:30
SC Wow, I just stumbled on a question that has kept me lying awake at night quite often, which is on Star Trek. Why is the con held by low ranking officers often I always wondered, why is Wesley Crusher, who's like nine years old steering the ship? Really great question. Turns out you know, the answers aren't as definite, but the answers are more like this isn't that weird. Don't freak out.
PF Just calm down.
BP It's Ken. It's all we can say.
SC Yeah. So one thing I heard you both say is something about a leaderboard, what was the leaderboard?
YE So we wanted to create some way of not just having an event where we tell people to go on the site, but to create some sort of a friendly competition as well. So the leaderboard itself came from a system of points that that we invented for this event. The points were intended to motivate consistent participation. So we don't want people just going in one day spending the whole day answering things and getting lots of points, but rather to go in and spend a few minutes every day over the over the course of the event. We wanted to encourage people to visit and participate on different sites and to visit new sites they hadn't seen before. We want to encourage creation of good content. And we also wanted to reward other types of participation like giving feedback and participating in chat. So there is a a point system that we put into place, it will be going up on the meta Stack Exchange site, and the posts to recap the events, so that should be going up sometime around when this podcast is published, and it cataloged through a series of SQL queries that I wrote that are querying some network aggregation tables that we have that that aggregate activity across the entire network is able to generate the points for all users, for all participants in in the event. And then through lots of manual copy paste steps in Google Sheets, kind of share those out a couple times a week, we had a few different lists of leaderboards, we had leader of leaderboard for all users for the newer users were less experienced for the more seasoned users as well. And also the for the teams were ranked as well. So if you had a teammate between less experienced and more experienced users, I think we counted it as 150% of the points from the less experienced user and 50% from the more experienced user, so that added a competitive aspect as well. And it allowed us to post in some of the different team rooms and you know, shout out to this person who's he was leading this team in the competition, the senior leadership team in the company, most of the most of the senior leadership team was participating in the event and they got a little bit competitive there as well. And then the the leaders in the in the very end the the top participants in each category ended up getting a little bit of special custom swag, as well.
19:26
PF I want to hear the other top five besides Parenting.
YE The other to the other top five, other than Stack Overflow was the most popular, but after Parenting came Seasoned Advice, which is our cooking site. Gardening and Landscaping. The Workplace.
BP Yes!
YE And Arqade with a Q.
PF Ahhh. Wait, is Arqade about gaming?
YE For games.
BP Yeah. All video games.
YE All video games and computer games and everything related to that, yeah.
BP I don't know if this was always true or if this has changed in recent months but of the top 10 Stack Exchanges, period, by traffic, which is a huge volume of internet traffic, all our technical and related to software or mathematics except for number 10, which is Home Improvement, which I have recently been availing myself up. So that's our own Stack Exchange in the top 10 that's not mathematics or code. I do like that one.
SC Is that true? Are you sure it's not English?
YE English, English language is number four in the list. If you go to StackExchange.com/sites, then you can see the top 10 by traffic anyway. English language and usage is number four overall.
BP So one other thing that I wanted to bring up, which I thought was interesting is that yeah, because Teams is now one of our central lines of business. And because that involves how people work and is also sort of built off the Stack Overflow, Stack Exchange model. We've now started dogfooding stuff internally and then releasing it like Articles. So that was really interesting because we internally moved a lot of stuff over from policy documents and wikis and FAQs to this Article about model in our internal teams. And then that was sort of how we figured out what to do before releasing the product. So the Community-a-thon was a great way for us to get to know what's going on. And also, now that we have Teams, we can kind of do Stack stuff internally as we build out product, which is neat.
SC Yeah, one really interesting thing that I've learned recently, is that one thing that happens on Stack Overflow and Stack Exchange proper that we don't really see in Stack Overflow for Teams is downvoting. People don't do a lot of downvoting and Stack Overflow for Teams because people don't want to downvote their co workers.
PF Yeah. Yeah, you don't, it's important.
SC Yeah! They're just amped for them. They're just upvotin'.
PF Well, also, you're you're never sure who knows.
SC Yeah, that's true. They could they could be an admin. I don't think that people can find out.
PF No, I doubt they can but you know, nonetheless, it's sort of like, using your work email versus your home email. It's it's that moment when all the Slack DMS get brought into the light and we all have that moment of reckoning.
BP Ouuufff.
21:58
PF I'm not ready. No one's ready. No one's ready for that moment.
SC Stephanie and Yaakov, do you think that this will happen again? Community-a-thon?
SCantorYeah. So I think based off of the success that we had this time around, and just how excited everyone was, we do want to continue doing it annually. So I'm really, really excited. Hopefully this time next year, we can plan for that.
PF Well, it's working out to like minutes a day, right? It's the you know, it's a low impact form of dogfooding. I mean, that's now 1/12th of the year, people are more connected to the product.
YE I lead with, with some stats about user participation, looking at employee visits and engagement on any site in the network over a 30 day period before the event. So it's been 30 days since the event is over as ended. So I just reread those stats. So the percentage of employees who visited any main site within the last 30 days went from 73% to 87%. That's a 20% increase. The percentage of employees who engaged on any site went from about 12% to about 25%, which is a 50% increase. And the percentage of employees who created content, who commented posted, reviewed or edited on any site in the last 30 days, went from 12% to almost 20%, which is 64% increase that's just changes in employee interactions on the network since comparing before the event to after. So, you know, I'm definitely hoping to see this as a trend that continues and looking to find more ways to, to encourage all different staff members from across the company, regardless of team to understand our sights and communities and to interact with our communities and feel more empathy for them as well moving forward.
SCantor And also to just continue giving feedback as well. So we shared the same feedback channels with the whole company so they can continue, like submitting bug report reports and things like that as well. So keeping that loop open.
[MUSIC]
24:00
BP Alright y'all this is the time of the episode where usually I shout out a lifeboat badge, which goes to a user that took a question with a negative score gave an answer and got it up to a score of 20 or more. But because today is Community-a-thon day, I think we're just going to read some of the amazing questions that are at StackExchange.com, they show you the hot questions from around the network. So we already did number one, which is Office Space. ''I'm not telling you to work nights and weekends, okay, but I am expecting that to happen. What does this mean when your boss says that?'' So I think we all know, another hot one from Workplace, ''My management block my internal transfer, but now I have an outside offer, should I tell them?'' A little bit of leverage there? And then right on to World Building, ''What properties of a magic system would delay its discovery into the late first industrial revolution?''
SC Fascinating.
BP Magnification? If magic is made of germs maybe?
SC Yep.
PF I do love how many questions on Workplace are basically the answer is, ''what do you want? What do you need?''
SC Yeah, what do you want? I was really surprised by the answer to my management block my internal transfer because the top voted answer on that one was kind of like, ''Don't trust management. They are evil.''
YE Ben, you're missing, the next one, I think is really just the key one. ''How did Mr. Bean managed to make money?''
SC Well, yeah, I've never wondered that. But now I do.
BP Mr. Bean has a house and a good car. But all he does is funny things and nothing else. How did he get paid?
SC Same. Same. Actually.
PF Yeah, they always cut out the parts where Mr. Bean is a drug kingpin, sells tons and tons of cocaine and they've just left that out of, you know, they edit that out for the BBC.
BP According to Wikipedia in the first film adaptation, he has a job as a guard and an Art Gallery in London. He's so quiet. He's perfect for an art gallery.
PF Mmm it feels like they red-conned that in to cover up all the drugs.
BP And then of course, there's all those silly questions and then from physical chemistry. ''Why is blackbody radiation curve smooth without a sharp cutoff?'' Hmm, I'll just I'll answer that one later when I have a few minutes.
25:58
SC Yeah, nice.
YE So it's amazing how many sites there are. So I mentioned the top five, but we actually had a questions and answers posted in 44 different sites during, during the event. And I would encourage anybody listening to this to go to StackExchange.com/sites, and to find some site that you've never seen before that looks interesting. And click through and find something to ask or answer.
PF Hey, Sara, I have some really good news for you. There is a Bitcoins site.
SC Oh, I know.
PF Oh, you know, how'd you find out? That's amazing.
SC Yeah, no, I know about it.
PF Well I'm gonna go in there and learn all about Bitcoin.
SC Yeah you can go learn about Bitcoin, there's an aetherium Stack. There's all kinds of different crypto.
PF Hmmm I'm so excited to learn more about the world of cryptocurrency.
SC Yeah, this week, and this week, I bought a lot of altcoins that was my that was my week this week.
PF Alright, good. I learned a lot. I'm actually fascinated by the idea of the Community-a-thon inside of an organization where people, you know, it's like, get involved. This is a really proactive low impact way to get people more connected and that's very powerful, especially if you build communication and sort of team empowerment tools. So yeah, make sense to me.
SC Yeah. We've actually had some customers reach out and ask us to help them host Community-a-thons, which is really cool.
BP Ohhh, that's cool.
PF Very smart. Very wise.
BP Alright, y'all. This is Ben Popper, Director of Content here at Stack Overflow. Thanks for listening. If you want to hang out, you can find me on Twitter @BenPopper.
YE I'm Yaakov Ellis I'm a principal web developer and community advocate on the public platform team here at Stack you can find me on Twitter @Yaakov or hanging out in Tavern On the Meta or meta Stack Exchange.
SCantor Hey, everyone. So I'm Stephanie Cantor. I'm the program manager for the community team at Stack Overflow. And that's it.
27:49
SC I'm Sara Chipps, Director of Community here at Stack Overflow. And I can be found at @SaraJo on GitHub.
PF I'm Paul Ford. I'm a friend of Stack Overflow check out my company Postlight.com. I'm a co-founder and we are a good digital strategy partner.
BP Alright everybody, thanks for listening. Be sure to hop on Stack Exchange. There's new stuff happening all the time and plenty of questions that need your help.
[OUTRO MUSIC]