For this episode, we spoke with Carol Lee, PhD, principal research scientist in the Developer Success Lab at Pluralsight, about her research into code review anxiety, how developers are coping, and how a workbook can help.
Carol is an applied clinical and intervention scientist: she develops and tests cognitive, behavioral, and social interventions that activate key mechanisms to elicit change. Learn more about understanding and mitigating code review anxiety (the full version of her article is here).
You can also check out the code review anxiety workbook.
Pluralsight’s Developer Success Lab is a team of scientists studying how developers work, learn, and innovate.
Explore more of Carol’s work on code review anxiety, her bio, or her other work, from developer productivity and stress management to coding with GenAI.
Connect with Carol on LinkedIn or Mastodon.
[intro music plays]
Ryan Donovan Hello, everybody, and welcome to the Stack Overflow Podcast, a place to talk all things software and technology. I'm Ryan Donovan, your humble host for the day, and I'm joined today by Eira May, partner in crime on the content team. So Eira, you brought us a great guest for today. Can you introduce her for us?
Eira May Hello! Yes. Today we are joined by Dr. Carol Lee. She is the Principal Research Scientist in the Developer Success Lab, which is a Pluralsight project. And her work is as a behavioral strategist, but her background is in clinical and intervention science, and she has been doing some work on a lot of different areas of developer experience and how developers excel at work, but what we wanted to talk with her about today is her work on code review anxiety. So Carol, welcome to the show and thank you so much for joining us.
Carol Lee Thank you so much for having me. I appreciate it.
RD I'm sure there's plenty of developers who could use an intervention on their code reviews. So Carol, at the top of the show, we like to ask our guests a little bit about how they got into technology, how they got to where they are today. What's your origin story?
CL Sure. So like you said, my background is in clinical psychology and so I've really devoted my career to improving folks' wellbeing, thriving, health, etc. And I've done this in a variety of ways– as a therapist in a hospital, as an academic, a professor, a scientist, etc. And in all of these scenarios, you do this with the help of software. And it’s kind of a silly thing to say, but at the time, I had never really considered that there are people who write the code behind this software. I hadn't even really considered the fact that I was using software until, obviously, one day I did and I had this moment of, “Okay, there are these people who write these things that kind of keep our society running, including our health systems, our education systems, transportation, etc. And how are they doing?” And the answer is that there's not a lot of science showing or really exploring how these folks are doing, and a lot of times developers aren't doing very well at all. And so this really kick-started my interest in, “Okay, well, how do we help these folks improve their wellbeing and improve their thriving, both at work and their lives, et cetera?” I kind of think of it as an ethical issue as a psychologist, actually. Folks are working on really important things. We need to make sure that those folks are doing well.
RD Nice. Developer happiness is definitely top of mind for a lot of folks these days.
CL Yeah.
EM So can you give sort of a high level overview of what a code review is? I know Ryan has worked as a developer, unlike me, and can maybe speak directly to the experience of code review and code review anxiety.
RD I haven't been a developer. I was a technical writer for 15 years, but I worked closely with a lot of developers.
CL Ryan, if you want to give an ultra cool technical definition, go ahead, please feel free to jump in, but kind of the TL;DR of a code review is that developers are checking each other's code and giving feedback on them to check for any errors, any inconsistencies, security risks, etc, etc. And so it's actually a great time for folks to ensure the quality of their code base as well as the security of their code base. But it's also really interesting because it turns into kind of a social and a community time to connect as well. So a lot of times folks will embed things like trust building or community building or learning transfer. They're kind of sharing knowledge with each other, kind of semi-mentoring folks on their teams with each other and through these code reviews. So it's both a tool and yet a sociocognitive process on a lot of teams.
RD It almost sounds like the workshops that I'm sure Eira and I have done where you write something and then everybody bangs on it until it's better.
EM That's what I was thinking of, too. Reading your work is thinking about my fiction writing workshops back in the day where you send in a story ahead of time and everyone reads it and then they just sort of go around the room telling you what's wrong with it.
CL Or for scientists, it's like anytime you send something into a peer review journal and you get back all your peer review feedback and you're like, “Wonderful. Thank you for ripping this apart.”
EM Thank you for sending back the tattered corpse of my draft.
RD So besides the weight of judgment of your peers, what makes these such an anxiety-riddled experience?
EM Yeah, what could possibly be stressful about this experience?
CL No, but it sounds like such a wonderful process. I think because of this evaluative process, it very naturally triggers our social anxiety. So if you think about code review anxiety as our social or performance anxiety with our teammates, this has an impact on us in terms of how we feel about ourselves, and it also really negatively impacts our ability to engage. One thing we know about anxiety is that it actually increases our avoidance of things, so it's a very natural kind of reaction. You're anxious about this thing, you get rid of the thing, wonderful, everything's great, but that actually makes your anxiety way, way worse in the long term. And so avoidance is not very helpful, and then also obviously reduces things like productivity, your team effectiveness, your career progression, all of these things.
RD Did you find at all that people were committing less code because of this anxiety?
CL A lot of folks reported that either they would completely avoid picking up code reviews, for example. So maybe someone's like, “Hey, I need a review,” and folks are like, “I'm just going to pretend I didn't see that request. Maybe somebody else will pick it up.” So just kind of completely avoiding it because this anxiety refers to not just getting your work reviewed, but also reviewing other people's work. And then folks might also procrastinate, they might just kind of put things off, or someone was like, “I always wait until Friday so I don't have to deal with it all weekend and I just push all of that until the very last minute.” So definitely you see a lot of avoidance.
EM That's interesting what you said about code review being a source of anxiety for not just the folks who are being reviewed, but who are doing the reviewing. I hadn't really thought of that. I was thinking about pair programming or over the shoulder code review and thinking, “Gosh, that could be really anxiety producing, especially if it was a new area of work for you,” but I wonder, could you talk a little bit more about the anxiety of people who are doing the reviewing, the people who find that to be a real source of stress and tension?
CL Yeah, of course. So I think there is this misconception that only junior developers or folks just starting out experience code review anxiety, with the assumption that it's only because you're experiencing the anxiety when your work is being reviewed. But if you think about it, anytime you are a reviewer, you're essentially asked to contribute your expertise and so there is an element of, “If I mess up this review, I was the gatekeeper of this code. And if I mess it up, that might be my fault.” So there's a lot of pressure there. There's also a lot of pressure for folks who are senior or principal staff, et cetera, just developers who feel a lot of pressure to be the best on the team, to have all the answers, and that's a lot of pressure to put on a person to have all the answers all the time, because not all of us have the answers all the time, of course. So you can imagine folks might say, “Oh, I'm going to ask Ryan to review my code because blah, blah, blah. He's been here for 10 years or whatever,” and now Ryan is in this position of always feeling like I have to be right all the time because I've been here for 10 years. So that is a lot of anxiety that comes with that.
RD We talk a lot about productivity metrics like the Dora metrics, which is time to delivery, and a lot of folks and companies have identified the problems in code review. That's a big blocker for folks. Did you notice any sort of measurable slowdowns in the process?
CL I didn't look at any measurable slowdowns in that sense. It was largely self reported in terms of productivity or blockers in that sense. But folks across the board reported a lot of just general slowdowns in asking and receiving code reviews because of this anxiety, for sure.
RD And then you mentioned the folks who would save it until Friday. That seems like a real tough one to build a software program around. Was that sort of just a one-off anecdotal one or did you see several ones where it's like, “We build this into a certain day.”
CL That was just a one-off anecdotal one. I will say that when that person shared that in one of the workshops, a lot of people kind of nodded their heads or kind of awkwardly laughed the, “Oh, I feel a little too seen” laugh.
EM So I know a lot of times there's a reluctance to kind of make recommendations based on academic research, but I also know that in part what you do in your work is try to think of actionable ways that developers can experience less code review anxiety or manage it better. Can you talk a little bit about some of the pieces of advice that maybe came out of this for you that you would give folks who are struggling with code review anxiety?
CL Yeah, absolutely. What we did was a randomized controlled trial, which as you know, is a very academic kind of research. This is the same thing that you see in a clinical trial, for example, but we actually made efforts to make that intervention widely available. So something that you'll often see in a lot of health or intervention science is that these scientists and academics will design this beautiful intervention, empirically test it, and be like, “Yay, it works! We have evidence that this intervention works.” And then that intervention just kind of disappears. They write it up in a journal, they don't disseminate it out for folks to use unless you pay a ton of money or something like that. So for us, you can kind of think about these values as being very similar to open source where the value is to create something that is used. I don't do this intervention work so that I can say, “Good job, Carol. You helped these people in your study with their code review anxiety.” Ideally, I would be able to help anyone who experiences code review anxiety, even making it available for folks who don't experience code review anxiety so that they can help folks who do. That's how you create change in the long term. So we created a workbook that distills the intervention itself that's been tested and shown to work. We created this workbook and very freely disseminated it online, so anybody can download it, anybody can use it. It talks about and teaches you the skills that we go over in the research study itself– skills that have been shown to work time and time again. So that was a big value there. I'm also just really hopeful that doing this work kind of sparks people to talk about their anxiety more. Something we see about code review anxiety is that it's actually very, very common, and it's very common not just in junior developers, but it's also very common in more senior developers. I think kind of strikingly, folks in my research sample had anywhere between 1 to 65 years of coding experience. So folks experience code review anxiety, everybody experiences this thing.
EM Across the board, regardless of where you're at in your career. And I can sort of see how it would come in different forms at different stages: junior devs versus a principal developer who sort of asked to be where the buck stops in terms of expertise, but there would be anxiety for everybody at that stage.
CL Yeah. It’s so interesting because it's so common but people don't talk about it. And there's this thing that happens where when you don't talk about an experience, you feel like that experience is very invisible. And when an experience is very invisible, you suddenly become very, very lonely in that experience, even though you're not actually alone. So we all have this anxiety but we all feel very alone in it, and that's keeping us from actually doing something about it. So I think a lot about this work as putting a spotlight on it to say, “Hey, this is a thing. It's normal. Y'all experience it, and we can actually do something about it.”
RD I think people do talk about it a little bit. They're starting to talk about it more, but they talk about it as imposter syndrome– as ”I don't feel like I'm good enough.” There was an interesting bit somebody said in an article that we published that part of what can get you over the imposter syndrome is actually getting the negative feedback. I wonder what you think about that.
CL I think imposter syndrome is really complicated. There's a lot of evidence showing that there are a lot of social factors that lead to imposter syndrome. A lot of women, women of color in particular, experience imposter syndrome for reasons because the systems that they're in are actually making them feel that way.
EM Right, because people actually don't treat them like they belong.
CL Exactly. It's not imposter syndrome if everyone's being mean to you.
EM A hundred percent.
CL So I hear something very broad like, “Negative feedback will help you get over imposter syndrome,” and I'm like, “Hmm, depends,” I'm going to say. It really depends. How is it being phrased? Are folks kind of recognizing your work as a person still? Is it not rooted in sexism, racism, whatever? So I think a lot about those kinds of contingencies, I suppose, as a psychologist.
RD I think that the helpful feedback, how you could be better, that's the sort of negative feedback that I think they were talking about, but you don't always get that.
EM I think there is something to being used to being on the receiving end of feedback that kind of allows you to approach it a little bit more dispassionately and not see it as a source of commentary on your abilities. I'm good at doing this with my writing, I like to think, because I've been a writer for hire for a long, long time and I'm used to getting that customer feedback and I can sort of see where, if I were newer to a subject, it might feel very crushing to get that constant feedback, especially in the early stages. So I can kind of see a parallel between code review anxiety and this imposter syndrome. But like you said, one of the things that Cat Hicks talked about when she came on our podcast last year, I think, was about how a lot of times imposter syndrome is kind of a convenient explanation for the way developers don't actually get a lot of support at work. I wonder, do you think there's something maybe similar going on with code review anxiety, that maybe it's not actually this process that's inherently anxiety producing, but the way that we handle it and frame it?
CL I think any social performance review process can be anxiety provoking. It just can happen. There are definitely things that can happen to make it even worse or set you up for success or failure. You made a comment about how maybe you receive a lot of feedback and in the beginning it feels really crushing, but as you kind of move on, you feel like, “Okay, this is just feedback. It's not about me.” And I think that's a really big key piece here. There are some folks who never get that mentorship, that never get that feedback, and never get that support that says, “This is really crappy feedback, but it's not about you as a person.” Somebody lets them know that, and that is a really helpful frame of mind. We call this process ‘de-centering’ in psychology. It's when you're able to kind of move past or kind of see the bigger picture. You don't think about this feedback as being indicative about who you are. But there are other people who receive that soul-crushing feedback and are never told, “Hey, this doesn't mean that you're a bad writer. This doesn't mean that you're a bad developer. This doesn't mean that you're a bad or unskilled or incompetent person.” They just are told, “This is bad.” And it's really hard not to think, “Well, maybe I'm just bad at that then.” So if you never received that kind of mentorship or that feedback or support from the environment, from a trusted person, whatnot– and as you alluded to earlier, women of color or other minoritized groups tend to be in this category as well where they don't receive as much support– then it's much easier to kind of go through your career and be an experienced whatever and say, “I still feel anxiety about this because the only thing I've ever heard is that when this bad thing happens, it's your fault.”
EM Totally. I think that's a really important point.
RD Is there a way to build that into the feedback you give in code reviews?
CL Yeah. Actually in our workbook we talk a little bit about how to give and ask for feedback. So for example, I think an important thing as developers asking for feedback is to actually express how you feel about your work. Oftentimes we just kind of submit our work and say, “There you go,” or, “Ready for a review,” and the other person has literally no idea what's going on. They're just like, “I guess this person feels confident.” It's very easy to review a piece of work and forget that there's a real person with feelings on the other side of it. I'm sure you can both relate to this.
EM Absolutely.
CL But when someone submits the thing, submits the code, submits the writing or whatever and says, “I wasn't really feeling confident about this part,” or, “Please be gentle with me,” it kind of cues you to be like, “There's a person on this other side. I actually quite like this person. This person is my teammate. I care about their feelings. And now I'm going to be approaching the review in a much kinder way.” And then similarly, when you're giving a review, I've heard people say it takes too much time to write something nice during a code review. And I get that. It takes time to write another sentence, but I'm just going to argue that taking the time to write that extra sentence of, “I see the work you've put into this, I see you as a person,” that can make a really big difference for somebody's career in the long term. So I just encourage you to take that time. I promise you, writing that sentence will not kill you and it will make a big difference for other people.
RD We've talked a lot about psychological safety in terms of making better teams, and if you're able to give honest, good feedback and you won't get shut down, or you're able to put in whatever code, maybe try something, you'll have a better team overall.
CL Yeah, for sure.
RD Speaking of which, did you look at the negative impacts of the code review anxiety? Besides people delaying doing the code reviews and putting less code, were there effects on software development?
CL I didn't look at that specifically, but there has been research done in the past showing that when folks avoid or procrastinate code reviews, it does lead to delays in software. It also results in less cohesive teams and lower trust teams, which is obviously very important in software development. So there are those connections that can be made.
EM I wonder if there's anything surprising that you uncovered when you started looking into code review anxiety. Did you sort of have a picture in your head of the kinds of anxiety that people were going to report, or what came out of it that was surprising to you, if anything?
CL I think this is a little bit twofold. It did surprise me how many senior or principal folks experience code review anxiety. I confess that was a little bit of a surprise. I also had this bias that it would be more junior or early career, which of course is very wrong once you dive into it a little bit deeper. I think the other surprise I had was that there is advice on how to manage code review anxiety out there on the great interwebs, but a lot of it is very, very bad. I'm an anxiety expert –that's my expertise– and I'm like, “Wow, this is a great way to actually make somebody's anxiety a lot worse.” So please don't do that.
EM Okay, now I have to ask you what was the worst piece.
CL So there's two main things that people will kind of push. One narrative– people are like, “Oh, don't worry about this thing called code review anxiety. We all have it. You're just going to grow out of it,” which is kind of wild because that's not how anxiety works. You don't just grow out of it.
EM And it doesn't sound like people sort of age out of it in career terms either if there's a lot of principal developers who struggle with it.
CL Exactly. It's a very strange thing to say. And also there's this other piece of it where, let's assume that we did grow out of it. If we did grow out of it, that doesn't mean that we should just ignore the fact that we're suffering in the meantime. You're like, “Oh yeah, it sucks right now. Oh well.” Very silly. The other main narrative is that people will say things like, “Oh, here are 10 things you can do to make your code cleaner, to write better code, and once your code is solid, your anxiety will just disappear.” And it's like, “Listen, you can be really good at something and still feel anxious about it.” Again, anxiety isn't something that you deserve because you did something poorly. It's also not something that you earn your way out of because you did something well. It's not like merit, it just kind of comes for us all. So I think those two are really big narratives that are pushed that really shouldn't be pushed because it kind of leads to a shame spiral where you blame yourself for your anxiety, etc, etc.
EM I mean, “Write better code and you'll have less anxiety about it,” doesn't seem to me to be very actionable advice. “Oh, I'm anxious about how my work is going to be perceived.” ‘Do better work.” If I could just do that.
CL I just can't even imagine if someone was like, “You know, Carol, your work is good, but since you're feeling anxious, it should be better.”
EM Thanks, that’s helpful!
RD What do you think I'm anxious about?
CL There's nothing to be anxious about. Just be better. Oh, okay.
EM I think we fixed anxiety, you guys.
CL A hundred percent. Next paper.
[music plays]
RD All right, everybody. It's that time again where we shout out somebody who came on stackoverflow.com and gave us a little extra knowledge. Today we're shouting out a Lifeboat Badge, awarded to Nidhi Agarwal for answering the question: “Logistic Regression on factor: Error in eval(family$initialize) : y values must be 0 <= y <= 1.” It's very long, not really a question, but they saved it. I've been Ryan Donovan. I edit the blog here at Stack Overflow. If you liked what you heard today, please like and subscribe. A rating and review really helps. If you want to reach out to me, you can find me on X @RThorDonovan.
EM And I am Eira May. I am also a writer and editor here at Stack Overflow. You can find me on the social medias @EiraMaybe.
CL And I am Carol Lee. I'm at the Developer Success Lab at Pluralsight, the Principal Scientist there, and you can find me on Mastodon at @CSLee.
RD Thank you very much, and we'll talk to you next time.
EM Thanks, everyone.
[outro music plays]