We chat with Isaac Lyman, a developer who wrote a great piece on code quality. Far from premature optimization, focusing on code quality early has huge benefits for the company's bottom line and its developers' mental health.
Isaac's piece, Code quality: a concern for businesses, bottom lines, and empathetic programmers, ran recently on the Stack Overflow blog.
A simple metric for code quality code be how easy is it to delete any given piece of code.
There's no algorithmic way to judge quality code, but experienced engineers know it when they see it.
Jeff Atwood's Performance is a Feature blog post gets a lot of mileage with our writers. But code quality isn't on the same axis; it's not a feature you can prioritize. It's part of the development process.
Isaac Lyman It really is a bottom line decision. Code quality pays off big time in a matter of weeks, you know, it doesn't even take months to years, it pays off in weeks. And study after study has found just what big of an impact it has. And so paying attention to code quality is a little counterintuitive from the business side. But it's so, so important, you know, for companies that plan to have a code base that, you know, that lives on beyond the, you know, immediate horizon.
[intro music]
Ben Popper Hello everybody, welcome back to the Stack Overflow Podcast. I am Ben Popper, the director of content here at Stack Overflow, and I am joined, as I often am by my colleague, Ryan Donovan. Hi, Ryan.
Ryan Donovan Hey Ben. You're silky smooth today.
BP I'm on this new USB only Shure microphone, and I love it. No more carrying around this preamp and mixer for me, man, those days are done. But we have a great guest and a great topic. We're gonna be talking about code quality today. And its impact, you know, for the business, obviously, and the bottom line, but also about its place within an empathetic workplace and the sort of, you know, like mental health or sort of working conditions of the people who have to grapple with the codebase every day. And so our guest today is Isaac Lyman, who is a senior engineer at Health Catalyst. He wrote the piece which you can find up on the blog. Isaac, welcome to the show.
IL Thank you. Pleasure to be here.
BP So I guess tell folks a little bit about, yeah, sort of like how you got into the software business and the position you're in now. And then tell us a little bit about sort of what inspired this piece that you pitched us?
IL Well, I've been coding since I was a kid, you know, like, I'm sure, like a lot of people you've met. And when I started it wasn't really something I ever thought of as a career option. You know, Silicon Valley hadn't really happened yet. And so for me, it was just, it was something that nerds did. And I was a nerd. And so I found it appealing and relatable. So I started out in c++, my brother had a PDF on the computer of c++ for Dummies, you know, I started reading through that and got two pointers and gave up more than once. But that was kind of where it all started. And from there, I learned Python, maybe a year or two later. And you know, in high school, I was doing a little bit of web development. And until my sophomore year of college, I never really thought of computer science as something I would do as a career. But, you know, I ran out of money my sophomore year, and so I was looking at jobs, and just about everything was tech related. And so I was an English major went on to graduate with an English degree. But I leveraged that to get a job as a technical writer. And from there kind of weaseled my way into QA engineering, into software development and software architecture. And by the time I graduated, I had a job lined up at a tech startup. And you know, it was such a slam dunk for me at that point, I had been planning to go to law school, but I didn't really want to go to three more years of school, just so I could, you know, make the same amount of money have, you know, roughly the same lifestyle that I would enjoy as a software engineer. So anyway, fast forward a few years and here I am at Health Catalyst, which is an amazing, amazing company really, really happy here been here for over four years. I'm hoping to stay for a lot more.
RD I was gonna say, a tech writer. I knew there was something I liked about you.
BP Yeah, exactly. I could smell the English major coming off of you.
IL Thank you.
BP You're looking at a former English and dance major so I'm employable no matter what the situation calls for. When you say you ran out of money, you mean to pay for school, or just like to have fun alongside of school?
IL Like all the way down to $0 in my bank account, I mean money for rent, money for food. You know, my mom had luckily sent me with a box of, you know, dry beans and rice and stuff. So I wasn't starving. But it was pretty urgent that I get a job.
BP Yeah, you know, it sounds like you've had a lot of both personal and professional experience in this world. So tell us a little bit about what inspired this post? You know, this particular post. You know, in talking with folks on this podcast over the last two years, I think one thing that comes up a lot is you know, sort of that old PSA about coming into a new company and sitting down and trying to go through the codebase not understanding half of what's written or half of the comments, or in the case of somebody like the guy from Dwarf Fortress, you know, you've been working on the same codebase for 20 years. Even you don't remember why you built it that way in the first place. And so I guess, yeah, when we talk about code quality, what do we mean by that? And why do you think, maybe it's more important or deserves a bigger you know, sort of like place at the forefront of people's minds, both the coders or within organizations that it has right now.
IL I think you've brought up all the right points there. Because really, to me, the best measure of code quality is how long it takes a developer to understand a piece of code, not just what it's doing, but what other things that may influence or what other things that may be influenced by, how long does it take you to go from zero to I understand completely what will happen if I change or delete this piece of code. And many people think of code quality is how easy a piece of code is to delete, which I think is also useful heuristic. But the bottom line is, it's something that is not generally well understood outside of software development. And it does take some investment. And so it's, it's naturally, something that has to be advocated for, it doesn't happen on its own. And it can be hard to advocate for, because you're explaining to someone who, you know, doesn't have extra time and money to throw around that you have to spend the time and money now, to write high quality code, or it's going to become, it's going to become technical debt. And it can very rapidly become extreme technical debt, technical bankruptcy even, when developers aren't able to understand the code, build on it, add features without creating a lot of bugs, as I kind of mentioned several times in the article, it really is a bottom line decision, code quality pays off big time in a matter of weeks, you know, it doesn't even take months, two years, it pays off in two weeks, then study after study has found just what they have an impact it has. And so paying attention to code quality is a little counterintuitive from the business side, but it's so, so important, you know, for companies that plan to have a code base that, you know, that lives on beyond the you know, immediate horizon.
RD Yeah, it feels like that a lot of companies, when they're sort of trying to get market fit, trying to get an MVP up and running, they're not going to think about this as much until they are, you know, have a sprawling codebase with, you know, hundreds of engineers working on it. Is there an argument to be made to starting with quality code from the beginning?
IL Yeah, absolutely. As with everything, there is some gray area, if you're making a proof of concept, or a quick prototype, and you don't expect it to last longer than a week or two, then it may not be worth the effort of writing high quality code, there is one study I found, where they found that for very specific kinds of projects, it wasn't worth the investment. I think those types of projects are fairly rare. And the trouble is that code quality, you know, as a concept, it's it's not magical. It's a set of techniques and rules that you can learn and apply, regardless of how you got into the field of software engineering, you know, regardless of how much experience you have, but it does require some amount of education. So it's more likely to be found in code written by experienced engineers, which becomes a cost consideration. A lot of startups are running lean, they don't have necessarily a large engineering budget. And so yeah, there is a huge temptation to hire cheaper, less experienced engineers when you're starting out. And if that's what you've got, then it's what you've got. However, you know, you do need to be aware of the long term consequences of not having solid technical leadership in your code base. You know, it means that there's going to need to be a lot of a lot of time invested later to pay off that technical debt.
BP That makes sense. I mean, I liked what you said before about, maybe the surest sign is knowing that you can delete something, and it won't cause issues. I think of that as like, can I cut this wire? You know, kind of like, red, green, blue? What will happen, you know, on that countdown clock? But I guess, you know, it's interesting to hear you say sort of like empirically, or at least based on a number of, you know, like maybe peer reviewed studies, we can see the impact of code quality or lack thereof, or we see which kind of projects it's, you know, productive foreign, which kind of projects maybe it's okay to avoid. Would you argue there's like an objective, sort of set of criteria or techniques through which you can determine and implement quality code? Or when people you know, come to the table with a bunch of different ideas about what's best, as they often do when it comes to programming languages or frameworks or, you know, approaches to architecture inside of code?
IL Yeah, you know, there is a study, I don't know if I linked it in the in the article, after all, but there is not extremely widespread agreement about what comprises quality code. in general. Most engineers agree that it's a measure of readability and organization. But as far specifics, there are going to be a lot of different people who come up with a lot of different criteria, right? As far as measure during code quality, I don't believe that can be done algorithmically. I do believe that experienced engineers have a pretty good feel for it. They know when they see it. And there are some things that could be measured algorithmically. But then they could also be gamed, as I mentioned. So as far as implementing code quality, like I said, it's a matter of education. A well educated junior engineer can write quality codes, and a well educated senior engineer can write quality code. There are a number of different like very specific factors in it, things like proximity and loose coupling. And I mean, there have been several books written on this subject.
RD So the standard best practices, solid principles and all that.
IL Yeah, exactly, exactly. So if you're looking for what is code quality, how do we do code quality, there is a glut of information available about that.
RD How did you learn about quality code? Was it something that that you got out of school did you get it through peer reviews?
IL It was something I learned partially by working with very experienced, and very high quality engineers. And the first startup I worked at my team was very, very good. And so I got a lot out of out of pull request reviews. And out of the planning process that we did before each piece of code, I also read a good chunk of Code Complete during my first few years in the field. And that made a huge difference, I highly recommend that book as a starting point. And then just just a matter of kind of picking things up as you go along. I mean, engineers do a lot of just in time learning. And if you're paying attention and reading a little beyond just the initial snippet of code you're looking for, you can often find best practices and start to build that intuition for code quality.
BP Something you said that interested me was this idea that you have to make the business case about the importance of this, and that perhaps it takes a little bit more training, or perhaps it means you have to hire more senior and therefore cost engineers or perhaps they'll go a little bit slower in crafting the code, therefore, you know, it'll be a little bit more time and expense. But I'm curious, you know, if you were at a startup or a company with a technical or engineering mindset, why they wouldn't appreciate this as sort of an intrinsic value, you know, in the way that somebody manufacturing an airplane would believe in triple redundancy, or something like that, like, where's the disconnect between leadership and management or entrepreneurs? and engineers, that means people are more inclined to do what you said, which is move fast and break things, Hope it scales, and then when it gets there pay down the technical debt? Is that because we sort of lionize that in our, you know, version of what a tech company a software company is, or what do you think drives that paradigm?
IL Yeah, I think it's it's not so much a matter of someone in the organization saying, we don't care about code quality. I don't think anyone thinks that or says that. I think you can ask any CEO anywhere, does your organization care about code quality? And they're going to say, yes, absolutely. The disconnect is not so much as to whether it's important, but as to whether the things that are required to make it happen, are priorities within the company. And there are several ingredients for code quality, that you can't skip out on and still get quality code, things like a healthy work life balance, you know, an engineer who hasn't had a break in a long time isn't sleeping, isn't eating well. I mean, we talked about hustle culture and grind culture, those things don't lead to good mental health. And you know, a clear mind, these are things that that are really, really essential to writing quality codes, is that you're caring for your mind and body. And then, you know, deadlines are another important factor, you know, is a schedule of very, extremely tight deadlines, and a culture of not properly testing things before they should. I mean, there are a lot of things where people will cut corners. And they don't think it's about code quality, but it ends up being very much about code quality, because you know, the engineers will still write code that works, because that's table stakes. But if they have to cut something out, and everything else that they can cut out has already been cuts, code quality's going to go right out the window.
RD It seems like you know, if you're lucky, you get code quality and the first draft but takes a lot of refactoring and revising. And if you have tight deadlines, if you have a bad work life balance, you're not going to have the time to to basically rewrite the code to make sure it's quality.
BP I think I sort of see where this is going. Isaac, tell me like, you know, I sort of imagined the conversation I've had before but it's kind of like a company decides that you know, wants to create a certain kind of product, it begins to have some success, it raises some venture capital funding. Now it's playing a certain kind of ball game where they have to scale at a certain rate. their growth has to be rocket ship like, and then that they can get to the next funding round and get to the higher valuation. So the executives are saying, well, we need to ship XYZ feature and show XYZ traction within this many months. And that engineering team says that's not possible. And they say, okay, well what do we have to do to make it work, and they come up with a bunch of ideas. And somewhere in there unspoken, is code quality will suffer, right? Like, I can imagine that conversation happening within a lot of companies. And it's hard to justify the same. And I guess it's hard to argue against because you have so many competitors, right? Like, if you had the funds and the time to do it, right, of course, you would do it right. But if you're have a certain amount of runway, you need to hit certain milestones, to get more money to pay these engineers, who are getting job offers every day from seven other companies, you know, I can see how it starts to fall by the wayside. Have you seen effective strategies for communicating it? across the company, so about up to leadership or management, but also, as you said, empathetically between coworkers, which is to say, doing this is also a way of maintaining all of our mental health, right, making it readable to everyone and transparent to everyone, what was strategies do you see that are effective for getting people to recognize the importance of this? And then also, to put it into practice?
IL You know, if what you say is, is correct, there are a lot of forces in technology in the field in the market, that are not conducive to code quality. And this, this rocket ship trajectory isn't right for most companies, in my opinion. And, you know, it's ridiculous to think that a company can continue to grow forever, I think. So yeah, there's a lot of things working against it, as far as things that are effective in advocating for it and making it happen. I think technical leadership, and a culture of respect are kind of the foundation there, you know, a good technical leader will stand firm on code quality, and will say we need to make other trade offs. And there are other trade offs to be made. In every situation I've seen, there's an opportunity to say, I know our big client really wants this feature, but we're, there's gonna have to be some compromise. You know, we decides what value to assign to each feature, how much effort to assign to each feature, this is the big job of project management, and then you prioritize, and by nature, that means making some people unhappy, and not delivering everything you would like to deliver right away. But you have to prioritize in order to make time to carve out time for the engineering process to happen in, you know, in the natural and in the highest quality way. It has to be a matter of of standing firm on it and and prioritizing, you know, sometimes brutally what you're going to deliver and when.
BP I guess yeah, you know, it's interesting to hear you say that the thing that comes to my mind is like, when you build a building in real life, you know, there are rules and regulations about quality that are unfortunately not always followed, but at least they should be. It's almost like within the company you need like, like you said, someone who can stand firm or, you know, an ombudsman, you know, like as a person who comes in inspects and says like, we don't have this minimum level of quality before we ship this, like we're gonna pay more dearly, down the road, you know, and so like, that's the person who gets to decide or at least put their foot down. But you know, draw a line in the sand that says it's not worth making these sacrifices now to rush to get the client the shiny, yes, biggest thing that you know, they asked for won't pay off. And I think that's really interesting to think about.
RD One of the things that made me think of talking about the people not prioritizing code quality. I'm not sure a lot of people think of it as a feature. I think there's a famous Jeff Atwood quote that that comes up in our blog posts a lot, that performance is a feature that you can prioritize it or not. And I don't think that people even consider code quality among that. They're just like, we got all these other features. Let's get him in there. Get it done with these deadlines. Is there some way to make people think of it as a feature that they should prioritize?
IL Yeah, I don't know that I consider it a feature on par with, you know, a web page or an API, because I think of it on a different axis. It's all about process. If your process is conducive to code quality, then you will get quality code. And so you know that there are ways that you can change your process. And I think a lot of us that have been in the field, you know, for any amount of time at all understand what makes processes work and what makes processes fail. Like engineers, just like anyone else will go the path of least resistance. So if there's no one there making the process happen, and forcing it, and you know, repeatedly reminding people, when it starts to fall apart, then you know, without that it won't happen. It's all about like, do we take time to plan before we start coding? Do we take time after we code to review each other's work? Do we take time to test, you know, for, you know, a period of time before we release to production is there, there are different ways to approach it. But I think if you build your process with code quality in mind, there are a lot of ways that you can achieve it.
RD So, before we go, are there bits of code you wrote in early days that that you wish you had a chance to revise? Are you embarrassed by your your past selves?
IL Of course. I think that's what keeps engineers humble, insofar as we managed to stay humble, is occasionally you see an old bit of code and you think, wow, I really should have done this. And this, and this, this is really hard to read. So you know, the other day, I pulled down a bit of code from my GitHub just that was, you know, just to just something to calculate levenshtein distance between two strings. I changed a lot of things. You know, I originally put that code up there several years ago. And it's humbling and also gratifying to see how differently I would do it now than I did then.
RD Alright, that's a sign that you've learned and improved over time.
IL Absolutely.
[music]
BP Isaac, it was a pleasure talking to you. And thank you so much for sending over the ideas. And I'm excited we get to do both the story and the podcast.
IL Oh, thank you for having me, and I look forward to seeing the article up on the site.
BP I am Ben Popper, the director of content here at Stack Overflow. You can always find me on Twitter at Ben Popper, and you can always email us podcast@stackoverflow.com.
RD I'm Ryan Donovan. I edit the blog here at Stack Overflow. You can find me on Twitter at @RThorDonovan. And if you have a great blog post idea, you can email me at pitches@stackoverflow.com.
[outro music]