On today's episode we chat with Lance Cleghorn, who describes himself as "working to integrate crowd sourced security into the DoD [Department of Defense] through innovative bug bounty projects. AKA Hacking the Pentagon." Nothing we've ever written on LinkedIn has sounded that cool.
Cleghorn works for Defense Digital Services. On Twitter, the group describes itself as "a SWAT team of nerds on tours of duty."
You can read more about the group's goals on their website.
You can see some of his work over on Hacker One.
Lance Cleghorn When you're working against a threat actor, you almost have to assume a breach. And you have to assume that something like this is going to happen. And then try and figure out like, you know, what should we have done?
[INTRO MUSIC]
Ben Popper Couchbase is a modern, multi-cloud to edge, SQL friendly, JSON document database for building applications with agility, performance, and scale. If you're new to Couchbase, and would like to learn more, the Couchbase Developer Portal is the best place to start. It's loaded with tutorials, videos and documentation, as well as best practice tips, quickstart guides and community resources, including the couchbase developer community forum. Ready to get started developing on couchbase? Visit couchbase.com/newtocouchbase.
BP Hello, good morning everybody, whenever you are, this is Ben Popper, Director of Content here at Stack Overflow, you're and listening to the Stack Overflow Podcast. So thanks for tuning in. Today on the show, we're gonna be talking about cyber security. And we're gonna be chatting with Lance Cleghorn. Lance, welcome to the show.
LC Hey, Ben, thanks so much for having me. I'm really excited to be here and get a chance to talk about some awesome security stuff.
BP Terrific. Well, yeah, it's it's been a popular topic. When I have hack somewhere in the in the title of the podcast episode or a blog post, it always gets significantly more traffic than your average post. So tell us a little bit about yourself. What is it you do? Where do you work? Just the sort of basic primer on who you are.
LC Yeah, for sure. So I work within the United States government, specifically within the Department of Defense in a little group called the Defense Digital Service. So Defense Digital Service, we like to self describe ourselves as a SWAT team of nerds, we work directly for the Secretary of Defense, and we aim to tackle some of the military's most difficult technology problems by sort of infusing industry best practices and problem solving to those. Within DDS, I'm in the engineering Guild, so we have sort of like different guilds based on people's skill sets. So I primarily focus on cybersecurity with a little bit of networking mixed in there. So that's, that's me.
BP Cool, you have guilds. You have classes and levels, is this a full Dungeons and Dragons like take?
LC I think we're working that way. So it used to just be like, you know, groups, and then we decided to rebrand as guilds, which I think is much more exciting.
BP Yeah. I mean, it brings in all the World of Warcraft players who probably know a lot about pawning people, at least. So how did you get involved in this world? Did you start out when you were young? Did you come from computer science? What led you to the position you're in now?
LC That's a great question. So when I was when I was very young, so I'm 30, right. And I originally grew up sort of in the era of like technology becoming a thing. And I can remember being like four years old and sitting on my grandfather's lap as we played Lemmings, like booting in the dos and stuff. So from like, I'm like a really young age, like I've been really into technology. And then, you know, going through high school, I grew up in sort of a rural area, but was able to get an NSA scholarship for school. And part of that involved coming back to the to the department and sort of doing like a, like a repayment of service. So it started out in DOD about eight years ago. And then I've done you know, anything, and everything you could probably imagine in the security field within DOD, from pentesting, to regulation and compliance. And then ultimately, like moving over to work with the Defense Digital Service.
BP So you went straight from school into the service, you didn't have any private industry experience or anything like that?
LC Yeah, so within school, I was fortunate enough to be able to work within I went to East Carolina University and fortunate enough to spend a lot of time there, actually working for the school providing sort of like educational IT support. So definitely count that as part of my time, you know, his full time gig, but then it moved over. I've been most of my career within the government.
BP And so after your computer science education, as you got into your career in government, are there certain languages, technologies or frameworks that are kind of key when you're in the world of security? And does that mirror what you would see in you know, a shop where you're building a mobile app or a cloud service?
LC This is a really good question. So I'm actually I wouldn't self describe myself as a, as a comp sci or a programmer. You know, I did technology, infrastructure and like networking, and then ultimately got a degree in cybersecurity. But it's a really great question, because the DOD is probably one of the, I think it is the largest single employer of human beings in the world. And so with that, like we--
BP Amazon's trying their best.
LC Right, they're working on it, maybe some Tesla there. But we have such a varied degree of people that you get almost every single language, Java, C, sharp, dotnet, everything you could imagine under the sun is within the DOD. And that not only presents like, you know, huge opportunities, but also creates massive challenges, right? Because it's not like security isn't one size fits all. And there's not really a great way to come out, you know, attacking and securing the DOD right, like it's huge.
BP And so you mentioned that you've worked in sort of, like different disciplines within that. Where did you start? And was this a natural transition, sort of based on what you were interested in? Or was this where the demand was like, what made you move around to the different guilds as they're called?
LC Yeah, absolutely. So you know, originally Starting out within within the government, I started out in sort of like, like almost like regulation and compliance. So, you know, looking at defense frameworks and how well DOD organizations were applying those, and I like to call that like compliance based security. And I think lots of people enjoy compliance based security. But, you know, for me, wasn't my favorite, you know, really took the opportunity to sort of focus on you know, how to get more to that like kinetic security where, you know, when the packets are actually hitting the firewall, and people are actually attacking a web app or something like how do you go about providing, you know, real world security. And so that's what I've spent a lot of my career focusing on trying to get better and trying to help the department really improve on.
BP So right in the first example, you're kind of going around and make sure everything is buttoned up and that the best practices are applied. And the second one, you're in a, you know, an actual situation, and you have to respond in real time or something to that.
LC Yeah, absolutely. And one of the major things that I work on at DDS is our Hack the Pentagon project. So you know, you said like, every time hack shows up in the in the podcast title, so you can definitely use it here. But Hack the Pentagon is essentially bringing crowdsource penetration testing, what's commonly referred to as like bug bounties, like you might see with like your, your HackerOnes, or your Bug Crowds, or sine x, and applying that to all corners of the DOD to help show where Yeah, we've done all the compliance based security. But when the rubber meets the road, this is how secure you actually are.
BP Yeah, I did a story about HackerOne, I think was back in like 2015. At the time, you know, that kind of bug bounty as a service platform is just sort of rising. And it's been fascinating to watch, I guess, do you think there's a need for that? Because there's just so much more attack surface and so much more has been digitized? Like, how come that is sort of the new approach that people are using?
LC It's a great question. So, you know, I think the sort of like the holy grail and people with with AI and security right now is figuring out how do you emulate like a real threat actor? Like how do you emulate an actual person with intelligence attacking your network? And that's one of the things that, you know, your your bug bounty effort really does that, you know, you could scan all day with something like necess and find vulnerabilities, but it doesn't really translate into, hey, I forgot to change the password, right? Well, you know, or I used a password that is ABC, 123, right. And that's where we found that like, these bug bounty pentesting, like crowdsource efforts, are really fantastic at approaching the problem from a really diverse background. And what we found is like people that grew up in India, or grew up overseas and learned, you know, maybe like a small school focusing on problem solving in a very specific way, are very different and approach problems differently than folks that might have, you know, learned from, you know, a major American university like like MIT or Stanford or something, right, a lot of times those folks will find things that, you know, an MIT graduate would never even think to look just based on how they've, you know, been educated and how they might approach the problem.
BP Yeah, that's fascinating. I was asking this question the other day, but obviously, you know, we were chatting before the podcast started about how everything has gone to remote. You know, that has its pros and its cons. It's a big adjustment for everybody. You mentioned that you did pentesting in the past, has there been a change in the world of cybersecurity now that nobody's in offices, there are no conferences, there are no work dinners. I mean, like, whenever I have to do these cybersecurity trainings, a lot of is about don't leave your laptop open at the Starbucks, like things that haven't been part of my world for over a year and may not be for another year. So I guess, has that like area of vulnerability, the physical, you know, security kind of diminished and has some other area grown in its place?
LC Yeah. So I think, you know, one of the biggest ways we've seen cybersecurity change is, so originally, when there was defense in depth, and there was sort of like layered perimeter based defenses, it was all about cracking the perimeter. And then we saw, you know, a huge shift in threat actors using spear phishing, and drive by downloads to sort of leapfrog over the perimeter. But now with with this remote work culture, that sort of blossoming, we're seeing sort of like a return to that like cracking the perimeter, like if you keep up with sort of major CVs coming out, then a lot of ASAs, a lot of VPN providers, and then most notably, like Zoom, and the major meeting, conference, software providers have been hit a lot, not only with just research, but actual, like findings and exploitation. So definitely, like, you know, waxing and waning, and sort of the things that are that are becoming the focus of threat actors and researchers.
BP And so I don't know what degree you can discuss it, but the big security story, you know, of the last year was about a sort of attack on the the supply chain, the infrastructure of, you know, what we think of as digital technology and networking technology, you know, and that made its way all the way into, you know, the US government in various places, but also across the private sector. So, was there a lesson that we should learn there? Was that something that you know, people saw coming but didn't have the resources to deal with? What about you know, that what was it called fire? What was the name of that?
LC So Sunburst, the Orion SolarWinds based malware.
BP SolarWinds, yes.
LC And so it's, it's a fascinating case study. So I think, over the past few years, and if you go back in time, this isn't the first time that we've seen like a supply chain targeted. I think it's probably the largest event and the most significant US based event, but it's not, it's not a new attack vector, but maybe one that we didn't take very seriously before, by and large as a community. And I think the trick here is it looks so genuine, right? Like it was very legitimate genuine looking traffic going to SolarWinds, that firewalls would classify as great, you know, from a network defense perspective, you would be certain that this was good traffic. And then in the middle of it, it was, you know, big time malware. So it's a, it's a fascinating case study and has like, I think, really far reaching impacts on, you know, how we approach security, and how we take things like, even like bug bounty efforts and apply them earlier in the development of products, and how we spread those out to focus not just on the application itself, but also the library is that it depends on the infrastructure that it depends on, and the other service providers that it may rely on for, you know, essential functionality.
BP I remember talking to my co hosts about this, and I was reading through like a Microsoft analysis of it. And they were saying, Oh, yeah, as soon as I see like, dot DLL, you know, my eyes just glaze over. You know, there's that that's at that point, they've got you, you know, like, and I guess, yeah, the question that came to my mind was, you know, Stack Overflow has some business and enterprise software, we have some clients who prefer to have everything delivered on prem and updated once a quarter as opposed to over the cloud. And we have to go through, you know, these sock one and sock two security audits, how come this vendor who had such a, you know, like, sort of high profile list of Fortune 50 clientele and people in government, how come they didn't have to have those same kind of, I guess, security audit on their end? Or did they? And then it was like they just missed this?
LC Yeah. And I think that's the trick is, like, they very likely did, like, I don't know, for certain that you know, what compliance audits they would have had to go through. But, you know, SolarWinds isn't a small outfit, right. Like, it's got a pretty significant permeation throughout the, throughout the industry, for sure, they went through audits, they probably had decent security, it's just, when you're working against a threat actor, you almost have to assume a breach, you have to assume that something like this is going to happen, and then try and figure out like, you know, what should we have done? Like, what kind of like code checking and like, code auditing should have happened before a new branch was checked in with this, but this malicious code involved? Right? Like, it's not outlandish to think that something should have caught this right.
BP Before we get off this topic. One more question about which is, you know, there was a lot of discussion about the fact that it had been there for some time and been, you know, at a certain level, that it would be hard to unwind, you know, hard to understand when less people had really left the network and, and really had cleaned things up to the point where they couldn't just sort of reinsert themselves. Can you explain for listeners who might not be as well versed in cybersecurity, sort of what the dilemma is there? And how you might go about fixing it?
LC Yeah, absolutely. So a lot of times what will happen with like, with a breach where a threat actor is sort of, like dug in or establish persistence, is that you really can't trust anything. If you go back and try and piece together from a logging perspective, what's happened, a lot of organizations don't keep logs past 30, 60, 90 days. So at some point, right, like, you're going to reach a situation where you're not 100% sure what's happened. And you don't really know how to go about dissecting what can be trusted and what can't. So you really get into a situation of you know, do you blow everything away? Do you just nuke it all and start over? You know, or is there some way to actually go through and just do a, you know, top to bottom audit of everything going on in your network. And you might imagine, like how complicated modern networks can be, and all the different things sort of flying back and forth, that's almost an impossible task, right? Especially if you don't have good logs, that can tell you exactly what's happened, you just don't even know.
BP So I think what actually got me inspired to reach out was that I was following someone on Twitter who I knew from my days in the New York tech scene, John Sawyer, and he was tweeting a little something about, you know, the sort of pros and cons of offensive versus defensive cyber security. Can you talk a little bit about that? Is that a paradigm that exists both within, you know, state actors, but also within, you know, the private sector? And how do people play on those two sides of the fence? Like, what's the current thinking about how you should approach that?
LC Yeah, there's a lot to unpack there. But I'll give it a shot. Yeah. So John, definitely, you know, is highlighting a really important thing that we a DDS have tried to focus on, which is the idea that red team exercises are actually like showing developers and networking engineers, and folks that may be securing the network, or that play some role in securing the network, exactly how a threat actor might think. That's very, very different than sort of blue teaming and, you know, building up your defenses, right. So, building the castle is great. But the first time, you know, I come and knock a siege weapon through the door, you know, you have to do something different with the door. Right? And so that kind of like sort of scenario based threat actor emulation is one of the most valuable things that we've seen. And it extends down like I know a lot of your audiences is developers, and it extends down specifically to developers, because I know a lot of developers look at security as sort of like this almost like ambiguous challenging thing, where people just come in and say no, right like, can't do this, can't code this way, can't use this language. But it's a lot more interesting, I think to look at what do threat actors actually do? And why do people come at them? Like a, you know, 'this isn't a good idea' perspective.
BP I guess, from my perspective, right? Like, it's clear. And I think out in public that at a nation state level, there's both offensive and defensive practices. And that, you know, there could be a healthy discussion about what you know, what level of offense is acceptable as a way of playing defense or doing espionage. Within the private sector, you're saying this is mostly done, again, sort of in the form of penetration testing, which is to say, Red Team, Blue team, like go through these sort of like attack scenarios, so that you learn the tactics and learn where your weaknesses are.
LC Absolutely. And, you know, if we look at, you know, if you look at, like the information that that company like fireeye publish on, specifically, the threat actors, and who the threat actors are targeting, largely, they go after private sector, right, like, you know, for sure there's government activity, but it's largely going after a major intellectual property, you know, significant technological advancements that may be happening. So there's definitely a lot of I think, importance from the private sector perspective, to look at what these threat actors are doing, and try and emulate their tactics as much as possible in order to secure things better.
BP And so red team is always the aggressor and blue team, the defender, this is like a white hat, black hat thing?
LC Yeah, that's the typical paradigm.
BP And so security seems like a really interesting sort of like subset of being a developer, my understanding is that it's actually even more in demand and well compensated than the rest of engineering, which is, you know, quite a lucrative profession quite often. Is there an additional sort of tax on that in the beginning, where you have to be educated or get a security clearance or work with government? Like, what's stopping the demand for this particular sector from being flushed the way it is, for other areas of software?
LC It's a good question, and one that I'm you know, not 100% sure, that I even know an answer for because it's something that I think it is lucrative, like, you're right, like, it's something that's in demand, specifically within the government, you know, having a developer that can talk security, and really like gets it is a skill set that I think we highly value. Because I think a lot of times, we feel like, you know, we sort of silo things and have our engineering group, our infrastructure engineers, and you know, our security folks will talk back and forth on security things, and they leave the developers out of the conversation, you know, and I don't know culturally where that stems from. But I think having a developer that that could develop that skill set is definitely something that we would highly value within the department.
BP Yeah, it's interesting. I mean, what you were saying before about, you know, the sort of the idea of blockers, right, like where it's like, don't use this language, or don't build it this way. And yeah, a lot of developers usually are under a deadline, you know, and they're doing sprints. And so right, they want security to be the afterthought, you know, after they've delivered the product, then it's your problem.
LC Absolutely. It's funny. So, in one of my previous roles, I oversaw a program that included a development shop. And one of the things we did is we installed like code gates for security events in our coding, CI/CD pipeline. And it was fantastic to have conversations with our developers on, you know, setting deadlines, and then adding in security at the very end, and like how that impacted our deadlines, and how that could set things back. And so I think figuring out how to better bake in security and secure coding practices earlier on it not only like secures the product, but also helps meet deadlines, because you know, when you're scoping these deadlines and working in sprint's you can actually determine this is how long, you know, ticket is really going to take.
BP Right. And so what is the gate? And that for people that don't know, like, what does a gate mean, in that sort of CI/CD pipeline?
LC Yeah, absolutely. So we use things like SonarQube, essentially, to figure out like code quality, and, you know, essentially flagging on, you know, bad coding practices, like, "Hey, you used MD five to hash something, instead of, you know, Sha 256, or 512." Right, you know, and then flagging on that and sort of determining from a heuristic space, what kind of score we should assign that code from a security perspective, like, how many of these interests how many of these like insecure coding practices did you implement in your code? And how can we get it to be better? Right?
BP Yeah. Is it strongly typed? Is it strongly secured? I mean, if you just make it easy for people, you know, to have a checklist up front, then it takes less time to revise.
LC Exactly.
BP So what about from a personal perspective? You know, I think most people have seen Snowden. There's that classic scene where he's putting the tape over his girlfriend's webcam, like, in your life? Are there things that you do that other people don't? Because you live in this world? Or do you think maybe it's more about sort of your mindset, you know, the way you would think about giving permissions to an app or something like that?
LC Yeah, absolutely. It does change your mindset. I think, from a personal perspective, I'm a lot more cautious about my PII, personally identifiable information, you know, when you get a credit card or home loan or something, and, you know, you're asked for your social security number, that, you know, I'm a lot more inclined to ask why and sort of figure out, you know, are there alternative ways that I can identify myself to you that don't involve giving up you know, the thing that we have decided most uniquely identifies us as American citizens? Yeah, so yeah, absolutely. And you know, webcam covers, you know, once you start assuming that things like assume breach happen, you just start treating everything from an untrusted perspective. It's not the worst way to live, but but also can be a little bit kooky, I think.
BP I guess yeah. I mean, that's, that's for you to tell me, you know, like, a healthy level of paranoia is obviously useful. You know, if you plan ahead before the pandemic, now you're patting yourself on the back. But yeah, you don't want to feel anxious all the time. So I guess, is there do you do something almost more outside of the technology, whether that's I don't know, meditation or reading but like, what do you do to relax? You know, if this stuff is stressing you out? How do you, how do you disconnect?
LC So I'm a big time gamer, you know, and so I spend a lot of time gaming. And sometimes that's relaxing, right. But that's, I think that's primarily my major way of destressing, you know, and then getting out in nature. I found, you know, pre Coronavirus, and you know, even now a little bit, but getting out in nature and sort of like, unplugging from everything for a little while is always really nice.
[MUSIC]
BP At the end of every episode, I usually do this thing where I read a lifeboat, which is a question on Stack Overflow that was asked, that had gotten down voted to a score of negative three or more. And then somebody came in and gave an answer and got it up to a score of 20 or more. And so they get like a little lifeboat badge. But usually when I have a guest who's coming from a particular world, I check in on the Stack Exchanges. So let's see what's going on on our information security Stack Exchange this week, "expose my browser cookie, with my header request. I exposed my browser cookie of my request header in a web form. After I realized what I had done, I logged out cleared my browser and logged in again, am I safe now?" Alright, we're gonna have to leave people to find out. I'm not going to answer the question. You have to read the show notes to find out but there'll be something to learn in there for everybody. Terrific. Well, Lance, thank you so much for coming on. Thank you for spending the time. If people want to learn more about you, is there a place they can find you online?
LC Absolutely. So if any of this sounds sounds really cool, and you want to bring your development chops to come do a nerd tour of duty with the DOD, you can check us dds.mil. And if it's really interesting to you click on the 'Join us now' to apply. Or if you want to check us out on Twitter, we're @DefenseDigital.
BP Very cool. Alright. I'm Ben Popper, Director of Content here at Stack Overflow. You can always find me on Twitter @BenPopper and you can email us podcast@stackoverflow.com
[OUTRO MUSIC]