Ben and Ryan chat with Craig McLuckie, cofounder of the Kubernetes project and cofounder/CEO of Stacklok, which helps developers and open-source communities build safer, more secure software.
Stacklok helps developers and open-source communities build safer software, secure the supply chain, and choose safer dependencies. Trusty is their free-to-use service that employs a statistical analysis of author/repo activity and a package’s source of origin to assess its trustworthiness.
Craig cofounded the Kubernetes project, an open-source system for automating deployment, scaling, and management of containerized applications.
He is also the former VP of Research and Development at VMWare.
Follow Craig on LinkedIn.
Congrats to Stack Overflow user netcorefan, who earned a Lifeboat badge with their answer to Need a workaround to access ReadOnlySpan
[intro music plays]
Ben Popper Are you a software engineer? Do you have experience with mobile and web development, cloud, Java or Python? Make an impact at one of the world's most global banks at Citi and explore open roles at jobs.citi.com/tech.
BP Hello, everybody. Welcome back to the Stack Overflow Podcast, a place to talk all things software and technology. I am Ben Popper, Director of Content here at Stack Overflow, joined by my partner in crime, Editor of our blog and maestro of our newsletter, Ryan Donovan. Hey, Ryan.
Ryan Donovan Hey, Ben.
BP So Ryan, have you ever heard of this thing called Kubernetes?
RD I’ve heard it mentioned a couple times.
BP I think we've written about it on the blog a few times. We may have talked about using or trying to use it in different ways at Stack Overflow. It has become an enormous part of what people work with in the software ecosystem, and we are lucky today to have Craig McLuckie, who is one of the co-founders of Kubernetes, on the program. We're going to chat a little bit about some of the old stuff, but also about a new company he's working on called Stacklok and some of what that's hoping to do in the world of the software supply chain. So Craig, welcome to the podcast.
Craig McLuckie Hey, thanks so much for having me on.
BP So we always ask folks, for listeners who aren't familiar, just give a quick background. How did you get into the world of software and technology and what led you to play a role in the creation of something that now is touched by so many developers around the world?
CM It's probably the same story that a lot of the listeners have. It started with, in my case, a Commodore-64 when I was a kid and the rest is history. I've always been drawn to technology and computers. I was very privileged to get a job at Microsoft straight out of school and so I learned a lot about building software as an individual contributor and later as a manager. But it turns out I'm actually not that good of a developer. I'd love to be, but I'm just not. But the thing that I do reasonably well is help empower the people to produce things. And so I was always drawn more towards helping and supporting than I was to actually directly executing, in a large part because I'm just not that good at coding, and that led me to product management and I had the opportunity to work at Google. And I'd been at Microsoft for quite some time and I was very attracted to this idea of cloud native. I built a lot of package software, I got to experience that, but I felt that the future was really cloud. And the first job I had was this opportunity with this guy, Joe Beda, on this thing that became Google Compute Engine, which obviously worked out okay– Google has done really well in the cloud space on the back of that offering. But just that opportunity to bring more of what we learned while building Compute Engine and the practical capabilities that Google had inside the company to the outside world to disrupt was just incredibly tantalizing. And so I felt very privileged to have an opportunity to work with some of the greatest engineers I've ever met on what became Kubernetes.
BP When the project internally to work on Kubernetes began, what was the mission? Was there like, “Oh, this is the pain point we're solving,” or “This is the market opportunity we see,” or “This is the thing all of our engineers are talking about and they believe we should go this way for this reason?”
CM I think it depends on who you ask. So I was the business guy, and for me, in some ways Kubernetes was almost an act of desperation. We built Compute Engine and it was very good technology. We built some really cool stuff: paravirtualized device stack, we built a global control plane, we built an overlaid network, we built this killer block device, really good tech. But because I'm a business guy, I recognized we just didn't have a route to market; we couldn't sell effectively into enterprise. And whenever you're building a business, there's two sides to it. Technology is important, but actually making money really matters. And so I saw this opportunity with Kubernetes as a way to effectively intermediate a competition with the technology that I thought we at Google could just run better, because our data centers were already built this way, and so my ambition was really motivated in the business sense. If you ask someone like Brendan, who was the creative genius on the project, he would say it was an opportunity to create something really special in the world. If you ask someone like Joe, I think he was just desperate to work around some of the structures and the things that were holding him back from producing great technology and he saw open source as this incredible opportunity to just bring his talent and wisdom to a problem space. So I think everyone had a different cut of it, but for me it was very business aligned.
RD Obviously Ben joked at the beginning if I'd heard of Kubernetes. It's become this sort of foundational cloud technology. Why do you think it's taken off so much?
CM I would say there's a couple of reasons. There's the technical reason it's taken off so well, which is that there never was a principled API that abstracted away infrastructure. It's actually a programmable surface area that lets you take your workload and deploy it into an infrastructure domain, and that just never really existed. There'd been attempts at it, but they'd always been tied to something specific about a specific environment and no one could really come together and agree on what that looked like. And I think the thing that was kind of novel was that we were able to do this because I didn't need to control that project, we didn't need to control it, Google didn't need to control it. It just needed to exist and we needed to be able to run it better than anyone for us to really succeed, and so the incentives were aligned with building something that became ubiquitous and widely used. And I think it became as ubiquitous as it was because we were able to bring in the very best of a lot of different places. Google had a very specific set of skills, not to quote that infamous movie, that they've built over many years but contains those certain skills, whereas Red Hat had a different set of skills, Microsoft had a different set of skills, Docker had a different set of skills, and being able to tap into and unlock this potential where each organization was able to bring their own unique capabilities, their unique perspective, and we were able to create an organization or an entity that was really invested in just the success of this writ large. The people that worked on that project early on didn't identify with the companies they worked for, they identified with the project they were working on. If you'd ask someone like Tim Hockin to do something that was going to bring Kubernetes down but lift Google up, he probably would have told you to talk to the hand. He had a very high level of personal integrity and wanted to produce something very special. I don't know that to be true, but I believe that to be true. Same with Brian, same with Brendan, same with Joe, same with Dawn, any of those early engineers were really invested in the project. Tate and Korren from Red Hat, same dynamic. And they cared and they loved it and they brought everything they had and they produced something beautiful. And it didn't belong to anyone, it belonged to the community, and I think that's what made it ubiquitous.
BP So just to catch us up a little before we get into your latest thing– you worked on Kubernetes and then you left Google in 2016 along with Joe Beda and you launched something called Heptio. Can you talk to us a little bit about what that was?
CM So one of the things that we observed was that a lot of enterprise organizations saw Kubernetes as this gateway to self reliance as a way to effectively decouple themselves from the environments that they were working in; that they could legitimately and authentically create a substrate that spanned multiple clouds, and almost every enterprise at the time was trying to figure out what their multi-cloud strategy was. And so we had this idea that if we could step into the space of filling the gap between enterprise organizations and this open source technology that we would be able to create further value in the world. And we did that initially by just helping folks that had a self service Kubernetes program succeed with services and with support capabilities, but over time we also used that to identify where the critical gaps were, like that it was hard to back up and recover Kubernetes clusters, setting up something that would operate at internet scale from an ingress control perspective is really hard, qualifying a Kubernetes environment and making sure that its upstream conformant was hard. And so we sponsored a series of small projects that each developed a life of their own that filled a very specific gap in the ecosystem. And unfortunately, or fortunately, depending on which way you look at it, we were only in business for two years. It became clear that we had this opportunity to join forces with VM which was really looking to redefine their narrative to achieve relevance in this multi-cloud world. And so I just saw this as an amazing opportunity to join forces with someone like Pat Gelsinger and Raghu and expand the mission very, very quickly and I felt really good about that. We made some good inroads there.
RD So obviously, like you said, Kubernetes was this sort of cross-company open source project, and I think a lot of the big technologies that are taking off are these open source projects. Do you think that open source has an advantage to closed source proprietary projects?
CM Well, it depends on when. It does have an advantage at a certain moment, but that advantage tends to thin out later. The way I tend to think about it is it's activation energy and then there's value extraction. So if you look at any commercial entity, the activation energy associated with an open source project is relatively low. It's easier to create an open source project that people pay attention to, that they engage with, that they contribute to, that they start to use, and you sort of generate a snowball effect where that open source project becomes ubiquitous. So open source represents the most practical and simple path to ubiquity of any of the technology types that I've worked on. It also represents the most formidable challenge from a value extraction perspective. If you think about the rake, the proportion of value that you can take out of an ecosystem, with open source it's capped because by building in open source, you create an opportunity for someone else to use your R&D investment and redeploy it into a different context. And I certainly see that cloud providers have done an amazing job of this. The cloud services are fantastically profitable expressions of open source technologies with relatively low internal R&D costs, and so that's the challenge associated with it. Has open source run its course because the cloud providers are able to effectively monetize? No. I think it's a very, very important part of the technology landscape. In fact, I think it's probably the basis, the cumulative sort of open source technology. Whether it has a commercial entity behind it or whether it's just being maintained by people that are passionate about the technology represents one of the greatest treasures of humanity. If you look at software, which powers all technology, 80 to 90 percent of software is just a reconstitution of open source pieces. So it is a formidably critical part of our technology landscape, but it's also an increasingly hard part of the landscape to monetize, because unless you have a practical SaaS based service and you are able to establish barriers to entry, you will see other people profiting from your contributions.
BP There's an old saying in the world of finance, which is that it's better to be the market maker than the trader. If you find yourself at the heart of the ecosystem, it seems like there are perhaps benefits to your company, even if they're not purely revenue: the amount of talent you can attract and your centrality to the ecosystem. And so all those things provide a lot of value to an organization that succeeds in open source, even though as you point out, maybe the raw revenue is capped or something like that.
CM Look, let's be clear. There's fantastically successful open source companies out there that continue to be very successful in the space. And I'm certainly not done with open source– very far from it. I'm very attracted to this idea of community centricity. I think if you want to tackle a really, really big problem, it's probably going to take a village and you're probably not going to be able to do it alone, and collaborative community-centric open source represents the very best possible way to tackle things that no single vendor can do. If you look at the work that I'm doing with Stacklok right now, my co-founder was the founder of this project called Sigstore, which is emerging as a critical way to tether an open source package back to its proof of origin, the repository that it was generated in. And I think a lot of different vendors may approach this ecosystem and try to solve it for themselves, but they would have failed. They would have had something that worked in a certain context, but they would never have got that groundswell of support. And I do see something like Sigstore, which is obviously important to us to support, and that's something that is a pretty key anchor to the work that Stacklok is doing as we tackle the supply chain problem, it wouldn't have happened if it were not a community-centric open source. So that's something we will support I imagine indefinitely. We're very appreciative of the work of Red Hat and Google and we'll continue to see where it goes.
RD I think open source always seems like the best middle ground between build versus buy. You can buy this thing that solves the problem you want to solve, but you can also contribute to it, make fixes, add features.
CM And you have optionality on the backend. It gives you optionality, which is that if you don't like your vendor, you're not locked in, and that really matters. My general view here is that there's things that have to be open source to achieve adoption. Kubernetes would never have happened if it wasn't open source and it wasn't community open source, it just wouldn't have. I think Terraform wouldn't have happened if it wasn't open source. I think HashiCorp did a beautiful job of creating single vendor open source narrative. They were seen as the purveyor of that. If you wanted a commercial version of it, by and large you’d work with them. I don't know if any of the other versions had much commercial success, but anyway.
BP I guess what I would say is that obviously there have been some big names out there where a huge ecosystem grew up around it, and in fact in some cases a business, but as you point out, there can also be a struggle to make that business profitable or to continue to grow it in a way.
CM Absolutely. The point being that there’s different approaches to achieve different outcomes, and you have to accept that the activation energy benefits sometimes mean value extraction deficits on the back end, and it's just a balancing act.
[music plays]
BP All right, everybody. Thank you so much for listening. It is that time of the show. Let's shout out someone who came on Stack Overflow and shared a little knowledge. Congrats to netcorefan for being awarded a Lifeboat Badge for helping to save a question, “I need a workaround to access ReadOnlySpan
RD I'm Ryan Donovan. I edit the blog here at Stack Overflow; you can find it at stackoverflow.blog. And you can find me on X @RThorDonovan.
CM I'm Craig McLuckie, the founder and CEO of Stacklok. You can find me on X @CMcLuck. And I invite you to come check us out at stacklok.com and kick the tires on some of these open source and free to use SaaS offerings that we've built.
BP Wonderful. All right, everybody. Thanks for listening, and we will talk to you soon.
[outro music plays]