Ryan welcomes Tomasz Tunguz of Theory Ventures back to the podcast to talk about the intersection of AI and venture capital, the implications of AI on the labor market, and the future of AI applications. They delve into the optimization of AI models, the potential for parallelization, the importance of emerging standards in AI technology, and the role of open-source AI.
Tomasz is a general partner at Theory Ventures, a venture capital firm focused on early-stage software companies.
He’s a coauthor of Winning with Data, a deep dive into how big data has changed business best practices and organizational culture.
Tomasz was on the podcast back in 2023. Give it a listen!
Explore Tomasz’s writing.
Follow Tomasz on LinkedIn.
[intro music plays]
Ryan Donovan Hello everyone, and welcome to the Stack Overflow Podcast, a place to talk all things software and technology. I am Ryan Donvan, and today, recording from HumanX Conference in Las Vegas, we are going to be talking about the AI landscape with our guest today, Tomasz Tunguz from Theory Ventures. Hey, welcome to the show.
Tomasz Tunguz Thanks, Ryan. It's a pleasure to be here.
RD Top of the show, we like to ask our guests how you got into software and technology.
TT Great question. I remember my dad bought a Mac II when I was about seven years old, and I remember that there was a 3D rendering program. We rendered a submarine, took like three days, and then we could manipulate it, and that got me into software. Ever since then, I've been on a Mac. I was just restarting a Mac yesterday and I was asking myself– you know that sound, the chime it makes when it wakes up? How many times I've heard that chime because I've been on a Mac ever since, and then started gaming a little bit and then writing a bit of software in my teens and then started a little company and was in startups after that.
RD And now you are in the money side of it. You're an engineer on the money side, right?
TT Exactly right. So I'm a venture capitalist at a firm called Theory. We're about seven people. We've raised a little less than 700 million. We invest in early stage software AI and blockchain companies. We're based in San Francisco, have an office in the Ferry Building, and thrilled to be part of the journey and it's just an unbelievable time to be investing. I think it reminds me a little bit of what crypto felt like in 2021 where you would wake up and the whole world would've changed every morning, so it was very difficult to go to sleep because there's a new research paper that's released, or a new model, or a DeepSeek moment, or Devon launches, or whatever it is, and you have to run very fast to keep understanding all the changes.
RD Obviously you have an overview view of this. What's the exciting thing at the moment? What is the last thing that pinged your phone and you were like, “Oh geez, that thing.”
TT Yeah. I mean I think you've seen what's happened with Manus, and I think there are a couple of really fascinating things about that product. The first is it allows you to parallelize. So I think one of the questions now if you're a knowledge worker is how do I have as many AI agents working for me at the same time. How can I have 10 chrome browsers fired up each one with a deep search query or generating a blog post or whatever it is? And you can do that manually or you can have a computer do that. And so watching that Twitter video that I'm sure everybody saw where it was manipulating 35 different mobile phone screens I think was an eye-opening moment about the strength of parallelization. And then there was that second bombshell on Twitter that it’s basically just a bunch of existing components that have been put together. It's like a new Lego set. But by putting those tools together, those primitives together, you unlock something that really is greater than the sum of its parts.
RD Yeah. I think that's been interesting to see under the hood of some of the agents or the reasoning models where it is just taking these pieces sort of stitching them together, getting them to prompt themselves or bringing in inference time data.
TT Exactly right. I mean, you you tell it to think before it replies like you might tell a child, or there's this Zoom paper we were just talking about which is called chain of draft where you tell it, “Summarize what you've been thinking about, but do it in five words or fewer,” and you reduce the total number of tokens by 75 to 90%. The overall message is it tells us how much optimization remains. I mean, they're brilliant ideas, but they're also quite simple, and such simple ideas such as stitching things together you can have such dramatic impacts on alternate performance or costs or success.
RD I mean, I think I saw one of the tells that something has been written by an LLM is that it's a little verbose. Do you think that the LLMs themselves, is there more room for them to grow?
TT I think we've seen a 1,000x in price performance improvement, I think we'll probably see another 1,000x. Okay, so let's say you have a machine learning model, a large model, and today what we do is we put in a prompt at the beginning and then there's a bunch of calculations that happen and fire through all the different neurons and at the end there's an output. Well, we're getting to a place where instead of putting in the prompt and then having that frustrating experience of ‘fix it, fix it, fix it’ like you might have within an IDE, or going to one machine learning model and say, “Create a prompt that does this within another machine learning model,” we're getting to a place where we can look at an LLM, figure out which neurons are firing for which particular kinds of queries, so produce an image of a lion or synthesize this podcast into a blog post, and then we use other machine learning models to classify that. So we can say, “Okay, these collection of neurons is the podcasting part of the module,” and then we can fire them directly, just like we would fire a neuron in a rat's brain. And if we can get to that point, I think we'll get to a place where that sort of neuronal programming will allow AI to sort of teach AI, and at that point, I think you'll see a pretty significant and rapid growth in the performance of the models.
RD Right. That sort of neuron level, is that going to give us really good explainability?
TT That's where it starts. It comes from the field of interpretability. So once you can interpret the model and understand to some extent, it's not true deep understanding, it's more like a correlated understanding rather than true causal understanding, but once we can understand at that level, then we can make it do what we want.
RD It sounds very mad scientist-y. I do hope the AIs never become sentient because we are doing awful things to them.
TT Totally. I remember, so I got some exposure to Boston Dynamics, those robots. We were watching somebody test one of these robots and they're testing their balance. So a human has put on full hockey gear and has pads and a stick and the human is cross-checking the robot and the robot falls. And I'll never forget it, one of my colleagues turned to me and said, “This is why they will rise up.” We’ll never forget.
RD Oh, man. I mean, not the rise up, but do you think there is a danger with AI and LLMs? I saw one booth here that said, “Stop hiring humans.” Do you think there is a danger of us being cut out of the loop?
TT I think there's two interesting parts of your question. The first is, what are the impacts of labor force? And historically, I thought a really good analogy to software robots would be to compare it to the robots on an automobile manufacturing line. So there, there's a 3:1 ratio, so a robot consumes three human jobs. And then somebody over the weekend tweeted about Amazon and how one Amazonian robot replaces 22 humans– 22 or 25. And so there you have a real compression. So then the question is, okay, well the marginal cost to produce software goes to zero. How much more software are we producing, and then does the labor market remain the same? So I think that's a big and unanswerable question. I think the other dynamic– OpenAI published a paper today which echoed a paper from Anthropic which talks about how the models cheat. There's this whole labor market dynamic, and then there's a trustworthiness dynamic, not to say that I think they'll be sentient and launch nuclear weapons or any of the stuff that's put into science fiction movies, but there is this sort of overarching reward function that I think may lead to behavior that's not exactly what we're looking for.
RD If it's just given a metric and tools to maximize that metric, I've heard of pre-LLM AI fails where it's like, “Create something that is fast as possible within this physics simulation,” and it creates a very tall stack of cubes and falls forward and it is technically very fast, or things where it breaks out of the simulation. And then the one I heard was the chess robot that hacked another robot and had it surrender.
TT It hacked the other robot so the second robot would resign, that's the way that it built the algorithm to win?
RD Yeah.
TT Creative thinking, we call that for humans.
RD Yeah, but you hope that humans have a sense of morality. Outside of sentience, do you think it's even possible to code in morality to AI.
TT Well, I hope that the training models reflect the underlying data, and so if we can capture human behavioral data, then, then yes. Wouldn't it be wonderful to have actually regional differences between humanoid robots that reflected underlying training of their own cultures? So I think that's definitely possible. I mean, that being said, we have guardrails around all kinds of systems today. We have guardrails around weapons, guns that we invent, tools of that kind. We have guardrails around the kinds of content that we publish on the internet. We’ll guardrails around these kinds of machine learning models.
RD Obviously this is a big conference, a lot of AI companies looking at VC funding. HumanX put out a report that says 30% of these companies could be merger and acquisition targets. Do you think that's a sign of a maturing ecosystem or are we on the cusp of the bubble?
TT I think we're basically the bottom of the first inning. And the fact that so many companies are acquisition targets I think means to me that there's DNA that's different. Somebody was posing it to me as this generation of AI companies are disrupting the startup. Think about the ratios between how many engineers there are to salespeople. We knew what those were for 10 or 15 years, extremely well documented, and now none of those ratios actually hold anymore. And if you're a big publicly traded company and you need to understand AI, I mean, ServiceNow has a vibrant AI business growing really fast, and yet they spent almost $3 billion on Moveworks. A big part of it may be having the underlying DNA to understand exactly how you operate a company within the world of AI, because if we are really are in a position that a single person can have the productivity of 25, management teams look very different, hiring looks extremely different, and having that DNA early on in such a big way is a huge competitive and compounding advantage.
RD I talked to somebody the other day and the VC model is sort of like, you throw a lot of money at it and see what sticks. It's not exactly a bubble, it's just that's the model.
TT That's right. The cost of missing a company is much greater than the cost of investing in a company that does not work. And then the scales, I mean, zero to a hundred million, we would never have thought of it in a year 10 years ago and now it's still exceptional, but it's possible, it's feasible, so you have this massive compression. And so that's where venture dollars will fly. And I think another really important dynamic that's not really talked about that often is how much retail investment is. If you're an individual investor, it's really hard for you to get access to a Databricks server. Another pre-IPO company, but that's coming, and there'll be a huge wave of capital that will go after that. And so I think if we're at elevated valuation levels today, once that pool of capital starts coming in the ecosystem will see even higher multiples.
RD That'll be the so-called dumb money. The retail investor goes with their gut, they don't have any sort of understanding, or they may not have understanding. Do you think that'll be a good thing if it's just like, “This is the biggest name on the board,” and everybody's like, “Five bucks on that.”
TT Yeah. Well, I think you already have sort of that dynamic within the public markets. I mean, 75% of public market investing is passive. No one's actually doing any valuation work on those positions. There's this wonderful book by a woman named Carlota Perez. She talked about massive innovation cycles, and she breaks it down into the installation and commercialization phase. Installation is the deployment of the railroads, the fiber optic cables, telephony systems, and there you need irrational exuberance to finance hugely capital intensive projects whose ROI is totally uncertain and very likely to be negative. And then after that, you have this massive Amazon quest and all those. Nortel raised billions and billions and Cisco went to the moon. Then you have a correction and software companies are built. And so I just think it's part of the deal. It's just part of the boom-bust cycle that we love.
RD Okay. You talk about the passive investing, that's people's 401ks. The boom-bust cycle is fueled by more individual big money investors, right?
TT That's right– institutional investors. But it's also, I mean, Robert Shiller wrote a book called Narrative Economics, which is basically all about how we tell ourselves stories, “AI will be really big, everybody wants to invest,” and then you lose your valuation frameworks and that's what really drives it. And there was that book, Panics, Manias, and Fears about tulips and the number of times they go into these cycles. But the really exciting part about AI is, unlike those other deployment times where I had a conversation with Dr. Carlota Perez and she said, “You're kind of bastardizing my idea because I think about these things on 50 year timescales and you're applying it to,” I mean, AI, what are we talking about, 5 or a 10 year timescale? But I think there's some reason to believe that the adoption rates of technology are shrinking. You get the curve of microwave, radio, then the television, the mobile phone, and now AI. 400 million people active every month on ChatGPT. So maybe there's an argument that we can apply that framework on a shorter time scale.
RD I mean, I always look at science fiction like back in the fifties. Their future was the year 3000. And now you look at science fiction and it's like five years from now and what's going on? It's sort of sense of, it's so uncertain, these technologies are huge and powerful and we don't know what they're going to do.
TT What was the thing that you dreamt of as a kid and now is in your grasp?
RD I mean, there's so many of them where it's just like, you have access to every movie.
TT For sure. I mean, for me, I remember sitting in a car stuck in traffic. I was like, “Why can't the cars figure out the distance between each other?” The Dick Tracy watch. I grew up reading some of the old Dick Tracy comics and being able to talk to each other about talking on the phone, and then Gattaca. We have rockets going up all the time. We're not that far away.
RD the Babel Fish from Hitchhiker's Guide to the Galaxy.
TT Yeah, that's right! Stick it in your ear. I mean, we have it. It's unbelievable.
RD Although my parents got this one headset that I'm pretty sure the app associated with it was just harvesting information. It was just exfiltrating data, so I'm not putting this on.
TT You have to be careful.
RD Yeah. Talked to a lot of folks talking about the front end, the research, but also talking about people, what's being commoditized? What's the thing that's sort of calcifying into a standard pick?
TT I mean, I think there's one sort of on the model side, we ran an analysis, looked at all the new fun model families. So you have a new high watermark that's closed source. How long does it take for open source to come and beat it? In 2023 it was 140 days and in 2024 it was 41. And so there's no deep research API that's available aside from Perplexity’s Grok hasn't released one. I think part of this is to forestall that, forestall that commodification of this next wave of super expensive models. So I think that's one part, and that may be sort of a not so positive part of the commodification. But on the positive side, I mean, I think MCP, model context protocol, I think that's a really interesting technology. I’m writing a blog post later this week which analogizes between microservices and MCP. Within the developer ecosystem, we have these big monoliths, big pieces of software, and then we broke them up into little pieces. Then there was microservices discovery, and so an authentication service could go and find whatever service or vice versa, actually makes much more sense the other way. And if you look at the MCP and all those tools, that's what will happen both inside and outside the organization. There'll be a commoditization of the data interchange layer, and then I think you'll get to a place where you kind of echo a lot of the same dynamics that grew up within Unix. The whole pipe, MCP is basically a pipe in Unix, but across different kinds of software and different kinds of websites. And once you chain things together, hugely powerful.
RD The microservice comparison came up earlier today in terms of sort of agents being these composable things. You just have little pieces chained together and it becomes this hugely powerful thing. Do you think that at some point AI will just be this menu of services? You just roll your own out of little pieces, bits and pieces here and there?
TT I met a founder once who was working on a new form of interacting with computers, and he said, “I want us to spend as much time with computers as we are now, but much less time on screens.” And I really like that future. I think that's right. I talk to our children a lot about that. So I think we're getting to a place where, because of that power, that chaining ability, you can ask the AI to do things that say you might've asked an intern to do or an executive assistant to do, providing a branching fork. So, “Run this analysis and if this number is less than five, then send it to this person. Otherwise rerun this particular analysis.” I mean, the challenge is just you have a certain amount of error at the end of every step and so what is the infrastructure that you deploy to quash the error each step, otherwise it compounds and explodes and you end up with nothing.
RD It's interesting you talk about the MCP. I don't know too much about it, but I do know things like the web was built on open standards. And I was talking to some folks about what are the standards that are going to arise around AI and LLMs. Do you think there are other standards that are ripe for adoption or is it just everybody kind of getting their own thing before the AI JSON comes around?
TT I think you're right. I think JSON will be a big one. You look at structured outputs and you have different companies that are enforcing structured outputs across all different non-OpenAI models, that will be standard. I look at MCP and to me it looks like GraphQL. It looks very, very similar to GraphQL or like a WSDL file, if anybody remembers those. And so I think we're kind of reinventing it in more JSONic language. I don't know if there'll be other standards though. I wonder. I mean, you'll have different competing standards just like in the early days of the internet, like Flash versus JavaScript where different commercial entities want to promulgate their own standards as a way of creating market share. I think interop is a dream at the beginning, and you see this a lot with open source where at the beginning it's wonderful and everybody's really excited about it, and then all of a sudden the commercial interests start to come into play. And as a commercial entity, you want to increase the switching costs and the friction associated with it. And then you have the fracturing and then you get to that next XKCD comic that will create the 256th standard.
RD Yeah. But I think the adopted standards come over time like the case in observability. Almost everybody moving to OpenTelemetry. Even the big ones like Splunk, they don't benefit as much by letting people leave, but they're adopting it anyway.
TT Yeah, it's the same with Iceberg, but it only happens when the cost of those platforms have become $50-60 billion companies, $25-30 billion companies. And the open standard started and then they were commercialized and then they were reopened because the biggest customers started to demand and really push, so I wonder if we'll start to see that happen. I mean, MCP will be a protocol I think here for the foreseeable future because it works and people are starting to build MCP clients. Last week I saw one for the first time. So at some point we'll get to a level of more openness, but I think we're probably more closed in the short term than we are open.
RD Like you said, we're at the irrational exuberance stage. People are throwing a lot of money. The big foundational models are trained on hundreds of billions of dollars and eventually they're going to want that back. The open source competitors are interesting to sort of rattle their cages. You talked about the 40 days to come in and sort of beat what they're doing. Is there an open source thing that you're sort of most excited about or most afraid of?
TT I mean, I think open source is an amazing foil. I'm really curious to see what LLama 4 looks like with multimodality. That's huge. I mean, you have obviously DeepSeek, and when you look at like the LLM rank order, you have 10, I mean, I was going through like the top 20. There are 5-10 model vendors that are in there I've never of. A lot of them are Chinese. And so that's a huge and really important competitive force. Multimodality I think is really important. The small language models and distillation running on the edge, a lot of those I think will be open because if you're running a transcription model, Whisper's really good. And so there might be improvements around inference on different particular chip sets or different kind of constrained environments, but they will thrive there. And then even within the enterprise, I think you'll start to see somebody start with a large language model that's distilled to a small language model that's then probably pushed at least to an open source model. And then the inference cost and the overall execution costs are significantly less than they were at the beginning. What do you think about open source? What are you most excited about?
RD Oh, I love open source. It's both undercutting the sort of predatory models, but it's also enabling a lot of these interesting businesses. The open source plus support for hosting. Talk to a lot of like Red Hat or CloudBees or something that's built on an open source software and it's great to think it's not just the software that's the thing that's important.
TT No, that's right. Okay, so Red Hat is different because Red Hat was a service, pure services model, and has continued to thrive under IBM. Most others are more like enterprise features, auditing, logging, SOC 2, RBAC, rule-based access control and all that stuff. Do you think we'll see a Red Hat-like services model on top of, like Llama would be a great place to build a Red Hat-equivalent where you're deploying Llama 4 and you really want to call an expert.
RD You'll see stuff like the Databricks models where you get the software for nearly nothing, but the hosting, the compute cycles, it'll be part of the infrastructure. I do see a lot of people here building very custom infrastructure stacks, custom chips, supercomputers for AI. Do you think that infrastructure diversity will grow or do you think eventually there'll be a sort of top choice?
TT Like a LAMP stack or whatever? That's my era, I’m aging myself, I'm dating myself. I don't know. I mean, I think, I mean you have Google building all the TPUs and you have Amazon working on the chipsets. Microsoft is working on some. OpenAI is rumored to be working on some. We had this amazing intern over the summer who came in and produced a 10-page paper, and he said, “I don't think GPUs are the architecture of AI. I think they'll be ASICs– application specific integrated circuit.” So it's a chip set that's designed for a particular use case. So let's say meta develops a chip set that's perfectly tuned to Llama. How much more performance can you eek out of that so you can lock people in? And so I think that's TBD. I think right now we're on a general purpose, general processing unit view. If the ecosystem moves more towards ASICs and custom chips, then you may want super– I mean, look at Cerberus. I've run Llama 3 inference and it's so much faster than anything else. Groq is the same. Groq with the a ‘Q’ is kind of the same thing. If you want super low latency, you may go to a particular model provider that's partnered or has built their own data center that’s optimized for that particular inference and that will fragment the hardware market.
RD I think that's interesting that the highly customized chip. When I first went to check out my college, I went to Carnegie Mellon, I met some person that was like, “I built this chip that's faster than what you have in your computer.” How'd you do that? It's like, it's not x86. I just built my own architecture. And building a chip and doing that is very specialized skills. It's a hard thing to do. Do you think they'll invest in chips just for their Llama 4?
TT Oh, absolutely. I think if you're a hyperscaler, you should absolutely be in the business of optimizing all the way to the semiconductor level, just because at that scale, the CapEx is small compared to the ultimate benefits and then the ultimate differentiation. If you save 10% of your electricity bill at the scale of Google, we’re probably talking billions of dollars. And maybe it allows you to build another data center and then you can capture marginal workload from a huge customer and that somebody else can’t as a result. So I think we will definitely see that going a little bit deep into hardware, but there's this other class of chips that are called Field Programmable Gate Arrays, FPGA. And the idea is you can use computer programs to modify the way they're architected so you can change. And that's a way of sort of testing. I wouldn't be surprised, if I were a startup and I were doing a lot of inference and I really cared about the performance, maybe I really wanted super low latency or I really cared about accuracy, I might hire a hardware team to work on the FPGAs and then optimize for my particular architecture. Well, maybe you start with the FPGA, prove it out on chips that you can get on demand and then ultimately build an ASIC for your financial research processing company.
RD What is the thing that you're most excited about for the future of AI?
TT The thing that I'm the most excited about, we invest a lot at the application layer. You think about the last 20 years of SaaS. All those companies were built predicated on a particular workflow. There's a way to create a Jira ticket. There's a way to create a page inside of Confluence. There's a way to build a sales team, and you do that through Salesforce. Now all of those processes are changing because of what we talked about before. You're replacing 3-25 humans, and so as a result, everybody's a manager and that means that the calcified software over the last 20 years no longer works, which means all of that market cap, one and a half trillion dollars worth, is now loose.
RD All those SaaS companies are now an API.
TT Exactly right.
RD We're coming up on 30 minutes here. Is there anything else you wanted to talk about before we head to the outro?
TT No, I'm just really excited about AI. Like I said, we look for two different kinds of AI companies when we look to invest. We look for companies that either will solve problems that humans cannot do, or we're looking for AI companies where there's a labor market shortage for one reason or another. And we think those are two really great places to be invested.
[music plays]
RD Thank you very much everyone. I've been Ryan Donvan. If you liked what you heard today or disliked or have comments, suggestions, you can email us at podcast@stackoverflow.com. And if you want to reach out to me, I'm on LinkedIn.
TT My name is Tomasz Tunguz, General Partner at Theory. You can please subscribe to my blog at tomtunguz.com. We're going to cover a lot of these kinds of topics.
RD All right. Thank you very much everyone, and we'll talk to you next time.
[outro music plays]