Ben welcomes back friend of the show Eliot Horowitz, cofounder and CTO of Viam, who’s also the cofounder and former CTO of MongoDB. They talk about the current status of robot assistants, why Viam is hardware-agnostic, and building robots to train cats (good luck with that).
Listen to our previous episodes with Eliot here and here.
Viam is a software platform for building, monitoring, and managing data from smart machines, including industrial robots, autonomous vehicles, smart home appliances, and IoT devices. Get an overview of the Viam platform or dig into their docs.
Connect with Eliot on LinkedIn.
Three cheers for Stack Overflow user mattl, who won a Great Question badge with How to remove all contents of a directory using Golang?.
[intro music plays]
Ben Popper Big topics in data architecture call for big conversations. Big Ideas in App Architecture, the new podcast from Cockroach Labs, invites innovators to discuss their experiences building reliable, scalable, maintainable systems. Visit cockroachlabs.com/stackoverflow to listen and subscribe. Make sure to use that link, and let them know the podcast sent you.
BP Hello, everybody. Welcome back to the Stack Overflow Podcast, a place to talk all things software and technology. I am Ben Popper, Director of Content here at Stack Overflow, excited today to have a guest on. We're going to chat a bit about robotics, we're going to chat a bit about AI, and we're going to talk about where software fits into all that. So I want to say welcome back to Eliot Horowitz, the CEO and founder over at Viam. Eliot, this is your third time on the podcast, maybe?
Eliot Horowitz I think that's right. And thanks for having me back for the third time, Ben.
BP You've got it. Your first time you were here to talk MongoDB where you were a co-founder and the CTO, and the second time was to talk about the launch of your own company, Viam. And at that time, it wasn't really out; I think we were inviting people into the beta. But now you've gone into general availability, so for folks who weren't here that first time, give us a quick refresher. What is Viam all about, and now that it's out to the world, what does that mean for folks who are interested in interacting with it?
EH So Viam is all about making it easier to build machines in the real world that interact with the real world in all sorts of ways. If you ask most software engineers to work on hardware, they kind of look at you funny and they're not that interested. If you ask most people about the devices in their lives that ought to be smart, whether it's anything from a coffee maker, to a thermostat, to a vacuum cleaner, to a cat food feeder, none of them are all that smart and none of them work together in interesting ways. You've got security cameras and sprinkler systems and you can't configure your sprinkler systems to turn off if your kids are playing in the backyard, or depending on your mood, to turn on when they're in the backyard. You might want them off or on depending on what you're going for. All these things are hard. The robotics space could do so much more interesting things in terms of anything from fixing potholes to better agriculture to better food in airports. No one loves the food in airports, but why can't we have fully automated kitchens in airports? And so Viam is a platform to make all these things easier– anything from very simple things like smart thermostats and smart cat food feeders all the way up through fully automated kitchens in airports. It's a platform that is easy for hardware people to work with that's fully extensible, easy for software people to work with, and does all the things that you need to actually go and build a product, build a business, and actually get things done. And now that we're GA, anyone can come in. If they've got an existing product, they can start using us and you don't have to migrate everything to Viam. It's very flexible and pluggable and you can just start using it for certain things that are problems, or if you want to go and build a brand new thing, it makes it an order of magnitude easier to go and just tinker and build something and put it out into the world.
BP So for folks who are listening who are not at all familiar with robotics or those who might be passingly familiar, what is the tech stack here? If you said to me, “You can make all the smart devices in your home smarter or you can even start coming up with your own inventions,” and I went to the Viam platform, would I need to know certain coding languages? Is this drag and drop? Does it work universally? How do I enter into your ecosystem?
EH Well, let's take an example. I've got a blog post on this also, so if you want to learn more, you can go read about it. I've got a cat and my cat likes to eat and be fed. And sometimes I'm away for a while; I’ve got a lot of work to do, apparently. So a while ago I bought a $60 cat food feeder, and it was okay. You put a bunch of food in –she eats wet food and dry food, this is for dry food– and it has some fundamental problems. It's got a little tiny LCD on the screen and to program it, you press the buttons, feed her a little bit at this time, at this time, at this time, but it doesn't adapt. If she's eating too little, it will just throw food out and kind of cause chaos. For some reason, the time drifts in weird ways. If I want to change her schedule, I have to go in and press all these buttons on the screen, which is kind of infuriating, and I have no way to monitor it at all. So one Saturday afternoon, me and my 11-year-old son, we basically opened it up, took out all the electronics, put in a Raspberry Pi and a Raspberry Pi HAT with a motor controller and a USB webcam, which took about half an hour of unscrewing things and drilling a couple of holes and just plugging things together. And then in about 30 minutes, we were able to connect the motor– there's a little motor that turns a thing that makes the food come out– to connect that to Viam via the Raspberry Pi, with the HAT it’s a couple of wires, and connect the USB webcam. And then so immediately I was then able to see her food bowl from anywhere in the world, I can turn the motor to feed her, and then we were also then able to start capturing images from this camera that we put on there, use Viam to train an ML model to determine the state of the bowl– whether it's empty, has a little bit of food or is full, and then we wrote about 30 lines of Python logic to determine if and when she should get fed. And so it has caps like, “Hey, we can feed her only this much per day because otherwise she'll just keep eating, pace it out through the day. If there's food in the bowl, don't put more in the bowl.” It's nice because everything about Viam is flexible, including languages. So you can use any language, whether it's Python or Go or C++ or Flutter. You can build mobile apps really easily. You can do this all on mobile if you want. It's completely flexible, it's all built on modern, standard tools like gRPC and WebRTC, and you can go build real things. This took an hour. I kept tweaking the Python code to make it a little bit smarter, so maybe a couple hours in total. And that was a couple of months ago and it feeds my cat. It's fed my cat every day for the last two months.
BP It's robust. But what about your cat, can your cat control this? I mean, did you think about the agency there at all? Can it decide if it's a little more, a little less?
EH We thought about it a lot. So far we've decided she has no agency, because if she had agency, we think she would do bad things. There's another guy in my office who's taking mine and extending it because he's got three cats who get different diets and he's going to do something a little bit smarter. But we thought about playing games with her. I also want to build a cat training robot that follows my cat around and if she misbehaves it squirts her with water or something. It feels a little mean so we haven't done that one yet, but it's on the docket. And so to answer your question, it's very easy just to do anything like this. I've done this for a security camera at my house. I ripped out my sprinkler system at my house because I was annoyed that it kept watering my lawn while it was raining outside, and I was like, “Well, that seems wasteful.” So it's all these things. And I do not have a hardware background; I have people at Viam who have taught me things, but I do not have a hardware background, and we're trying to make it easy for a hardware person and a software person to really do almost anything.
BP So you're sort of a robot maximalist here. You think they should be improving airports and fixing potholes. Listening to this I’m very intimidated. I would love to fix up some of the stuff in my home, but I'm not sure I’d be able to do it as sort of a non-coder. What's the response been like in terms of building a cohort of early users and a community so far?
EH So we've got a very nice cohort of early users and a community and everything on the machine that we write is open source. You can do whatever you want. We also have this thing called the registry that we can talk about more a little bit later, which is how other people can extend the platform for different motors and cameras or algorithms, that is also starting to build a nice ecosystem around it. So people can share code, share drivers, share algorithms in interesting ways. And then we're starting to get bigger companies starting to use us for various things also because we solve problems that they have. And I think a lot of people are, like you, intimidated by playing with hardware and things like this, including we have 10 to 20%, maybe 25% of our engineers at Viam are either very heavy robotics backgrounds or hardware backgrounds, robotics PhDs, or have a lot of robotics experience or device experience. The other 75% are “generic software engineers,” and almost every one of them, when they were interviewing, was like, “I don't know anything about hardware. How could I do this?” And we were sort of like, “That's the point. We actually want you to be working on this because we want you to build a system that you want to use and that you could use.” And we need the robotics PhDs to make sure we're getting everything right in some ways, but we want it to be like, “Hey, you know some Python code and you can plug some wires in and you can follow a tutorial? Cool, you can go do this stuff.” And there are 30 million software engineers in the world. Most of them sound exactly like you. They're like, “That sounds intimidating.” One of our goals is to really change that.
BP Cool. And so you said you have this platform that allows you to come with backgrounds across multiple coding languages, you have this registry, which we should talk about in a second that's going to make it extensible. On the hardware side itself, are you involved there at all? Do you sell parts or kits? Through a tutorial would I know what to get? Let's say I wanted to fix something up at home or build a sprinkler that moves around. Do you work with other folks to let people know how to get the hardware, or that part is where you sort of step back?
EH So first and foremost, we are almost entirely hardware agnostic. We will work on Raspberry Pis, on NVIDIA Jetsons, on Intel boards, on Mac laptops, we'll work on anything. And we want to work with any kind of camera, any kind of motor, any kind of arm, we don't care. We think there's a lot of great people working on hardware and we want to support all of it. For our tutorials, and we have a lot of tutorials on our website, for those we'll give a shopping list and be like, “Hey, here's what you should buy.” You can obviously do whatever you want and tweak it and change it, but we'll give you, “Hey, here's 10 things to buy and then you can go build this.” We do think it's important for people who don't have hardware lying around, who have never played with a robot that can move, for example, to be able to experience what that is and experience the platform very easily. So to do that, we've done two things, actually. One is on our website, you can actually go and take over a robot in our office. You can just go and take it over for free, you can drive it. It's got a little octagon that you can play in and you can just go take it over. You can just play with it on our website. There’s tools where you can just drive it around, or you can then go write code against it and try out the whole platform. In addition, we wanted there to be a pretty robust, pretty cheap kit for people to get a robot that can move around. We couldn't find anything on the market that we liked so we just built one. It's something you can buy from our website. It's 99 bucks, you buy your own computer. Basically we're selling it at cost, just as a way for people to start playing with things. It's not our business, it is not what we want to be doing, but if there are things that we can do in the hardware space to make it easier for people, we’ll probably do it. But mostly we are entirely hardware agnostic.
BP So you mentioned the registry and how that would allow other folks to bring in some of their ideas and maybe expand the ecosystem. So can you explain in just a little more detail how the modular registry works?
EH As a setback, everything in Viam is pluggable. So there are components– there's a motor component, there's a camera component. Those are pluggable, including mobile robots, and there's an API for those things. Everything's pluggable there. Same on the algorithm side, so computer vision, mapping and navigation, all of these things have their APIs. We have default implementations that you can think of as sort of textbook implementations or wrapping existing open source implementations. We think there is going to be both a lot of hardware that needs to be supported that we’re not going to be able to write the drivers for, and a lot of innovation in these spaces. So every one of those things is pluggable, whether it's building machine learning models, running machine learning models, everything's pluggable. So you've always been able to sort of write a plugin, but once you write a plugin, now how do you distribute it, how do you make it easy for other Viam users to consume that in their machines and their robots? And so the registry lets anyone write a module, you can open source it or make it closed source or make it commercial, and we’ll actually enable monetization for things if you want in that space. You write a module, say you open source it, then you can publish it on GitHub, then you can add it to the registry. And once you add it to the registry, it means that any Viam user can just put it in their configuration and say, “Cool, I want to use Ben's SLAM module, and I'm going to use version 3 or the latest stable version,” and then automatically it is deployed to their machine. We'll manage the deployment, we'll do these things, and you can do things like, “Hey, I want Ben's SLAM module to be deployed on all of the robots in my fleet.” You can do that in a template, and all these things just kind of work. So it's a way to make it really easy for people to share code, share things, and everything that we write that goes onto your devices is open source. Your code, other modules can be closed, we're not opinionated about what you do there, but all of our stuff is open source so it's very easy to contribute, very easy to work with, very easy to understand, very safe, very easy to vet all the security primitives.
BP From that, it sounds like what you would want, as you mentioned, is as new hardware comes online and you said you're not going to write the driver for everything, would be to have individuals, but also maybe research labs or collectives and then companies, contributing to this so that, as you said, there can be this universal place where anybody who's getting into robotics can connect all the different pieces they want. So I saw that another announcement was that you now have a partnership of sorts with Nvidia. Can you talk us through that a little bit and how that came to be and what is ultimately the goal there of working with them?
EH Yep. So Nvidia is interesting. They got into the space via the GPU market because obviously a lot of things in the true robotics space are using GPUs for machine learning inference, and obviously the Nvidia GPUs are very good at that. And so Nvidia has a line of single board computers and chips: NVIDIA Jetsons. You can think of Raspberry Pi as a cheaper entry point into the single board computer space, and then NVIDIA Jetsons have actually a very wide range of price points. And those are generally pretty nice because they've got a pretty high end –and certainly some of them have very high end– GPUs embedded. And so what we've done is make it really easy that if you want to deploy models to those things, we abstract away how you do the model inference. So whether you're running on any kind of device, you can run the models, and then it just so happens that on a Jetson with their GPUs, a lot of models are going to run considerably better because they've got really good GPUs and they're very fast. And so they want people to use their devices and they've got great devices for it and we want people to be able to deploy models and build models really easily and get them out into devices that can be made smarter with them, and so it's a great partnership with them and we can do very cool stuff together. And we're just making it very easy to sort of have a life cycle or a product that gets smarter and smarter over time.
BP Nice. So I think one of the things that got me super excited over the last few months was the reveal from folks in the R&D department at Alphabet or Google, that they had started layering large language models onto robotics as a way to interact with them and give them commands about the real world. So we've all been talking about ChatGPT and large language models, they have this incredible ability to speak with you in natural language and then also to some degree reason through things and have some sort of metacognitive abilities. So they used that approach with these robots and suddenly they could say, “Pick out the extinct animal off of this table full of all kinds of things,” and it grabs the dinosaur. They'd say more recently, “Can you put the apple between the bowl and the cup?” and it nails it in a way that it didn't before these techniques were used. Give me your thoughts on what's going on here. Is there something that is coming down the pipe in terms of combining the incredible capabilities of today's LLM with robotics, and where do you think that is headed?
EH So it already works. We've actually got some fun demos with ChatGPT and Viam where you can have your robot talk to you via ChatGPT or control the robot via ChatGPT. So all these things are actually pretty easy and I do think they're relatively important. If you've got a device in your house, let's take smart home. There's obviously a lot of different aspects to what we do, but if we take smart home as an example, how do you want to interact with your coffee maker, with your thermostat? You've got a lot of devices in your house that you probably have apps for. I know for me, I've got 15 different apps that are for different things in my house, and it's kind of a pain in the ass. Whereas I think the power of some of these models and where I think a lot of things will go is you just want to talk to them. You want to talk to your coffee maker and say, “Turn on and make me coffee.” And I think you turn the lights on and off, that's how you want to interact with the machines. And if you're in a factory setting, the last thing you want to be able to do is sort of work next to it. It doesn't have to be a humanoid robot, it could be a little device that's following you around and being like, “Cool, I'm stopping,” or “Follow up” or “Slow down.” And having to have an iPad out with a little screen on it is not a great way to interact with things in your environment. And so I think the combination of being able to build better devices and making them smarter while also sort of helping the human interaction side via large language models and speech is a very big deal and will help move the space even faster.
BP Right. So you've mentioned a couple of things where it's like you have this smart device and you want to make it smarter. It knows when your cat needs to eat or how not to give too much food. It would be great to talk to my coffee maker and say, “Tomorrow morning at 7 start brewing coffee, and I'm having guests so make a full pot instead of half like we usually do.” That would be sort of revelatory. If I could do that, I would be very excited. But another thing that occurs to me is that what we've always all been yearning for is the Rosie the Robot who I can speak to in that way and is sort of multimodal– could make me coffee, could clean the house, could feed the cat, whatever it is that I'm in the mood for. From your perspective, obviously now you're getting deep into the world of robotics, to what degree is that still a far off pipe dream? I have seen that Boston Dynamics has that robot. There was a video of it washing dishes and loading them into the dishwasher and stuff like that. But to what degree are we still basically, as I think you're describing, manipulating and extending the capabilities of smart home objects, occasionally making the mobile, versus having a sort of fully functional robotic assistant that, in its ability to interact with the world, is more similar to ChatGPT and its ability to interact conversationally? That's what blows us away about ChatGPT. You could go in any direction and it's able to handle things. Obviously, it makes mistakes. Where are we on that trajectory of the Rosie the Robot fantasy?
EH My personal opinion is that Rosie is a bit away, and I think that's okay. I don't think it matters too much for Viam from a business standpoint, and I think there's a lot of other things, but I think we're a bit of a ways away from that, both on the software side and on the price point side. Right now I think there would be a very price prohibitive and not nearly as good as you want, but I think there's so many ways you can make smaller incremental progress and actually solve interesting things. Having the devices be able to communicate in a more easy way, having one way for you to view all the apps, and then you can start getting a little bit smarter in little interesting ways. Take fridges– people have been sort of joking about smart fridges for a while like, “What do you really want a smart fridge to do?” I know what I want a smart fridge to do, which is I want cameras inside of my fridge and I want little tiny cameras inside of pantries so if I'm at home and I'm going to make dinner that night, I can look and see live what is in my fridge and then figure out what I need to pick up on the way home. Now, the cool thing about that is that you can then start training a custom model, if you have the right platform, for your fridge so your fridge could know what's in your house. But you don't need that to start, or you don't need to have that product built because that's a complicated thing to build and I don't know how accurate it's going to be in the short term. But a human with the live data view can actually seriously improve their experience making food and do these things while collecting the data you need to actually build the really smart product. So eventually you're like, “Well, hey, I'm going to make meatloaf for dinner,” and the ChatGPT is going to go look in your fridge and be like, “Cool, I need to order these things and Instacart it for you and it's going to be magic.” I think that's a ways off, but I think we can actually solve real problems today that make people's lives a little bit more convenient while working towards those goals. Same with kitchen. I don't think Rosie is going to make you dinner anytime soon, but I think you could have a robotic sous chef. I think you could have an arm that lives in your kitchen that helps you chop vegetables, that stirs certain dishes, that monitors the temperature and adjusts the temperature of a sauce while stirring it so that you can multitask more easily. There's a lot of ways you can sort of slowly move into things before you get Rosie.
BP Very cool. Is there anything we didn't discuss that you want to touch on in particular?
EH Look, the thing I would encourage people to do, especially I think a lot of people listening here are software engineers who, like you said at the beginning, are potentially a little intimidated or don't know where to start with hardware. And what I would say is, just try. Think about something in your day to day life that is like, “You know what? I've always wanted to solve that problem. I know I can do a good job.” A lot of software engineers can see the software solution problem in their life and then making the sensors and the actuators come to life is the magic black box that they're scared of. And I want to encourage them to look at what we have and then just reach out. Reach out to me, reach out to us, join us on Discord and just ask. We've got a lot of people here, including me, who love helping people go from some cool idea that they want to do at their house or their business or whatever and then figuring out how to make it come to life. Ask and try. It's not as bad as you think.
BP All right. If you listen to this and you got motivated to try to step over that line that you've been afraid of and work on some robotics, you can email Eliot directly. He's ready to go.
EH I'm ready. I love email.
[music plays]
BP All right, everybody. It is that time of the show. Let's shout out someone who came on Stack Overflow with a little curiosity or knowledge and helped to share it around the community. A Great Question Badge, awarded to Matt L. “How to remove all contents of a directory using GoLang.” It's a great question, Matt. You've helped over 94,000 people who had that same question, so we really appreciate you coming on and asking that, and in doing so, spreading a little knowledge around the Stack Overflow community. If you want to get in touch with me, I am Ben Popper, Director of Content here at Stack Overflow. Find me on X, or Twitter as I used to call it, @BenPopper. Email us, podcast@stackoverflow.com. And if you have any questions or suggestions, email us or leave a rating and review for the show, because it really helps.
EH I'm Eliot Horowitz, CEO and founder at Viam. If you want to check out Viam, just go to Viam.com and there's a lot of resources there. You can find me on Twitter, X, Instagram, LinkedIn, just email me, eliot@viam.com. It's easy to guess, most people do. Ask questions, find me, and let's chat.
BP Sweet. All right, everybody. Thanks for listening, and we will talk to you soon.
[outro music plays]