We chat with Eliot Horowitz, co-founder and CTO of MongoDB, about the inspiration for building a new kind of database, what the chief technical officer of a mid-sized public corporation actually does all day, and how the company is evolving in the cloud services era.
Sara reveals that she won a $500 gift card at a MongoDB hackathon, building an app that removed mustaches from people's pictures. This was many years ago, and no we were not paid in JetBlue gift cards to have Eliot on the show, although MongoDB is a client of Stack Overflow in other areas.
Mongo comes from humongous, cause, ya know, scale. That, plus HumongousDB.com was already taken and is a real mouthful to say.
Eliot talks about the frustrations he and his co-founder, Dwight Merriman, experienced while working together at DoubleClick and ShopWiki. DoubleClick began as a New York City ad tech company and evolved into the heart of Google’s real-time ad business after being acquired.
Frustrations with the database systems available at both these companies led the pair to decide it was time for a better mousetrap. Today, MongoDB is a public company worth north of $7 billion and a staff of more than 1900 people
We chat about why relational databases are still the core of computer science education in high school and college across the United States, and whether or not this will ever change.
During the show we skimmed some of the latest questions on Stack Overflow related to Mongo. Eliot took it back to his team and Tom Hollander, the PM for Mongo's chart product, delivered a great answer! Can you believe this website is free?
The Stack Overflow Podcast - Transcript - Eliot Horowitz
00:00 Paul Ford: It's like a little mouse of a database...
00:03 Eliot Horowitz: It's still very powerful.
00:04 Ben Popper: The Mongo mouse.
00:05 PF: Yeah, strong muscular, but also very sensitive mouse that lives in your phone.
00:10 BP: Perfect.
00:10 [INTRO MUSIC]
00:19 BP: Hey everybody. Welcome to the Stack Overflow podcast. This week we're brought to you by mParticle. What could you do with better data? PMs, developers, and data analysts at customer obsessed companies like Spotify and Airbnb, use mParticle to unify and validate their customer data, streamline their growth stack, guide product and make business decisions. Visit mparticle.com/stack to learn what mParticle can do for you.
00:46 BP: Welcome everybody to the Stack Overflow podcast. We have a guest with us today, Eliot Horowitz, the cofounder and CTO of MongoDB. Welcome Eliot.
00:56 EH: Thanks for having me.
00:56 BP: Yeah, our pleasure. So tell us a little bit about how this company was founded since you are the cofounder.
01:04 PF: Wait, let me swoop in. What is MongoDB?
01:06 BP: Yeah.
01:06 EH: That's an easy and hard question to answer. So MongoDB is a database and it's a a document database with a lot of great distributed features and what it is and how it created really are almost one in the same. MongoDB started because we were fundamentally very frustrated with databases and we wanted a database that solved our needs and that actually worked for us and for us that came down to a database that actually work in the data models and with the languages in a way that made sense. So documents and queries and indexes that made sense for modern development languages and that worked with distributed system so you could use it in the cloud, you can go from one machine to thousands of machines and not wake up at three in the morning when one machine breaks. And so how did this company start? Well it started because fundamentally Dwight and I were very selfish and wanted a database that we wanted and if no one else was going to build the database that we wanted to use, we were just going to go out and build it ourselves.
02:01 BP: So were you working together at a different company and then using a database that you didn't like or?
02:04So we worked together two companies prior, so I originally worked for DoubleClick for Dwight and we use a lot of databases there, you know, all the usual suspects, Oracle, et cetera, et cetera. Then we started another company together called ShopWiki. When we were sort of winding down our engagement with ShopWiki, we had another idea for an application we wanted to build. We very quickly realized that the database was yet again going to be a problem as it had been an every other project we'd ever worked on together. And as we started designing the database for this application, we realized we were way more interested in the database than the application itself and said let's just build a database.
02:39 Sara Chipps: You know, I remember when y'all were getting started, I was really involved in New York City development scene, MongoDB and Twillio I think have the best developer evangelism programs. I think out there. What made you be so early in developer vandalism? Cause there wasn't a lot of people doing it. Microsoft had done it for a long time, but they weren't going to hackathons and doing the really fun stuff that y'all were. So how did you get started doing that?
03:05 EH: So when we started, we didn't have a a marketing department and we didn't have a developer relations department and we didn't really have anything. So we were built this thing. We were super excited about it. We told a couple of our friends, they were super excited about it and basically we were like, I guess we should tell some other people, how do we do that? So we went to some meetups, we hired, we had two very young developers who worked for us and they're were like, Hey, we should go to some hackathons. It turns out I really enjoy just going to hackathon and hanging out with people. There was some at NYUs where I would just love showing up at 10:00 PM and is hanging out till 4:00 AM just talking to people. How they were building in how they were using Mongo and we just kept doing it because frankly it was fun and we enjoyed it. And people like Mongo. And we just kept doing it.
03:45 SC: That's great. I won $500 gift card to Jet Blue because of Mongo cause we made an app that removed mustaches.
03:53 EH: That sounds like a very good app.
03:56 That was way ahead of the curve before Instagram and Snapchat had those face filters...
03:59 EH: And that's like a noble cause.
04:02 PF: Can you explain why that used Mongo DB?
04:04 SC: Actually we had to store the images. We stored the images. It was like an overnight hack. And so we just stored the URL of the images in Mongo. This is before local storage was big.
04:15 BP: And what about Mongo? What's the meaning there? DB database. Mongo means what?
04:19 EH: Mongo comes from humongous. So one of the core value propositions of Mongo is that I can scale as big as you need it to. Humongous is obviously a word that means big and...
04:27 BP: gotcha. So humongousdb.com was already taken.
04:30 EH: Mouthful. It's a mouthful.
04:33 BP: I might start using that.
04:35 SC: You're going to make humungousdb?
04:36 BP: [inaudible] just Mongo.
04:38 SC: Oh Mongo, I like that.
04:39 BP: So talk us through a little bit of the earliest stages of the company. Was it immediately something that you were able to bootstrap, you thought about venture funding, obviously you'd been at other companies. Um, what were the early roadblocks, you know, that you had to get through in order to scale it up as a company?
04:54 EH: So we bootstrapped very, very briefly for four or five months. And so that just means we didn't make money because just the two of us. So we just just started building it in about seven months in or so. So in may of, uh, 2008, we met with Albert from Union Square Ventures who had a very similar idea of what a database of the future could look like and had experienced both personally and in a lot of the companies that he'd invested in pain points around scaling around data models around like, okay, we're all, you know, we're using modern programming languages, we're using Python, we're using Ruby, we're using Java, and we're using databases from the 70s.
05:28 PF: Does Albert have a last name or is he...?
05:30 EH: Albert Wenger.
05:31 PF: Oh okay not like Prince who was...
05:32 EH: e is kind of like Prince, but no, but Albert Wenger. And so he had, he had actually written this blog post and we were like, Oh, that sounds like what we want to build. And so we talked to him until we raised some money. And originally we weren't, we wanted to build a database. We also wanted to build sort of a platform around the database. So for example, one of the big challenges early on was we realized that we couldn't do everything we wanted. So we just focused just on the database. And then the spring of 2009 we released the first public version of Mongo. And very quickly, you know, in a matter of months, people found it and were like, wow, this thing is actually cool, which led to about three years of not sleeping and insanity. And you know, here's a, you know, we had a interesting, unique problem, which was that the excitement around Mongo was so immense and our team was very small. It was a, one of my favorite stories was probably about six months after we launched this relatively large internet brand called us and said, Hey, we're looking at Mongo for some really important apps. We'd love for you to send, you know, a salesperson and some presales engineers out and we were like, that sounds great. At the time we were four people, it was me, Dwight, and two engineers right out of college
06:40 PF: Four pre-sales engineers.
06:41 EH: So we had no one and so me and one of the other junior engineers got on a plane and we just showed up and just started walking them through stuff. I think they had, if they had any idea that that was half the company at the time, they may not have made that a decision.
06:54 SC: I remember there were, at the time you guys were the first really big ones. There was a few databases were changing at the time, and I remember there being two reactions, maybe you can confirm, but half of it were people that were used to relational databases being like, Oh my God, what is this? I don't even know. There's just going to be data laying around and no one's going to know where it's even there. And then there was other people like myself that were like, Jason, I'm so excited. How did you convert those early folks? Because now you know, everyone understands that non-relational databases are great and have lots of uses, but at the time, how did you convert those folks?
07:31 EH: So a couple of things. One is I would say we're still fighting that same fight, right? I would still say that the default for most people is still relational and it's changing, right? You know, every high school, every college teaching relational databases. And so we're still working. And the nice thing is when you talk to developers who are just getting started, when you talk to people who are taking college classes in web, you know, in building web applications, you talked to people in coding academies, MongoDB and documents make so much more sense than relational databases. So those are actually in some ways the easiest people to talk to cause they're like, Oh, I get it. I'm using Python, I'm using Ruby, I have the structure. I can save that structure, I can create indexes on an inquiry on it. This all makes sense. The people who are used to relational databases. Some of them were like, Oh my God, this solves all the problems I've ever had. Others are, wait a minute, what's going on here? These are the people that you're talking about. This is all new and scary. So a lot of it was how we convince them it was showing them if they did it, what the benefits are, right? Look, your developers can move faster. This thing makes more sense in terms of your application design. Oh look, maybe you can get rid of this caching layer that you have where you're just caching, frankly, a document composed of 20 tables that you've got to maintain, right? There are tons of products out there whose entire existence is, I've got this relational database that doesn't quite work for what I need to. I'm just going to cash it and this other thing to make it fast, right? We can make your architecture a lot simpler.
08:58 EH: So that was part of the story. The other part of the story was adding more and more features from relational databases that they were like, Hey, we want some of this. So, for example, we added support for Json and Schema, right? Because the document model isn't about, you know, freedom to do whatever you want and complete wild west. Yes, there are certain cases where you want total flexibility. Maybe for some use cases you're like, I want a developer to put whatever they want in there. And other cases you're like, I want a developer to put whatever they want in this section of the document, but this section should be very well structured. I want these five things. Maybe there's an array. Everything in this array should have these fields. And so Json Schema for example lets you actually have structure. Let's a DBA or someone puts some rules around the database while at the same time still having the flexibility you need if you need it.
09:40 SC: That's great. So now how big are y'all?
09:41 EH: So Mongo is public now. So the company is 19ish hundred people.
09:46 SC: Wow!
09:46 EH: So it's grown quite a bit. Some of those four people a decade ago
09:49 SC: You have presales engineers now.
09:50 EH: We have a couple of hundred presales engineers, um, who make my life a lot better every day.
09:55 SC: That's great.
09:56 BP: And how global is that?
09:57 EH: From a field standpoint, we are, you know, there's probably a MongoDB employee in almost every major city around the world. The MongoDB engineering team is also quite global. You know, we've got still the bulk of our engineers, probably not the majority but close to a majority in New York. But we've got big engineering offices in the West coast, big in Sydney, big in Dublin, people in Barcelona, in Germany. And so just people everywhere.
10:20 BP: And so do you have like one product you offer? Do you have a suite of products? Is there something that you offer now that's completely different from, you know, sort of the foundational stuff that we were just talking about?
10:29 EH: So not completely different. We've certainly expanded and we think about data and thinking about data problems developers have, right? So our goal hasn't really changed from the beginning. It's me as a developer. Working with data was always a pain. Our goal is to make working with data easy. So we have the core database, obviously that's where we spend a lot of our time. We also have MongoDB Atlas, which is sort of the hosted version, which is now sort of the the main way a lot of people are consuming Mongo cause it's a cloud service. You can just come in there, you can start for free scales because you want use it on any of the big public clouds. So whether it's Amazon, Google or Microsoft, it's fine. You can migrate between them, you can pick and choose. So you get all these sort of great benefits of that kind of a service and very easy to use. And then we're also doing things around that to, you know, help you with other kinds of data problems. For example, we have a charting solution on top of this where you can just, if you want to build a dashboard or charts and then embed them in your application, maybe you've got a blog and you want to embed a chart and your in your blog posts or you actually want to embed something into an application with user-specific data. So I want to build a chart of my activity. You can just go build the chart, build the dashboard in this tool, and then embedded into your web application. Other things to help build a web applications, I mean announced, you know, a graph QL support, for example, a few weeks ago where if you're building a, a react application in a web application and you want to use graph QL to get the data from Atlas directly into your application, you can do that without writing any backend code. So lots of things around the database, but all in the spirit of how do we make working with data easier for developers.
11:56 PF: What do you do all day as a CTO?
11:59 EH: Question I ask a lot.
12:00 SC: [laughs] You ask?
12:02 EH: Well, no, I asked myself, it's like I look back on my week, I'm like, what did I do this week?
12:06 PF: You're hacking Mongo, like things are going. They're, they're busy. There's four people in the company now. There's 1900 people and you're the chief technology officer. That's, that's gotta be a little different.
12:16 EH: Definitely different. I still spend more time writing software than you'd expect though. It's been about five years since I wrote any production Mongo code.
12:25 PF: They're not, they're not letting that happen.
12:28 EH: They were very happy when I said, fine, I'll stop. No one wants to do code reviews for me.
12:32 PF: We're not shipping that...it's like editing the Editor in Chief.
12:36 EH: And so that stopped. So look, my job is predominantly four to five things. We'll see if I can remember which ones they are. So one and probably the thing I spend the most time thinking about is product, right? What should we be building? What do our users want? Where is the space going? What are the kinds of problems developers need? You know, is this product good for developers? Can they understand it? You know, just kind of sort of product management 101.
13:00 PF: You gotta do some listening to get there. Right? Like who are you talking to?
13:02 EH: Anyone who will talk to me.
13:03 PF: Okay.
13:05 EH: I taught, you know, I speak at a lot of conferences, a lot of Mongo events and I just love going to Mongo events and just, you know, talking to people, seeing what's, what works, what doesn't work. We obviously have a lot of people talking to customers. So reading that feedback I talked to, you know, another main thing I do with my time is talk to customers, talk to users, whether it's in person or just, you know, I love talking to users, I love helping them understand where Mongo fits, understanding what's good about Mongo, what's bad about Mongo. So that's another thing I spend a lot of time doing. One of the things that has changed with last few years, it's been any more and more time thinking about how do we help people understand where Mongo fits into the broader ecosystem. You know, where does Mongo, you know, what is different in Mongo and big cloud vendors? How do we fit in? Where are we going explaining the Mongo story. I think there's a lot of different pieces to it and it's, there is a lot of noise in the world about sort of the developer space. Obviously everyone wants developers to like their stuff like their platform. And so I spend a lot of time sort of figuring out the right ways to do that. And a lot of that ties directly to another thing I spend time know. Obviously the company's strategy, you know, where do we want to make big investments? What are the things that are going to matter to developers, not just today but over, you know, in three years, five years, 10 years, where is this space going? And last certainly not least is I do have a large team and I do have to manage it at some point. So I got an actually, you know, I gotta you know, go talk to them and deal with HR issues and all that kind of stuff.
14:23 BP: So I know Mongo is a public company. Maybe you're limited in what you can say, but where are the places that you have been, you know, making investments within the last year or two and one of the things you see on the horizon, are there trends?
14:35 PF: Let's get some bold, bold predictons.
14:37 BP: Yeah, are you going to acquire Whole Foods? No. Um, are there things like, you know, machine learning or deep learning that fundamentally, you know, would change where you can apply Mongo in an interesting way?
14:47 EH: One of the big investments we made last year as we acquired a company called the Realm. So Realm is a mobile, you know it's an opensource mobile database, so about a quarter of the apps on your phone are using realm as the core storage engine. So one of the big things that we're working on is changing the way people think about sort of data and synchronization between mobile and web and all these sorts of areas. So for example, a lot of the apps on your phone you also are using on your desktop or in a browser or you're sharing data between one phone and a different device or someone [inaudible]. It's a collaborative app and synchronization is a huge problem, right? I'm editing something in my phone, I've got a to do list on my phone. It's using an app. The synchronization often works well. Sometimes it doesn't. Sometimes I've got to like restart the app because the synchronization doesn't quite work. Doing synchronization right between mobile devices and web application, then backend is really hard. So we think that we can just solve that problem, right? We want building a mobile application that is collaborative and synchronizes with back ends and real time to be as easy as writing a standalone mobile app. Right? We don't want that to just be like, Oh my God, I've got to build a mobile application and any of this whole team that knows how to do synchronization and manage conflicts, we think that's a whole space that can just, we can just make a hundred times easier and just solve it once and for all.
16:01 PF: I mean like GitHub for data.
16:03 EH: Exactly right.
16:04 PF: But no, you're not doing anything. The computer is doing it all for you.
16:07 EH: Right. It's, it's much more of a, let me think about it. And you know, that all ties into also with sort of privacy and security, the panacea and where we want to go is I've got data, I've got some sort of, I know roughly that my data looks like. I want to set some rules around that data based on attributes. These documents Elliot owns, he can do whatever he wants these documents. If he wants to do something bad, fine, that's okay. I don't care. Elliot is saying, you know what, someone else can just, you know, read these documents or maybe they can read only these fields on these kinds of documents. Or maybe they can even edit these fields, but only these fields under these conditions. How can you set up some very simple rules on the back end so that your applications can just consume the data? And the nice thing about that is if you just have simple rules, it's not just sort of the basic crud applications that work. Then if you can push those rules down into analytics and aggregations, then Elliot can go write queries and understand the data. Or developers can go build dashboards in their applications that query the data without having to think about the privacy concerns because someone thought about them already, they set the rules in place. And everything I'm doing is just on top of those, on top of those rules.
17:08 PF: So you've got Realm, which is sort of in its own world, right? And then you've got Mongo. How do those worlds sort of come together?
17:15 EH: So inherently they're very similar. So Realm is a fundamentally an object database. For your phone. You take, like if you're on an iPhone, you've got a Swift object and you can perceive this Swift object. So very much similar to a document.
17:29 PF: So you have Realm save this, I'm going to want it later.
17:31 EH: Right, Save this. And then if you got different threads, for example, if you've got one thread in the background, updating an object and you've got a UX component in your app that's based on this object, it will automatically update the UI and response to the data changing.
17:43 PF: This is good for people to think about it, right? Like it's not your phone is doing 5,000 things at once.
17:47 EH: Exactly.
17:47 PF: So this is a database that's cognitive. The fact that your phone is doing 5,000 things at once.
17:52 EH: Threading is also cognitive, like power saving mode and sleep. How's it supposed to respond when you're low on power? How's this versus when it's in the background versus when it's the act of application, all those sorts of things. No developer wants to think..
18:06 PF: I very little, it's very sensitive.
18:09 BP: Now at the same time. So the data model is fundamentally very similar to Mongo, right? Structure to raise documents, et cetera, et cetera. And so the key thing for example that we're working on is how do we take that data and automatically synchronize that back to the cloud. With privacy, with security.
18:27 PF: And in this case, the cloud means Monk. Atlas. Okay.
18:29 EH: Yup. And that just, you know, again, makes developers a lot simpler. It makes their lives a lot simpler. They don't have to solve this problem.
18:36 PF: And we should also make developers simpler. I mean, that's the goal here. Ultimately it's just very hard.
18:40 SC: Like them as people?
18:40 PF: Oh my God. Yeah. Wouldn't it be great if we didn't have to like keep building software but could just simplify the developers.
18:45 EH: Well, I mean there's a facetious way to take that, but the way that I think about it is you want to make development more approachable for everyone. You know, some of the early biggest fans of Mongo were people who are new developers, maybe people in places where they couldn't, you know, didn't quite have the resources to have the, you know, the best education or people who could just, you know, so the people who could just learn it quickly and get started and build applications. And I think that development for so long has been seen as this like Holy grail of like, Oh my God, if you're a developer you have to be brilliant to get anything done. Which I think is not what's going to be the case for the foreseeable future. Right. It has to be something that's more, a, more approachable, more consumable by most people.
19:28 SC: So yesterday I had dinner with my, with my dad who listens to this podcast, hi dad. And he said to me..
19:34 PF: Hi Mr. Chipps.
19:34 SC: [laughs] I was in the Apple store before and my, my phone went off and I picked up my phone and it was like, how is the Apple store? And it was terrifying. How do we help people understand? How do you, do you think about this often? You know when you're thinking about privacy and data, how do we make the average person more comfortable?
19:55 EH: So I think one thing developers and technologists have not done a good job of explaining things in simple ways.
20:03 SC: Yeah.
20:03 EH: I have tried to explain Mongo to my grandparents and you know, having tried to do that many times I think I finally have come up with decent analogies that they understand but I think your technologists have to spend more time thinking about how do I make it simple. I was talking to someone earlier this week about open source and why open source is such a powerful concept and they were like I agree with you but then she said, but the people that I am working with, the people that I'm trying to get to use my solutions hear the word open source and assume it's scary as soon people are going to like, it's less secure and it's these things that as a developer like no open source tends to be more secure. It tends to be safe, its better. But for people who just sort of don't know this are people who aren't technologists open source sounds scary. It's like, Oh, but my wait, this is, this is for data. That means all my data is exposed or does that mean people are gonna see the codes so they can find bugs faster and they don't understand how this works. And it requires, frankly a lot of time and effort and just working through different ways of explaining it. I think if you just keep saying the same thing over and over again, louder and louder doesn't tend to work, you've got to actually come up with better analogies, better ways of explaining why things matter and where you're going and at the end of the day actually make simpler paradigms. Right. If you think about Mongo verse relational databases. We think Mongo is a fundamentally simpler way to think. It's not less powerful. We haven't removed power, we've just done it in a way that is easier to understand, easier to, you know, and easier to build on. Right? Going from day zero to day 100 to day 200 is a simpler path. And I think that's sort of important for all technology.
21:35 BP: I mean you were saying before that you can really connect with students around this, but often in high school and college they're still learning relational database as far as that's kind of the fundamental and the curriculum and as a novice, you know, sort of coder myself. I mean I think one of the things I think about a lot is if a lot of people are going to have these kinds of jobs, we should treat, you know, computer programming ia one of the sort of fundamental pieces of literacy, you know, reading, writing, arithmetic, whatever it is. And until that changes, I think it will continue to be kind of a, you know, somewhat frightening for a lot of people because they'll get to middle school or high school without really having touched any of this stuff.
22:10 EH: Uh, totally agree. And even the way that public education is approaching this, in some cases it's very good. And some cases we're still figuring out, for example, there's one like one plug, there's one um, project called Bootstrap. Not the Twitter bootstrap UI thing, but it's a Bootstrap is a middle school math curriculum that has been proven to teach math...proven is a strong word, but it seems to be teaching middle schoolers algebra better than standard math curriculums and teaches them computer science along the way.
22:39 SC: So cool, they're like so related.
22:40 EH: Exactly. And then I think, but like you know, I know that some of the people who created it, you know, they've been thinking about this problem for 20 years. They finally have sort of cracked some core concepts of how do we teach kids these fundamentals and get them to understand math better? Cause you know, most people will say algebra is not something that most middle schoolers enjoy is intuitive, but they're doing it in a way that's more intuitive and in teaching them programming at the same time.
23:05 SC: I would have loved that in school. I feel.
23:07 EH: And so, and so I think these things are super important. Not just for middle school, but you know, think about it in high school and college are the same way. Computers are just now starting to become a standard part of curriculum. You know, I look at my daughter like she uses Google in school presentations in Google very easily and that's like, it's not like a computer. Like I remember when I was a kid, there was a computer class and that was like a special class where you got to use a computer and that's not a good way to teach kids how to use computers. But if you're like, okay, you have to do a history presentation and you're going to do it on Google, you're going to use Google presentations. That makes sense. That's a way to teach them about something with tangible results in a way that actually is useful. And I think a lot of things needs to be rethought that way. And you know, what do we really want to teach kids? How do we help people understand what's important and why and how it works.
23:56 [MUSIC]
23:59BP: At the end of every episode we usually do a thing where we shout out the community a badge called lifeboat, which is a question that was down voted meaning people were like, this is not a good question since and then therefore not going to be answered. And somebody came in, answered the question and it was up voted and you know became sort of part of the, you know, the stack of knowledge that we have. So this question was asked just seven hours ago by user Jasper with one reputation. So brand new contributor, hello Jasper. He says how to plot uniform time series in MongoDB charts? Think you can handle this one?
24:27 EH: A uniform time series in MongoDB charts. It's going to be very hard without a screen in front of you because it's a visual product.
24:36 BP: Well I'll read the question to you and then maybe I can share the screen. You have time, you have time after this episode is recorded to go in and answer it and get that lifeboat badge. I've just started to use MongoDB charts to plot incoming data from a series of IOT devices that send at regular intervals. Each device sends a package with a timestamp and some data. Json are no sequel DB and I would like to plot several devices on the same chart. To visually check the continuity of the data flowing in ie. if a device fails to upload, I want to plot each data point over a continuous time series X axis. Does anyone know if Mongo DB charts has a feature to make the X axis continuous? Currently the chart plots one point per observation and spaces them equally no matter the time in between. And then he's left an example here, which is hyperlinked example of data points one two, three I'll have six minutes in between but visually appear to be non uniform. So think about it. If you've got an answer, you can say it now on the pod. If not, maybe you can write it in.
25:30 PF: What? In my...dear Lord.
25:34 SC: [laughs]
25:34 PF: Also Ben just brought up something where it's like 12 lines and the numbers one, two, three scribbled by hand. Above. So anyways..
25:42 BP: This is the CTF, he can't solve it. Who can?
25:45 EH: I, I've got a couple ideas. I've got a couple ideas. One is there's a new, a relatively new feature in charts where you can do a pre-processing phase. Fundamentally charts creates an aggregation pipeline where you can put a pre-aggregation pipeline that does this cleaning. We've got a couple of functions that will let you actually do a uniform distribution. So you'd have to not do that in charts itself, but do it in, you can create a view in Mongo. It is a view of the data that becomes normalized.
26:11 SC: So you're thinking like to use that feature and figure out if there's any missing timestamps.
26:16 EH: Exactly. And then you sort of normalize the time across the range.
26:19 SC: Cool.
26:19 EH: So you find the minimum, find the maximum and then, and you could do that in a MongoDB view uh, then you can chart the view rather than the raw data. And I'm surprised there's not a charts feature for this that probably on the backlog. That one I'll have to look at, look into, I don't know off the top, I can't remember at the top of my head.
26:33 BP: If this question results in new feature I'll be super happy.
26:40 SC: This is the biggest, yeah, it's everyone's biggest nightmare is someone in very leadership coming back and being like, you know, I just got an idea today.
26:46 BP: Podcasts are changing the whole rug mat.
26:49 EH: That sounds like my team's worst nightmare. That happens every day.
26:52 PF: How many, how many Mongo questions are there?
26:55 BP: Oh, there's a lot. Uh MongoDB stuff. I go to tags. There is basically 125,000 questions with the MongoDB tag. 62 asked us today, 433 this week and then there's a bunch of sort of other ones, MongoDB query, MongoDB.net driver, MongoDB Java, so then there's a whole family. I mean those are all much smaller.
27:14 PF: I mean that kind of makes the point, right? This is a whole world. Once you've started with Mongo, you're going to keep going in all kinds of different directions. Alright. I mean it's, it's a, does that Elliot, does that...
27:24 EH: We just solved the question live on the air? That was pretty cool. Well we're not live.
27:27 PF: Well we didn't validate it yet. You know, let's, let's be real engineers here. When you hear hundreds of thousands of questions, is that a good thing?
27:35 EH: I think that's a good thing. I mean, in a perfect world, everything would be so intuitive that everyone understands everything and never had to ask a question.
27:42 SC: Never happened to me.
27:44 EH: That's never going to happen. That's never good. And it will never happen in the history of programming. So, you know, one of my, you know, for the first five, six years of Mongo's existence, I made it my mission to answer every question about Mongo, anywhere on the internet as fast as humanly possible. So we had a mailing list. I think I'm still the number one contributor on the mailing list and it's not because it was like I want to be number one. It was like, no, like this was my life. I literally was close to 24/7.
If someone asks a question, whether it was on Google groups, in JIRA, on the MongoDB public IRSE channel that I'm monitored nearly 24/7, how can I help them as fast as possible? My life was maybe not the most fun for a few years there, but it was great. I mean literally it was just like people just asking questions, helping them, adding features. You know, this is when we were moving fast. We could just add features, add things all the time. I remember adding a feature for, you know, a big user. I was in Silicon Valley talking to some people and there was, I had an evening free and I sat at a hotel bar drinking a glass of wine and added some cool Mongo features for the next six hours and just hacked away and you know, and they put them into production two weeks later and they worked and it made their app better. And being able to sort of listen to users, add features fast, put them in user's hands and see them sort of make progress because of those sort of hard to beat that feeling.
29:02 PF: I mean that's, it sounds like it was critical to your growth, right? I mean just like, yeah, I mean that this bizarre angel floating around answering every question.
29:11 EH: I wouldn't put it that way, but I think the key is the concept of Mongo was very sort of exciting for people. The reality is that at that point we had put three person years of effort into the database, the amount of person years that has gone into any other database that people use. Oracle, my SQL, Postgres, I'm not even sure what, you know, how many digits there are there, but many, many orders of magnitude more. And we were tiny and so the ability, you know, so we can move the needle fast. When we added distributed transactions to Mongo, we think it was about a roughly hundred person year, maybe even more a hundred to 200 person year effort that we couldn't even fathom adding features like that originally. Cause like we had, you know, four people in the entire company and so pros and cons.
30:00 PF: Well and you were going to do what you could do.
30:02 EH: Yeah. We wanted to make users successful. You know, cause the first Mongo users are the very first people use Mongo were friends of ours and so it was very easy for me cause like you were my friend, similar personalities, I'm going to make you as successful as possible. You've got a problem, I'm going to fix it. If you're have like I remember one of them had a big release that was going out in the middle of the night and I was like I will be up with you and we're going to sit next to each other and we're going to do this release together. I will be watching the Mongo side. You're going to be watching your side and we're going to do it together. And that was sort of what carried us through those first few years. I started acting like that with every person using a Mongo.
30:34 BP: We really appreciate you coming on and sharing your story and all that knowledge with us.
30:38 EH: Thanks. Really appreciate you having me. It's been alright.
30:40 BP: I'm Ben Popper, director of content here at Stack Overflow. You can find me on Twitter @BenPopper.
30:46 SC: I'm @SaraJChipps on Twitter. I'm the director of public Q&A here in Stack Overflow. If you have some extra laptops laying around, send them through code cooperative. They're looking for some.
30:55 PF: And I'm Paul Ford. I am the cofounder and CEO of a software company called Postlight. Boy, if you're an engineer or a product manager or designer looking for work, you should get in touch @Ftrain on Twitter.
31:08 EH: And I am Eliot Horowitz, CTO and cofounder of MongoDB. Send me tweets @ElliotHorowitz on Twitter and ask Mongo questions. Mongo feedback, always happy to chat.
31:19 [OUTRO MUSIC]
31:20 BP: Terrific. We're all done. That's a wrap.