Jaime Torrealba, a frontend developer currently at Push Security, joins Ryan to talk about 3D graphics and web development. Their conversation ranges from the evolution of technologies like WebGL and WebGPU to the importance of performance optimization for different devices to the frameworks available for creating 3D experiences. Jaime also explains the layers of abstraction in 3D development and shares some insight into what the future of 3D applications might look like.
Tres.js is an open-source 3D engine for Vue built on Three.js.
Find Jaime on LinkedIn or GitHub or explore his creative lab.
Push is a browser-based identity security platform that detects and blocks identity attacks, enforces security controls, and monitors employee logins to cloud accounts.
Shoutout to Stack Overflow user zwol, who earned a Lifeboat badge with an excellent answer to How would you write the equivalent of this C++ loop in Rust.
[intro music plays]
Ryan Donovan Hello everyone, and welcome to the Stack Overflow Podcast, a place to talk all things software and technology. I'm Ryan Donovan. I edit the blog and host the podcast here at Stack Overflow, and today, we're going to be talking about 3D on the web. Everybody, you know, talks about 3D in the, the games, but you can have it in a webpage. Our guest today wrote in to, uh, suggest this topic. Jaime Torrealaba is a front engineer at Push security. How you doing, Jaime?
Jaime Torrealaba I’m really fine. Thanks for having me.
RD Of course. So, at the, uh, the top of the program, we like to get to know our guests, find out how they got into software and technology, and how you came to be interested in 3D on the web.
JT Ooh, well I believe that was a couple of years ago, maybe four or five years ago.
RD Mm-hmm.
JT And I always loved games, and I always love design. So, at some point, the 2D design in the web spectrum, like, you know, just the dev and the UIs, uh, I believe get just shorter. So I want to expand more to create more. And then I realized that it's not that difficult that you might think. It's, it's, it's actually quite, um, for the entry barrier, it's quite easy. It's because there are so many abstraction already done for you that you don't have to do it, you don't have to think about it. So yeah. And I have to mention also the Three.js Journey, a course made by Bruno Simon, which is an amazing guy. That was my, my, my first introduction to the 3D on the web.
RD Oh, okay. And you also, uh, were a contributor to an open source 3D JavaScript project. Is that right?
JT Yeah, yeah. I, I was a, a contributor and open source maintainer in Tres.js, which is the, uh, the file library for creating WebGL experience with Vue.js.
RD Mm-hmm.
JT It's using Three.js, uh, under the hood. Um, it's in, in any case, it's the equivalent to react-three-fiber, which is the most famous one.
RD Mm-hmm.
JT Um, but for Vue.js.
RD So let's, let's talk about what 3D on the web looks like. How, how does it work? Um, I mean, obviously, you're using some JavaScript, but, it's not just, like, you know, fancy CSS, uh, manipulation, right? It's a little more than that.
JT Yeah, yeah, yeah. Uh, so I would like to start saying that we, we don't use basically none CSS. Uh, we use JavaScript, and we also use GLSL, which is, uh, a scripted language, C-like. It's more– it's not JavaScript.
RD Mmm.
JT With that, you can actually send instruction to the GPU directly.
RD Hmm.
JT Uh, and, and it’s, it’s quite interesting because change your mentality of working with the CPU. Now you are, uh, working with a multithread, uh, GPU stuff. So, yes. Uh, one thing, one good thing is that we use WebGL, which is basically a grabber off OpenGL. And OpenGL is– it's, you know, in every environment it’s, it’s working in C++ and working outside the web, so when you learn GLSL, you actually can do stuff out of the, of the browser. So yes, Three.js use OpenGL under the hood. OpenGL and WebGL, it's uh, quite challenging if you want to do it by yourself. It’s– you have to, you know, do a lot of stuff. But Three.js come with a lot of abstraction, so you don't have to think in, you know, creating the camera by the file or creating the, the starting mesh, or a cube, or all the mathematician – mathematical, sorry –
RD Mm-hmm.
JT Uh, stuff. So, starting with Three.js, it's actually quite doable.
RD Mm-hmm.
JT In fact, you in the, in the website, you just basically copy and paste and you have the example.
RD Right.
JT And yes, we create a scene there. We basically started, to play with it, basically.
RD Mm-hmm.
JT Now we got, uh, the transition from WebGL, which now I think has, like, 15 years in, in, over there. And we are in transition to WebGPU, which is the, a new API using Vulkan. Uh, Vulkan is an API for working with the GPU.
RD Mm-hmm.
JT WebGPU promise much more, um, better performance, um, better compatibility with other systems. Like, for example, Unity can use WebGPU to embed, uh, games in the web.
RD Mm-hmm.
JT Yeah. And, um, computer shaded, which is a little more complicated concept, but it's, it's come from the file.
RD Mm-hmm.
JT Um, so yeah, that is, that is the future now; it's, it's, it's going there.
RD That's, uh, that's awesome. That's a great overview. And, you know, I think, obviously, my, my initial– when somebody says they can do 3D on the, the web is, you know, how much, uh, how much power, how much fealty can you get towards, like, the, the graphics, the 3D game graphics, right? Because there's a little bit of, uh, computing overhead that the browser takes.
JT Well, one of the thing that you have to consider always in the web is the performance, right? You have to be–
RD Mmm.
JT –carefully because you can have this gamer PC, and everything will run smoothly, 120 frame per second, but then you have to create experience for a mobile. So, you have to think in, okay, it has to actually run in a, in a low performance device. So, so how we handle that can't be for, uh, frame per second because a mobile will not handle the same frame, frame per second for– as, as a game PC. Uh, I believe that is one of the main reason why the games in the web, like, games within with JavaScript are not that, uh, realistic. That– let's say that good.
RD Right, right, right. So does, um, does most of the, the sort of 3D processing happen on the client side? Or is there some server side work first?
JT Yeah, yeah, yeah. It happen, uh, in the client side. It happen, uh, it used the GPU basically. So, so, yes.
RD Hmm. Um, yeah, I mean we've, we've talked to, um, folks doing 3D and game engines, and they also talk about, you know, you don't know the, the sort of– the specs you'll have on the, the client side, right? Like, somebody running Call of Duty may have, uh, a potato machine, right? They may only be able to run the, the smallest part. And how do you, uh, besides having, you know, a very low poly game, are there ways to accommodate both the high, high performance and the low performance machines?
JT Yeah, yeah. Uh, fortunately the, the graphic industry has quite a lot of time, and there are many, many people, many papers, um, techniques, how to optimize, written by, you know, amazing people how–
RD Right.
JT –doing this for many time. So yes, actually we can, we can handle really good performance, but you have to, you know, read stuff, how to do it, how to performance, how many, uh, triangles – because everything is triangles in, in, in the 3D world. And, yeah, how to, how we can reduce the triangle or, or, or the textures, or many, many things that we can do for optimize different experiences.
RD Hmm. Do, do you have, um, a favorite sort of optimization technique that, when you learned about it, you were like, “Wow, this is, this is kind of a, a mind-blowing way of thinking of it?”
JT Uh, yeah, indeed. Uh, the basic for me is optimize the model, which is optimize the, the texture. Again, texture is just a fancy word for saying an image. Uh, for example, a PNG let's– using a reduced, uh, reduced sizes in JP, for example, if you're going need the, the, the transparency. Uh, actually the, the model try to reduce the topology, or using something that we call a L of D, which is level of detail, which is, at certain– it's kind of like the, the break point you see as a, a certain view, certain distance of the camera. We, we show some model, like, if you are really close, you have high model, and then if you are– that kind of stuff. That are my favorites, definitely.
RD Mm-hmm. Yeah. I, I think, uh, just seeing how much of this is math, um, has been consistently eye-opening. So, the JavaScript end of it, you said it's, it's sort of, uh, shortcuts to rendering on, on the web. Um, like, how easy is it to, to spin up a scene, uh, get textures, and what kind of download package does, does the client need to actually run, uh, a 3D game?
JT Well, there are probably, there are more, but, uh, and now that I can remember, there are three main players, and the one that I use and I love is Three.js. But also there is Babylon.js–
RD Mm-hmm.
JT –and, uh, PlayCanvas. They're, like, the, I would say the three main ones in, in the webs, uh environment. Fortunately, all three basically come with all sorts of functions, abstraction, that you basically using. Uh, for example, I want to use, uh, uh, a texture, and then you use the, the lot texture function–
RD Mm-hmm.
JT –attaching to the model or whatever. So, it's actually really easy. It's actually quite, quite, you know, doable.
RD Yeah. So it almost serves the function of, of, like, a game engine. Like, uh, Unreal or, uh, Godot, or something.
JT Exactly. Yeah, yeah, yeah.
RD Yeah. Are there any 3D, um, frameworks or anything that are looking at, um, WebAssembly?
JT Uh, that is a very good question. I am not so sure now. What I can, uh, tell you is that some, some repos that are written in other, uh, languages can be used, in WebAssembly. And one of the best example is, uh, Rapier. Rapier? Rapier? I don’t know the right pronunciation, but, basically, it's a physics engine.
RD Mmm.
JT And this is the, the original is written in Rust, and with WebAssembly, we can use it with JavaScript. And what it does is provide you all the complicated calculation of the physics. Basically, you have a–- you define your sphere, right? And then you define a plane, a floor.
RD Mm-hmm.
JT And then what you do is, okay, engine, uh, Rapier, I got, I got these two rigid bodies–
RD Mm-hmm.
JT –and then I want you to actually animate it to apply the physics, and then you can pass stuff like mass and uh, uh, yeah, how much the gravity strong it is and stuff like that. So it actually animate and create a simulation of the physics for you. And that is using WebAssembly.
RD Mm-hmm. Well, that's, that's interesting. So it's adding in these, these extra– um, you know, the sort of, uh, standard stuff for most 3D AAA games, you're, you're adding it in through these WebAssembly. Yeah.
JT Yeah, yeah, yeah, yeah, yeah.
RD So, in, in working on the, the sort of, uh, JavaScript frontend wrapper, what, what are the, the sort of things that you had to consider when sort of writing that, that sort of the engine code, basically, for JavaScript?
JT Uh, let me start explaining a little about the– what I call obstruction layers.
RD Mm-hmm.
JT And, as I mentioned, the first one, or the core is, uh, WebGL/WebGPU now–
RD Mmm.
JT –in the process. And then we got these, uh, libraries like Three.js, these, these engines, like, uh, Babylon, um, which allow you to, uh, not starting from zero, basically. Uh, they come with the camera, with models, with how to import 3D models, whatever, whatever. And then, there's another layer of obstruction, which is, uh, the one that I, I was maintaining.
RD Mm-hmm.
JT Tres.js, uh, Reqtify, uh, Threlte for, uh, Svelte, uh, Angular Tree.
RD Mm-hmm.
JT And what they do, it's provide you a way to create this, uh, experience way more easier–
RD Mm-hmm.
JT –in a declarative way. So you can do all the power of this, uh, uh, frontend framework. Like, let's, let's call React for– because it's probably the most, uh, common one. And you can, uh, divide your, your receding components, you can pass props, or emit events, or, uh, apply some reactivity. That kind of stuff. That, that power gives you this, this framework, this. So, yeah.
RD Yeah. Um, can you give an example of that, that sort of, like, highest level of, um, like, what's, what's the separation of concerns there? I assume the WebGL is just, like, writing the pixels on the screen, right?
JT Yeah.
RD Yeah.
JT Exactly, yeah.
RD And then Three.js is, uh, working with models in space. Is that right?
JT Three.js provide you, yeah, the, the, the engine, the 3D world, the camera. Well, to be honest, Three.js has so many classes, studies, you know, it's, it's, it's too many for mention here. Um, so the lightened, uh, shadows, uh, yeah. Basically everything that you need.
RD Mm-hmm. The short, shortcuts to calculating the pixels, right?
JT Exactly, exactly. And, and it's really flexible. So if you want a scene, let's say, with an orthographic camera, you can do it. If you want a, a– I don’t know, if you want to add post-processing, you can do it. If you want contrast, like, add a contrast tool to, to the book you are, you're seeing. What– for the book and stuff too. All that comes with Three.js. Uh, in fact, all these, uh, extra layers that I, that I mentioned before, like Tres.js, react-three-fiber, they use all the classes of– basically all the classes of Three.js.
RD Mm-hmm.
JT So it's a easier way to write Three.js code, basically.
RD Mm-hmm.
JT Um, something very important to mention is that this, uh, frameworks – react-three-fiber, Tres, uh, develop – they also come with their own ecosystems of many of the abstractions, like, to, to be more specific, like, uh, I don't know, I want water, right?
RD Mm-hmm.
JT I want to put water there. I just import a component I already made that someone had done–
RD Right.
JT –and put the water. That, that kind of stuff. Like, I want, I don’t know, a sky and to just drop the component and tweak this, the volume that you want it.
RD Right.
JT So, so again, it's really, it's really accessible. These, these, these things.
RD Yeah. So you can sort of just easily drop in, you know, pieces of, uh, of a scene. The, uh, the sort of Tres.js, is it, uh, just to sort of, to get a sense of it, is it, is it, like, getting the Three.js stuff in a sort of idiomatic way for the frameworks?
JT Yeah, yeah, yeah. In clarity way and component stuff. Yeah. Yeah, yeah, yeah.
RD Okay. So were you a, a Vue fan before you worked on this? Or?
JT Yeah, yeah, yeah. I'm a, uh, Vue lover. Yeah.
RD What, what do you like about Vue?
JT When I started, I always loved the frontend. I, I, I started coding in the university in 28, uh, with, you know, with basically PHP and no frontend framework. JavaScript was, I think, to, to tweak stuff, little stuff. Uh, but I, I love how what you can do with CSS. After that, I started to work, like, more professionally and I discovered Vue. Um, since then, I just, I just fell in love. I, I don’t know, I really enjoy how fast and how flexible is Vue. Then I have the opportunity to work with React for two years and, you know, to have the comparison and everything. And I don't think React is a bad thing; in fact, in the open source world, React is, has an amazing ecosystem.
RD Right.
JT Uh, but I just prefer the Vue style. I don't know if it's a bias for starting with Vue or just–
RD Maybe.
JT –you know? Yeah, it could be. It could be, but, but I actually do–
RD You never forget your first love, right?
JT Yeah, exactly. So, so I don't know. I just love Vue. I just like the, the way Vue works and–
RD Mm-hmm.
JT –and, yeah.
RD Yeah. Do you think it's, uh, easier– I mean, you might be biased for this, but do you think it's easier to implement the 3D, uh, graphics in Vue over other, other frameworks?
JT Do you want me to answer that? Uh, I would say, “Yes!”
RD Yeah, of course. Right. Um, but do you think there are particularities about Vue that make graphics, uh, easier?
JT I think, I think they are always optional. Let, let's be honest here.
RD Mm-hmm.
JT They are just more comfortable to work. But, if you take Reqtify, uh, Threlte, Angular Tree, they are really similar. Really, really, really similar. So, so in that case, it's basically what you, what you want. I don't think there is a advantage of using react-three-fiber over Tree–
RD Okay.
JT –except the ecosystem.
RD Mm-hmm.
JTJust, just start with that because the, the react-three-fiber ecosystem is, is bigger. Um, but that is also because they were the first, they have, you know, more contributors, stuff like that. But except that, I don't think there is, you know, an advantage of using one or using the other one.
RD Mm, okay. So, you know, to take a, you know, a little peek into a crystal ball into the, the future, do you think there will ever, ever come a time where the, the web browser is the primary delivery system for 3D games?
JT Hoo, I would love that. Yeah.
RD Yeah?
JT I, I, my, my, my, I don't know if a prediction, but my, what I want to see is more, uh, of this special, special, um, 3D engines like Unity out, out of the web. Like, Unity, Unreal deliver more content to the web. Like, I would love that. I know that there is some experiments transporting the Unreal engine to the web. It was a game. Uh, I try it, it looked good, but it takes ages to load.
RD Oh, yeah.
JT Because it was too, too heavy.
RD Right.
JT But I, I would like to see more of that in, in the web. I don't think we need to be reinventing the wheel, in this, in these things. I mean, I don't think we need to come with one big, uh, engine.
RD Mm-hmm.
JT You know, reading in JavaScript for the web, if there are really good stuff like Unreal Engine or Unity or, or Godot.
RD Yeah. Yeah. I mean, I think the– it does seem like the barrier will be the, the load– the how big of a, a download you need to get there.
JT Yeah.
RD Um, do you think that there could be a sort of, like, server side of that and just do the rendering on the, uh– I mean, I, I know, uh, Stadia tried to do the games-as-a-service, uh, but I don't think that took off as well.
JT Yeah, no, I, I, that is a, a very good point. I, I honestly, I don't know. I don’t know.
RD Yeah. That's fair. Um, do you have a favorite project, uh, either of yours or somebody else you've seen that, that uses, uh, Tres.js?
JT Uh, again, this is bias. I love my, my creative lab.
RD Uh huh.
JT I got plenty of examples, all done with Tres.js and, you know, it's, it's just a lot of examples that I just do in my, in my free time. They're not the better code, but definitely if you, if you want to learn more what you can do with Tres, you can check it out with that. And, again, everything that you can do with, uh, with Three.js, you can do with Tres.js. So. So yeah, that amazing stuff done with Three.js. Amazing.
RD Okay.
JT I will, I will just mention something, which is in the web. We are point in which we create 3D experience for e-commerce, right? For, for just a portfolio, for stuff like that. It's not just for games.
RD Sure.
JT Uh, that, that point, that, that environment, creativity, it's, you know, opened the eye for many things. If– it's not the same, if you just show, show a, a photo of the product, let's say a, a shoes right, a Nike, shoes whatever, and now you can actually see in, in, in the space of 3D. So it's, it's, it's a, an open door for many things, and I would love to see more of that.
RD Yeah, I think a lot of people would. I think that'd be a very cool implementation.
[music plays]
RD Alright, everyone. Well, it's that time of the show where we, uh, shout out somebody who came on to Stack Overflow and dropped a little knowledge. Today we're shouting out, uh, a Lifeboat Badge winner. User zwol came on to Stack Overflow and dropped an answer that was so good, it got a score of 20, and he dropped it on a question that had negative three or less at the time. The question was: how would you write the equivalent of this C++ loop in Rust? So, if you're curious, check it out. It'll be in the show notes. I'm Ryan Donovan. I edit the blog, host the podcast here at Stack Overflow. If you liked what you heard today, or, if you have a topic you think you should cover and maybe you should be the one to talk about it, email us at podcast@stackoverflow.com. If you like what you heard today, leave us a rating, a review, and if you wanna connect with me, you can find me on LinkedIn.
JT I am Jaime Torrealaba, um, I'm senior software engineer in Push Security, and you can contact me in X by, uh, @jaimebboyjt. It's quite similar to Jamie in English, and JaimeTorrealba in my GitHub.
RD Alright. Thank you everybody, and we'll talk to you next time.
[outro music plays]