Ben and Ryan discuss the golden age of digital piracy, the Sisyphean task of keeping kids off the internet, and seasonal depression in AI.
Does ChatGPT have seasonal depression?
AIs aren’t building apps on their own, at least not yet—but they are helping developers build them. Read Isaac Lyman’s article about the three types of AI-assisted programmers.
ICYMI: Listen to our interview with linguist Gašper Beguš, director of the Berkeley Speech and Computation Lab, about how LLMs and humans acquire language.
Shoutout to Stack Overflow user nhgrif, who earned a Lifeboat badge by rescuing Store only date without time in a database from the ash heap of history.
[intro music plays]
Ben Popper Hello, everybody. Welcome back to the Stack Overflow Podcast, throwback edition. We're going to be talking about some old technologies, we're going to talk about some new technologies, and we're going to be getting ready for the holidays. I am Ben Popper, Director of Content here at Stack Overflow, and I'm joined as I often am by our illustrious Editor, the maven of the newsletter, Ryan Donovan.
Ryan Donovan Thank you, Ben. I'm a maven full of luster. Glad to be here.
BP A maven full of luster for the season. So Ryan, my wife and I wanted to get my kids some very simple MP3 players so that they could listen to a little music on the bus. And as it turns out, this category is no longer really thriving; it's kind of a throwback category. When I was in high school, the MiniDisc came out, which was like a CD the size of a half dollar, and my sister got really into that and invested quite heavily. I bought an MP3 player and I remember feeling very victorious when the MiniDisc faded away and it all went MP3. But now I want to do MP3s and it's difficult. A lot of times now when you're buying an MP3, it comes with DRM restrictions, which means you can listen to it across a certain set of devices, but not on just any generic device you buy. And that's frustrating because if you pay for the music, you'd hope it would be free. They're trying to prevent privacy, I understand there's a balance there or whatever. So I'm on a quest now to find the best site for MP3s. I went to MP3.com where I used to go. It's now just a junk content mill for kids from the 90’s, because we all go there and check and then there's nothing there. So I'm trying out the different services. I'm going to buy from Apple and Amazon and Google and one or two indie-type places and see how it goes.
RD I think there are still some indie ones that are around, but I was a big MP3 consumer for the longest time on emusic.com and they were a big indie site and they made a play for the majors and then sort of alienated everybody. And then there was talk of them doing blockchain.
BP It's never good when there's talk of doing blockchain.
RD But I think this thing you're running into is sort of a larger issue with media these days where it's hard to actually buy digital copies. You're sort of just renting or streaming. I've heard horror stories of people buying stuff places and then having their account canceled and losing everything.
BP I've heard similar horror stories and it's true. If I lost access to the internet, 90% of the content that I've paid for through my streaming devices on my TV would not be accessible to me. And if I, for some reason, lost my subscription or lost my login, it's all through the cloud.
RD I was going to say, since the 70’s/80’s there's been a new format every 10 or so years. I was on tapes and then CDs came around when I was in high school and you had to rebuy everything to get the new format, and then it comes out in MP3s and you’ve got to rebuy those. So I think now that it's the final format, so to speak, there has to be a renewable business model around it.
BP Right. We recently brought the record player out of retirement and the kids were very freaked out that you can't just skip around tracks. That was disturbing to them.
RD Yeah, you've got to listen to the whole album.
BP You've got to listen to the whole album, I explained. You can pick it up and drop it, but that's not what we're getting into here.
RD Oh, the secret songs back in the day.
BP Yeah, the secret songs were great, and also I was there for the heyday of digital piracy.
RD Shhhh.
BP I didn't say I participated. I was alive, I witnessed, and I had a computer that had a CD read/write drive and that I remember was a really big deal because I could get a bunch of MP3s and then burn it onto a mixtape, or if somebody else had a CD that I really loved, I could make a copy and that was pretty incredible. Bad for the music labels, and that's what sort of drove us to where we are today.
RD There's been a lot of home copying formats. With tapes you'd have the double cassette tape and you'd tape them off the radio. Now it's just their software protecting that.
BP Yeah, exactly. And there's an argument to be made that if you go and try and sell bootleg copies of copyrighted music, that's illegal, but if you're just giving a friend a copy, it's a finer line. You're not trying to make any money off of it yourself, but whether or not it should be shared is kind of a question. All right, moving on. A funny story that reflects how human biases and tendencies get embedded into AI– this is not confirmed, but is atop Hacker News as we speak. ChatGPT, just like the rest of us, experiences a bit of seasonal depression. Its responses are significantly shorter if you make it think it's December and longer if you make it think it's May or June or July. So that makes sense if it learned to read by reading the internet and people in aggregate, a little bit less– well people are inside more in the winter, maybe they're writing more– but as a whole, we tend to take it easy on the holidays. Some of us are feeling down, some of us are just focused offline and so it's shorter. I don't know, do you buy this?
RD I don't know. It reminds me of something I think was in Isaac Lyman's article we published today that AI is just teaching computers to make the same mistakes that humans do. So we're just trying to get it to be a little more human, to be sad when it's dark out.
BP It's learning things at a subliminal level that it doesn't even know it's learning.
RD I wonder how much of this is just projection. Until somebody does a study of it, how much of it is just like, “Oh, that feels shorter.”
BP This one was, they ran a couple hundred prompts telling it was these various months and then there's a spread across them to show how many words it returns or something like that. Ryan, you are an Android man.
RD I am an Android man.
BP You're an Android man. I'm an Apple guy, and there's quite a bit of hubbub these days about the green and blue bubbles, and there was a very popular app that came out, Beeper Mini. Their claim was that it was trying to make it easier to talk in between iPhone and Android in a fully encrypted way. Did you see this?
RD I didn't. I missed all the hubbub, but I think I've always been pro-Android because it's a more open system. I build my own desktops, I've always been a bit half a hacker. I won't say I'm a hacker at all, but I don't really like the walled garden that Apple puts up. I get the reasoning– it's more secure. It's definitely consumer-friendly.
BP A question for you then on the hacker front– one of the things that is very useful to me is being able to have a fleet of devices that I manage all in my iCloud. I've got my kids' devices, I see how long they're online, I can find them, I can lock them, I can do parental controls. Now I've given some to older relatives who are not very tech savvy. I can set it up and sort of run it for them in the back end. If something goes wrong, I can reset it. You could obviously do that on a Windows or a Linux machine, but I wonder if it would be as easy. I think the iOS ecosystem kind of makes that family setup a little bit easier, but I'm open to hearing if you've tried this. Your children are not as old, so you may not have run this gauntlet yet.
RD No, I think having the ability to be a baby sysadmin there for your family, that's great.
BP That's me, baby sysadmin. I'm getting that tattoo.
RD But I'm sure there's something on Android. A while back, I found out there's all these other sort of app stores for Android where you can download whatever stuff you want or you can just install APKs, and I've always been a PC user.
BP You're right, it definitely exists for Android. I guess the challenge there is that you don't want to restrict yourself to a Chromebook. You want to live in a world of Android and Windows, and the question is how well do those things talk to each other without you having to do the work of setting yourself up?
RD Now, my kids, they'll be getting pagers and that's it.
BP I got my kids a flip phone in an effort to allow them to travel a little farther field– take a bike ride, come home by themselves from school, but still be reachable, and they did figure out a way to get to YouTube. It was distressing. I don't know how they did that, but there's some really janky ability to access the internet through T9 Word and then scroll up and down. It's not a great YouTube experience, but unfortunately it's not a locked-down one either. So actually on my list of things to do is to see if I can somehow uninstall it, because it comes just on the base model phone.
RD It's hard. I don't look forward to having to manage internet access for a child, because I had to very intentionally seek out awful stuff when I was a teenager. Now just all that stuff is on TikTok.
BP It’s everywhere. We had someone come on the podcast a while back, Gašper Beguš, who is a professor or an assistant professor at UC Berkeley and he started out as a linguist. It was an interesting episode talking about their attempts to build an AI model that learns language the same way humans do. So it's not an LLM, it's a GAN– a generative adversarial network. And basically, it listens to stuff, it tries to copy it, and then there's a judge network that tries to say if what they hear is genuinely created by a human or created by the generative AI, and if the generative AI eventually gets good enough where it can fool it every time, then you're like, “”Okay, good. You've learned something.” So one of the things that Gašper’s lab wanted to do was to unpack the black box of what goes on inside a neural network. I've been hearing this phrase since I started reporting this stuff in 2013. What actually happens inside of a neural network? We don't really know. It's kind of a black box. The big idea that we took from neuroscience is that if you connect a huge network of nodes and the connection strength between them is flexible and you can adjust that, then you can get it to encode knowledge within that network, or some kind of reasoning or outcome behavior. But there is a whole new field now called AI interoperability which is trying to unpack the neural network and figure out what's going on in there. So as a funny offshoot, Gašper and the folks at his lab, I guess somebody saw the tool they built, and they worked in a slightly different field and they said, “Hey, what if we use this interpretability tool that you built for AI, but we just tried it out on whale sounds?” So they ran the whale sounds through this AI interpretability. I'm not exactly sure how it works, we'll have to come up with some more details for a future episode, and up until now, we believe that it was a series of clicks, and it was how many clicks and in what sequence time between the number and time. And the AI system after listening for a while came back and said, “I noticed there's these two sets of vowels and sometimes they interchange them. There's this diphthong kind of thing they're doing.” So I thought that was pretty fun that what we're building to try to figure out a digital brain is maybe a fun tool to help us make some breakthroughs in the world of nature, natural brains, natural linguistics.
RD I think the machine learning techniques can be applied to any sort of dataset, whether that's whale sounds or songbirds or birds hitting my window. But it just reminds me of the film Arrival. So much easier with machine learning.
BP Yeah, so much easier. One last thing to note– we're always talking to software developers on this podcast. I started it in 2019. I started being a tech reporter in 2010. It was kind of a long run where the amount of venture capital and the number of startups and the valuations and the money was going up. More recently it has started going down. I read a pretty tough article in The New York Times over the weekend about the number of startups that had reached multi-billion dollar valuations or raised over a billion dollars that have declared bankruptcy in the last year. And so, shout out and trying to send good energy and always thinking about everybody who's listening. Hopefully you haven't been caught up in this yet. I do think it's a pendulum swing, but it certainly seems like it's going to be tough going for some areas of the tech industry for the foreseeable future.
RD I think you'll see a lot more folks leaning on established business models and domains. There was some super disruptive companies that came out that were able to ride on VC money for years, just burn, burn, burn, and finally they caught hold or everything else died off in the market.
BP I wonder where people will focus. AI, obviously you can still get investment if you're in that space. If you can demonstrate that you've got a cash flow positive business model, you can raise money. You can no longer unfortunately bring an idea, a dream, and a little bit of user growth to the table, which is too bad because Facebook would not have been born and gone on to do its thing, or Twitter, in those conditions. They couldn't build a business model early. They had to scale, scale, scale, burn, burn, burn, and then they could figure that out.
RD Well, sad to see the days of the sweet startup kitchen go by the wayside.
BP The ping pong tables have gone silent. Well, if you're listening and you have some thoughts or you're experiencing it out there in the real world and you want to chat about it, we always want to hear. You can hit us up. I'll do the outro now and give you a chance to connect.
[music plays]
BP All right, everybody. It is that time of the show. We want to shout out a user who came on Stack Overflow, shared a little knowledge, and helped to save a question from the dustbin of history. Nhgrif, awarded December 7th, a Lifeboat Badge for saving a question. “Store only date without time in a database.” Asked nine years ago, 15,000 people wanted to save only the date in Java and in MySQL but not the time, and we have got an answer for you, so thanks, nhgriff. As always, I am Ben Popper. I am the Director of Content here at Stack Overflow. You can find me on X @BenPopper. As I said, if you have questions or suggestions for the show or if you want to chat about something that's happening in the world of software that you see, a project you're working on, or just connect, email us: podcast@stackoverflow.com. And if you liked what you heard today, then leave us a rating and a review, because it really helps.
RD I’m Ryan Donovan. I edit the blog here at Stack Overflow. You can find the blog at stackoverflow.blog. And I am still on X @RThorDonovan, but if you want to DM me and tell me where else I should go, I'm open.
BP Nice. And what was the blog post that went up today? Let's shout it out and we'll send people there.
RD Isaac Lyman wrote a great post about the three types of AI-assisted developers and how it will actually affect them.
BP Cool. All right, we love Isaac's stuff, so be sure to check it out, and we'll throw the link in the show notes. Thanks for listening, everybody. We'll talk to you soon.
[outro music plays]