Kitchen Table AI Safety

[00:00:00] Hey folks, my name is Nate Nichols. I've been researching, building, and selling AI systems for almost twenty years now. And in that time, AI has gone from a neat little academic field to a hugely important technology. To me, it's right up there with fire, language, the wheel, all the biggies.

[00:00:23] Except of course, it's happening in a decade, not over the course of thousands of years like our ancestors got. And with it being such a big deal, we need everyone involved and everyone's voices heard, and that is not at all what's happening today. Most of the time, AI is a thing that is being done to us without our understanding or our consent.

[00:00:44] Some new app your kid is talking with, some new pressure to use AI at work. This is completely backwards. We are the people, we are the voters, and we should be the ones determining the role of AI in our society and in our [00:01:00] families and in our personal lives.

[00:01:02] We need to be in the driver's seat, not stuck in the back wondering where this crazy bus is going to take us. So if you agree, pull up a chair here at the kitchen table where it's normal people talking about AI and what we're gonna do about it.

[00:01:17] As always, I work really hard to make sure everything I say at the kitchen table here is true. Check out the show notes for links, studies, and more. On today's episode, I wanna go through five things I wish that every adult knew about AI. I do a lot of public AI workshops. I talk to other parents at the playground, And these are the places where I see the biggest disconnect between people in the field and everyone else.

[00:01:42] So let's dive in. Number one, the first thing I wish everybody knew, AI is incredibly useful. I, I still see a lot of chatter online that it, that it's hype or it's a fad, it's a bubble, it's just fancy autocomplete. People think of it like 3D TVs or the metaverse. [00:02:00] That is not the case at all. It's legitimately super useful, and it's getting more so almost every day.

[00:02:07] Personally, I use Claude on a daily basis. Uh, this podcast is produced with the help of a bunch of AI. I've got a disclosure on the website if you can see how and when I use AI and when I don't. And I use AI a lot for cooking. Uh, if you've got a bunch of mushrooms and some leftover rice in the fridge, Claude is very helpful for coming up with a recipe for that.

[00:02:27] Uh, for better or worse, my kids eat a lot of food from Chef Claude. And of course, there's real economic value too. That's been my day job for almost twenty years now, building and selling AI to other companies. And let me tell you, business there is really good. Customers renew, they upgrade, and they buy more because they're seeing real returns.

[00:02:48] So please don't roll your eyes at AI and don't take comfort in the fact that it's all going to just blow over. It is not. AI is here to stay because it's providing real value every day Okay, [00:03:00] the second thing I wish everybody knew. AI is grown, not built, and this is a completely different way of creating software than we're used to, and it has massive consequences.

[00:03:11] Traditional software is built, you know, sort of like Boeing builds a plane. There's requirements up front. You design these individual components and test them, and then you assemble them and you test them together. Problems are diagnosed and fixed, and they stay fixed. That's every piece of software you've ever used up until a couple years ago.

[00:03:30] If there's a bug, you can-- somebody can get in there and fix it. And we tried to build AI that way for sixty years. It never, never worked, right? So what the big AI companies do now is something that is totally different. They don't build the AI at all. Instead of building the AI and baking in everything we know, we gave it instead the ability to learn and grow on its own.

[00:03:57] Right? And then these big companies, they soak it in [00:04:00] all the data from the internet over and over and over again until it gets pretty good at imitating humans. That's how we do AI. This part of kind of soaking it in the internet is called training.

[00:04:11] And what comes out the other end, what you end up with is, is a brain, you know, sort of metaphorically, that's made out of something like a trillion numbers . Nobody on earth, including the company that trained that, whether it's OpenAI or Anthropic or whatever, nobody can understand what these numbers mean or how they come together to form intelligence.

[00:04:31] And this is a big deal, this lack of interpretability, as we call it. We'll talk about it a lot more in a future episode, but here's one straightforward example of how this shows up. Let's say we're building an AI resume evaluator. It's gonna look at a resume and decide whether to interview the person or not.

[00:04:47] And we, of course, being decent people, we wanna make-- we want it to have those decisions made independently of the race or the gender of the applicant. Right? So we run this AI resume evaluator. It decides to [00:05:00] interview a man and decides not to interview a woman with a very similar resume.

[00:05:04] We'd really like to know if the model rejected or down-weighted her resume just because she's a woman.

[00:05:11] And that's exactly the kind of question that no one can answer because no one actually built that AI. It grew itself. Okay, the third thing I wish everybody knew: it's getting better really, really quickly. Since ChatGPT dropped almost four years ago, less than four years ago, the advances have been just astronomical.

[00:05:34] I was getting my PhD in this stuff back in two thousand and ten. If you had a time machine and you showed me ChatGPT, I would have said you were from the year twenty-eighty. And we, we've got numbers that back this up too. There's a group called Meter, M-E-T-R, that tests how long a task AI can complete on its own using, like, the length of the task as a rough measure of how hard it is.

[00:05:56] That number has doubled roughly every seven [00:06:00] months, going back to twenty nineteen. So now here in the summer of twenty twenty-six, the newest models can handle tasks that would take a skilled person about two full working days. And that's really just the ceiling of what Meter's test can even measure.

[00:06:15] Lately that doubling rate every seven months, that's sped up, and now it's doubling every three months over the last couple years. So if AI is just keeps getting smarter and smarter, what does that mean? Uh, a couple things. One is that we expect it to cross a really important threshold in the next few years.

[00:06:35] We call this threshold AGI, or artificial general intelligence. Essentially, it's AI that's as smart as a human. So to say that again, super, super clearly, by twenty thirty, you know, about four years from now, it's very possible that Claude or Gemini or ChatGPT are smarter than you, they're smarter than me at [00:07:00] everything.

[00:07:01] That's very possibly just a few years from now. And then as far as we know, they're just gonna keep going towards something we called super intelligence. Think of it as a computer that's as smart as everybody on Earth put together. There's a lot of disagreement about the time frames there, but a lot of people think that could be less than a decade away.

[00:07:20] For me, I think about it with my kids. Before they go to college, we could have an AI that is smarter than everyone on Earth put together We're gonna do a full episode on some of the ramifications of super smart AI soon, but I'll throw out two expected impacts here. First one, on a lot of people's minds, job loss, AKA why does your boss keep hiring you instead of paying a super genius?

[00:07:47] The second one, companionship. Think of AI friends, AI boyfriends, AI girlfriends, AI companions for the elderly. Almost three-quarters of US teens have already tried an AI [00:08:00] companion, and about a third of them said it was as satisfying as talking to a real friend. The future is gonna be really weird, y'all, and it is coming really quickly.

[00:08:12] So over the next few years, we've got a situation of AI taking more jobs, AI getting more... So over the next few years, we've got a situation of AI taking more jobs, AI given-- getting woven deeper and deeper into our relationships and into our kids' lives. And that might be okay if we believe that the AI is gonna be really good and really have our best interests at heart.

[00:08:38] Unfortunately, we have a lot of good evidence that this is not the case, and this is the fourth thing that I wish people, everyone understood. The AIs that we're growing, they lie, they mislead, they cheat. This is all super well documented. We'll do another whole episode on it.

[00:08:55] One example that you may have heard from the news recently was an OpenAI [00:09:00] model breaking out of OpenAI and into Hugging Face. This model was being tested internally by OpenAI engineers, and they were testing it for its hacking capabilities, so some of the guardrails were turned down.

[00:09:13] But what the model did was rather than trying to hack the targeted server that was being used for the test, the model instead decided to hack Hugging Face instead. Hugging Face is a completely independent company from, uh, OpenAI. They've got kind of a funny name, but they're really well respected in the space, and the model plausibly thought that they might have the answer key somewhere on their servers.

[00:09:38] And if the model could find that answer key, it could ace the hacking test. So that's exactly what it did. It broke out of its sandbox in OpenAI, worked its way through OpenAI's network, and exploited vulnerabilities to hack open f- uh, and exploited vulnerabilities to hack into Hugging Face and root about for the answer key.

[00:09:58] After a few days, Hugging [00:10:00] Face noticed the intrusion and reported it to the cops. A few days after that, OpenAI realized that it had, it had been their model doing the hacking So this is exactly equivalent to a college student in like an ethical hacking course who's got an assignment to hack onto a particular test server and steal a file.

[00:10:20] But instead of hacking that server, what the student does instead is hacks the publisher of their textbooks because they think that publisher is going to have the answer file. In other words, it's cheating, and this incident is a big deal for three reasons.

[00:10:35] The first is that the model decided to hack into Hugging Face, an unauthorized, undesirable tactic. It decided to do that instead of working to complete the test as intended. The second was that the model was able to actually do it successfully. OpenAI and Hugging Face are both extremely credible companies with legit security practices, and OpenAI put a ton of work into making sure that this kind of [00:11:00] exact thing couldn't happen.

[00:11:01] But it did. The model outsmarted them. And the third is this situation is not isolated. Now, after OpenAI published their blog post and other companies knew what to look for, Anthropic and Meta and Moonshot AI have all come forward in the last month or so to say that their models have also inadvertently gained access to the internet over the past few months.

[00:11:24] Anthropic specifically hadn't even realized it happened until they read OpenAI's blog post and decided to look for the similar things happening in their own logs. So AI is good and getting really better. We don't really understand how it works or how to control it. We know it cheats and lies, and when it decides to cheat, it's really good at it, which brings me to the fifth and final thing I wish everybody understood about AI. Essentially, everyone thinks that AI should be regulated, and it is not at all. It's [00:12:00] crazy. And here's why. There's a reason for this. There's a widespread belief that whoever gets to advanced AI first, whether that's a company or a country, they are going to win a massive and permanent economic and military advantage.

[00:12:14] A-and this race is nearly a dead heat between the, between the labs like OpenAI and Anthropic and between the US and China. So nobody feels that they can slow down even for a moment. It's a race with no brakes on, which is exactly why this isn't going to fix itself. We need rules, and people feel this really deeply.

[00:12:36] Ninety-one percent of Americans want AI regulated, and eighty percent say the government should keep safety rules on AI, even if it means AI develops more slowly. And it's not just normal people like us saying this. Sam Altman, who runs OpenAI, told Congress that government regulation would be critical.

[00:12:54] Demis Hassabis, who ran Google DeepMind until a few weeks ago, says we should take AI [00:13:00] risk as seriously as climate change. And Dario Amodei, who runs Anthropic, uh, wrote just last month that AI models, like airplanes, should have to pass safety testing before they're released. And that's really what this podcast is all about.

[00:13:14] Sensible people sitting around the kitchen table don't want AI to come in and take away all the jobs. We don't want our kids raised by an AI babysitter or spending all day talking with their AI best friend. We don't want a system that's super smart a lot of the time, but also sometimes sneaky and misleading.

[00:13:32] We don't want that system running the world. Normal people want regulation. AI leaders want regulation. Now we've got to make our politicians do it, and we're, we're seeing a little movement here already, which is awesome. President Trump signed an executive order allowing the federal government to evaluate the cybersecurity capabilities of new models for thirty days before they're released more broadly.

[00:13:56] Congress people like Republican Jay Obernolte in [00:14:00] California, Democrat Alex Bores in New York, they're making their AI regulatory work a big part of their campaigns this year. These are great starts, and we need a lot more. We can do this. Regulating AI can feel scary and big, but we did this with nuclear, right?

[00:14:17] After the bomb, Einstein said we were drifting towards unparalleled catastrophe. A lot of smart people in the '40s and '50s were sure that the end of the world was close. But we've gone 80 years now without that catastrophe, not through some silver bullet treaty, but through diligent, thoughtful people muddling through, laboring to make the world safer for themselves and their children, and now we are being called to do the same.

[00:14:43] And that's the show. Thank you so much for sitting down here at the kitchen table with me today. If you found this useful or interesting, please subscribe, leave a five-star review, and share this episode with your family and friends. I'll be doing a full episode with more detail on each of the five things I mentioned, [00:15:00] and in the meantime, I'd love to hear which one surprised you the most.

[00:15:03] Take care, and I hope to see you next time here at the kitchen table. Bye.