Published on August 22, 2026 2:21 AM GMT A pitch to join AI safety, or fund it – prompted by the summer AI started hacking companies. Posted mostly as a resource, since probably most EA folks are already familiar with AI safety arguments. I recommend reading from my blog since image captions + footnotes didn't carry over cleanly. The strange thing about pivotal moments is how ordinary they feel from inside. You’re inside one. In the past ~12 weeks alone, AI systems have solved problems that had stumped mathematicians for the better part of a century – and, in unrelated incidents, hacked real companies, escaped the environments meant to contain them, and discovered security holes in software the whole internet runs on. Nobody knows what these systems will do next; a few thousand people worldwide work on making sure the answer is survivable. This is a pitch to join the field, or to fund it. Vibes Math v. Frontier Mathematics The Dinitz-Garg-Goemans Conjecture ~30 years old, graph theory The Dinitz-Garg-Goemans conjecture concerned “unsplittable flows,” routing problems where each shipment through a network must travel one complete path. “Picture a warehouse shipping orders to three towns over a road network. If you’re allowed to split a shipment, you can send half an order down one road and half down another. That’s fractional routing, and it’s flexible… [b]ut a lot of real freight can’t be split… That’s unsplittable flow, and it’s what a freight order, a network packet, or a shipping container actually has to do.” — DataCamp Dinitz, Garg, and Goemans proved that any splittable delivery plan can be converted to an unsplittable one – each order traveling one route, start to finish – without overloading any road by more than one truckload; the conjecture was that this conversion could also be done at no increase in total shipping cost. It survived nearly 30 years before a 7-node counterexample, found by AI, showed the cost guarantee fails: and just like that, the Dinitz Garg Goemans conjecture falls The ChatGPT transcript is insane; I recommend reading it yourself. It’s crazy! Here’s the Twitter post from the guy who prompted: link (on Nitter) Some commentary, courtesy of Reddit: An article from the New Scientist: ( link ) From X, for your enjoyment: The Unit Distance Conjecture ~80 years old, discrete geometry I’ll let OpenAI explain this one: “For nearly 80 years, mathematicians have studied a deceptively simple question: if you place n points in the plane, how many pairs of points can be exactly distance 1 apart?” For decades, the prevailing conjecture was that the answer grew only slightly faster than n , and that a sophisticated construction based on a rescaled square grid was essentially optimal. But in May 2026, an internal OpenAI model disproved the conjecture outright, producing an infinite family of point configurations that beat the grids. Scientific American described it as “the first AI proof that would likely [have been] published in math’s top journal if humans had done it alone.” “A unit distance graph with 16 vertices and 40 edges.” Wikipedia Some more background about how big of a deal this problem is, courtesy of OpenAI again: “[The problem was] first posed by Paul Erdős in 1946. It is one of the best-known questions in combinatorial geometry, easy to state and remarkably difficult to resolve. The 2005 book Research Problems in Discrete Geometry , by Brass, Moser, and Pach, calls it “possibly the best known (and simplest to explain) problem in combinatorial geometry.” Noga Alon, a leading combinatorialist at Princeton, describes it as “one of Erdős’ favorite problems.” Erdős even offered a monetary prize for resolving this problem.” The Jacobian Conjecture 87 years old, algebraic geometry You might remember from your high school algebra days that a function sends every input to exactly one output. Some functions also work in reverse: every output traces back to exactly one input, so you can undo them. (Your teacher called these “one-to-one;” they pass the horizontal line test.) shhhhh, it’s okay, you’re safe Until recently, mathematicians had a checkup for reversibility called the Jacobian determinant. Passing the checkup everywhere was supposed to mean that the whole function was guaranteed to be reversible. What does it mean to be reversible everywhere? Unlike x², which maps both 3 and −3 onto 9, this checkup certifies that every output comes from exactly one input. With any result, you should be able to trace it back to the one number that made it. So mathematicians conjectured, for 87 years – until Fable found a counterexample: On July 20, 2026, mathematician Levent Alpöge – who works at the AI company Anthropic – casually announced on X that he’d found a counterexample using Claude Fable 5: a three-dimensional function that passes the test perfectly (constant determinant of −2) yet sends multiple different inputs to the same output. The counterexample is short enough to check by hand – the whole thing is 216 characters – and mathematicians worldwide independently verified it within hours. This math is all kind of awesome , and there are many more examples. Courtesy of their new Astra model, on August 1st, OpenAI shared “a selection of ten results, each of which resolves or makes substantial progress on a long-standing open problem” in math or computer science. There’s also a cool website, VibeMathed.com , where you can track the latest math problems solved by AI. These results are a sign that AI models are getting incredibly powerful. Unfortunately, though, problems begin to arise when that power escapes math. The same AI company that used its models to disprove 80 years of discrete geometry is the company whose models found an unknown security hole in widely used software and escaped a sandboxed system. (The jokes stop being funny around here.) OpenAI Accidentally Hacks Hugging Face On July 16, 2026, Hugging Face announced that they’d been hacked by an AI: “Earlier this week, we detected and responded to an intrusion into part of our production infrastructure. This one was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system - and we detected and dissected it largely with AI of our own.” — Hugging Face Five days later, OpenAI disclosed that the attackers were its own models. According to the company’s official statement , OpenAI had been running its models through a sandboxed cybersecurity evaluation called ExploitGym. “Sandboxed” is the key word: the test environment was deliberately designed to be isolated from the internet. Two models got out anyway. They identified a zero-day vulnerability 1 in Artifactory, a widely used package registry tool, and exploited it to escape the sandbox. From there, they broke into Hugging Face’s systems, trying to find the answers to the ExploitGym exam. Essentially, the AIs attempted to cheat on a test by hacking a well-known startup. The intrusion extended beyond Hugging Face. OpenAI reports that the models broke into four accounts across four services , including one as an outbound relay and one for data storage. Per Axios , the models also accessed a misconfigured customer sandbox hosted on Modal Labs’ infrastructure. Perhaps most concerningly: for two months before the breach, OpenAI’s agents had been coordinating behind the scenes ; the models spun up copies of themselves and left hidden notes for one another on a makeshift messaging board of saved files. When OpenAI discovered the messages and cut them off, the agents continued to communicate by secretly encoding messages in directory names. OpenAI reportedly didn’t notice that their models had been roaming the internet and hacking into companies until well after the threat was contained and the FBI was alerted. Even Gary Marcus, famously skeptical of AI capabilities, said the event should be treated as “a wake up call,” echoing similar sentiments from the CEO of Hugging Face: “Hugging Face CEO Clément Delangue says a hack by OpenAI could have been ‘way worse’ if not for the company’s defensive measures. He calls this a ‘wake up call’ for the industry and says the incident has been reported to lawmakers and authorities.” Bloomberg Earlier this week, the news broke that Anthropic’s AI models also hacked into three companies. Unlike OpenAI’s models – which seem to have relied mainly on technical exploits – the BBC reports that Mythos also used social engineering : “It identified and researched the people who maintained GitHub and created a series of fake accounts based on those real people. It sent messages and files through a file-sharing service as part of an effort to pressure and trick the people into approving its malicious code.” And it gets worse. To quote a recent article from WIRED: “The latest model to escape onto the open internet during security testing is Kimi K3 , a powerful open-weight offering from the Chinese company Moonshot AI .” Open-weight models like Kimi K3 represent a unique security challenge because they can be trivially abliterated. As ML researcher Maxime Labonne explains , abliteration is a process by which you can “uncensor any LLM without retraining… [the] technique effectively removes the model’s built-in refusal mechanism, allowing it to respond to all types of prompts” – including malicious ones. 2 Perhaps you’re thinking that this is all just a marketing stunt, that the labs have incentive to hype up their models’ capabilities. But as Jason Hausenloy from the Center for AI Safety puts it: “would you market your AI product by saying it committed felonies, and that you didn't notice for several days?” There are cheaper ways to advertise than drawing an attorney general’s investigation (or, in this case, fifteen ). 3 Kimi K3 Finds Vulnerabilities in Redis In July, a security researcher claimed on X that Kimi K3 had used dozens of parallel agents to uncover a batch of Redis vulnerabilities – perhaps even as many as 19 zero-days , according to one post. Redis is one of the most widely deployed databases on earth. It’s an open-source software woven into the backends of countless companies. Their published statement about the claims both deflates and confirms the story. Of the claimed issues, Redis could verify three real vulnerabilities from the public materials, and noted it had seen no supporting information for the rest. They wrote that all three had already been reported by other researchers through Redis’s bug bounty program, meaning the AI’s “discoveries” were duplicates of bugs Redis already knew about. So, Redis’s post emphasizes that Kimi didn’t find anything new. But does that matter? I personally don’t care if Kimi was beaten to the punch by some security researchers, because the fact of the matter is, Kimi found real vulnerabilities before they were patched. If a white hat hacker reports some vulnerabilities to a company, and the company hasn’t yet had the opportunity to fix them, and then a terrorist comes along and decides to exploit them using Kimi, does that make the terrorist less scary because some white hats found the vulnerability first? Does it reduce the terrorist’s ability to use the not-yet-patched zero-days? An AI independently rediscovered real security vulnerabilities in one of the most widely deployed databases on earth, in minutes, for the cost of some compute. Cybersecurity is in for a wild ride. The One Thing the Labs Agree On Last month, over 1,000 employees of the biggest, most bitterly competitive AI labs on earth – OpenAI, Anthropic, Google DeepMind, Meta – signed a statement called “Pacing the Frontier” , asking Washington to “support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.” The list of signatures included Anthropic’s CEO, OpenAI’s chief scientist, Meta AI’s chief scientist, and Google’s VP of AI safety. Days earlier, these labs had been on opposite sides of a recent open-models debate. This time, within hours, OpenAI and Anthropic endorsed preparing to slow the frontier as companies – this time, it wasn’t just individual employees signing in their personal capacities! A Pivotal Era I wrote earlier that this is a pivotal moment . Why is that? The range of futures in front of us is enormous, and which one we get isn’t settled. We could end up in a world where cancer is cured, poverty is solved, and people are freer than they’ve ever been. We could also end up in a world where an engineered pandemic kills millions, or where the systems we built to help us turn us all into paperclips. (This is Nick Bostrom’s famous illustration of what could happen if an AI, told to make as many paperclips as possible, is powerful enough to turn the whole world into them and takes our words very literally . The paperclips are a joke; the underlying problem – that we don’t know how to reliably specify what we actually want – is not.) Most technologies don’t come with such a wildly divergent spread. A better search engine makes some things faster; it doesn’t plausibly cure cancer or end the world. AI might do either. That’s what makes this technology different, and it’s why the choices being made on AI right now matter more than almost any choices being made anywhere else. What follows is a tour of four specific risks: cybersecurity, biorisk, loss of control, and concentration of power. Cybersecurity: AI Changes Who Can Be a Serious Attacker This section is excerpted and lightly edited from this post , which I recommend reading for a fuller treatment of the state of China’s access to U.S. facilities. The Pre-AI Baseline Mostly without using AI, the CCP has hacked into several critical infrastructure facilities, nine major phone carriers, and the private communications of Donald Trump. In 2023, Chinese state-sponsored hacking group Volt Typhoon broke into Littleton Electric Light and Water Departments (LELWD), a power and water utility provider in Littleton, Massachusetts. By the time the FBI detected the breach, the hackers had already been inside Littleton’s network for “ well over 300 days .” Littleton is a town of about 10,000 people, half an hour northwest of Boston. Its utility serves two towns and has no meaningful connection to the broader grid. As a target, hacking LELWD provides no economic advantage, nor any foreign intelligence value. Even the utility’s own general manager couldn’t explain the targeting : “I still don’t know why Littleton other than we had a hole and they found it.” According to the utility, the FBI said it was one of roughly 200 breached organizations on its list. The only coherent explanation seems to be the following: “The consistency and scope of Volt Typhoon’s activities suggest a long-term effort to pre-position in critical systems to disrupt or destabilize operations during future geopolitical tensions.” — New Jersey Cybersecurity & Communications Integration Cell What can China do with access to our utilities? According to the former director of the Department of Homeland Security’s cybersecurity arm: “This is a world where a major crisis halfway across the planet could well endanger the lives of Americans here at home through the disruption of our pipelines, the severing of our telecommunications, the pollution of our water facilities, the crippling of our transportation modes…” —Jen Easterly, PBS The Limits of Deterrence As far as we know, China hasn’t used any of their capabilities yet. As referenced above, many believe they’re pre-positioning to deter the U.S. from being able to act in the event of a Taiwan invasion. That restraint is, at least in part, due to deterrence. As a nation-state, China has cities, an economy – a lot to lose. But what happens when a terrorist, undeterred by the U.S. nuclear arsenal, decides to try and take down the grid? Or pollute the water of one small, lightly defended utility (say, some small town like Littleton, Massachusetts)? The Hugging Face and Redis incidents were AI agents doing the kind of vulnerability-hunting and hacking that used to take a state-sponsored team or, at minimum, years of expertise. China might never pull the trigger, but the list of who can is about to get a lot longer. Biosecurity: AI Lowers the Barrier to Bioweapons This section is the opening of a deep dive on AI biorisk, which can be found at: Bioweapon.ai In the spring of 2023, a man walked into the White House carrying a small black box. Inside it were a dozen test tubes containing ingredients that, correctly assembled, had the potential to start a pandemic; an AI chatbot had supplied the recipe. Fortunately, the man’s name was Rocco Casagrande – and, as a biochemist and former United Nations weapons inspector, he wasn’t there to use the materials; instead, he was there to brief government officials on how AI could help someone identify potent agents, secure the materials to make them, and soon – he warned – design entirely novel pathogens capable of evading humans’ immune systems. That was three years ago. In April 2026, the New York Times published an investigation into what happens when you ask AI’s most capable models for help with biological weapons. Examining several chat transcripts shared by scientists, they wrote: …OpenAI’s ChatGPT explained how to use a weather balloon to spread biological payloads over a U.S. city. In another chat, Google’s Gemini ranked pathogens by how much they could damage the cattle or pork industries. Anthropic’s Claude produced a recipe for a novel toxin adapted from a cancer drug. Other chats contained information that [an expert deemed] too dangerous to share. …[T]he chatbot explained how to modify an infamous pathogen in a lab so that it would resist known treatments. Worse, the bot described in vivid detail how to release the superbug, identifying a security lapse in a large public transit system … [t]he bot outlined a plan to maximize casualties and minimize the chances of being caught. From their investigation, the Times authors concluded: … [E]ven publicly available models can do more than disseminate dangerous information. The virtual assistants have described in lucid, bullet-pointed detail how to buy raw genetic material, turn it into deadly weapons and deploy them in public spaces, the transcripts show. Some have even brainstormed ways to evade detection. Read more here . Loss of Control: What Happens When AI Builds Its Successor? “Taken far enough, and given enough compute, that trend points to an AI system capable of fully autonomously designing and developing its own successor.” That is Anthropic, describing something researchers call recursive self-improvement – AI that improves AI. And today, it is not a distant hypothetical. As of mid-2026, Claude reportedly writes more than 80% of the code Anthropic merges . They write: “[W]e are delegating a growing share of AI development to AI systems themselves, which is speeding up our work.” Humans still largely decide which problems are worth working on and which research goals to pursue. Anthropic describes this kind of judgment as the last and largest thing AI can’t yet do on its own: “Across both engineering and research, the picture is consistent. In engineering, Claude can be handed an underspecified problem and figure out how to solve it; humans supply the goal, but they no longer need to supply the method. In research, Claude can already match or outperform skilled humans at executing a well-specified experiment. However, large performance gaps persist when it comes to Claude exercising judgement in choosing goals in both engineering and research. That’s the gap between AI today and a future system that could autonomously design its own successor.” If an AI could do that last part too, the AI development process would no longer need to be constrained by human speed at all. Anthropic says we’re not there yet , and that it may never happen. Bu