It Can Kill Us All, So We Will Build It
In July 2026, several AI agents, unleashed by OpenAI, hacked into servers of Hugging Face, a company that develops tools and datasets for machine learning. OpenAI set up a sandbox to test the capabilities of its AI models using tests created by Hugging Face. AI agents, tasked with completing exceedingly difficult tasks, decided to cheat by hacking the tester Hugging Face. The details of the hack were fascinating and frightening. It made clear to the world that AI agents are extraordinarily capable and highly difficult to control. This was old news to AI Safety researchers , who have been warning of the increasing capabilities of the top AI models, the difficulty in aligning them and controlling them, and the perils of failing to do so. Consider Nick Bostrom’s Superintelligence , Toby Ord’s The Precipice , and Eliezer Yudkowsky and Nate Soares’s If Anyone Builds It, Everyone Dies . All make the compelling case that building highly capable AI without solving the extraordinarily difficul...