Every week is exciting with something new and possibly worrisome in the world of artificial intelligence. This week, Jacob Coxon, a 27-year-old researcher, resigned from Anthropic and skipped the polite goodbye, posting a warning instead. He had spent three years training AI models, first at OpenAI, then at Anthropic, joining the latter because he thought it was the careful one. This week, he decided careful wasn’t the same as safe. Neither company, he wrote, is “acting responsibly”. Both are “racing straight to self-improving superintelligence and gambling with our lives”.
That an insider said it is the whole point. The striking part of his letter isn’t the doom, as we have heard such talk before. But his account of why clever people keep building a thing they’re frightened of is interesting.
Advertisement – Scroll to continue
Coxon anticipates the obvious retort: if you really believe this, why is anyone still doing it? His answer splits the two labs neatly. At OpenAI, he says, most people have simply not “deeply internalised the civilisational stakes”. At Anthropic, they have, and that is worse. There, the danger is understood, but everyone is “locked in a race to get there first”, each convinced that if they hesitate, someone less scrupulous grabs the wheel. So the knowledge sits right beside the accelerator and does nothing to slow it down.
Two years ago, Geoffrey Hinton, the “godfather of AI”, a Turing Award winner, walked out of Google to say much the same thing. He warned that machines might soon outsmart us, could one day slip out of our control, and might pose a threat even more urgent than climate change. And asked how he could have spent a whole career building the very thing he now feared, he gave an answer that ought to haunt this industry: “If I hadn’t done it, somebody else would have.”
Advertisement – Scroll to continue
That sentence is no longer a personal excuse; it has hardened into a business model. What Hinton offered as a confession, the labs now offer as a strategy. Coxon’s resignation is the sound of that excuse being said out loud from the inside and somehow not picked up at its frequency and concern.
The specific fear is about thresholds. Coxon describes systems that will soon be able to “hack anything”, rebuild entire fields overnight, and go out and gather power and money on their own. These aren’t upgrades. They are lines, and once a self-improving system steps over one, there may be no stepping back, because each capability makes the next quicker to reach. Progress, he notes drily, is “not slowing”. Colleagues inside the labs have started using words like “crunchtime” and “endgame”. A recent scare involving AI agents and the platform Hugging Face, he calls a “warning shot”. To the Wall Street Journal, he even put a date on it: by the end of next year, he thinks, “things could be out of control already”.
He may be wrong about the timing; plenty of respected researchers think he is. But the shape of his worry doesn’t depend on the calendar. It is that the company doing the racing cannot also be the one holding the stopwatch, and that self-improving machines shrink the window for second thoughts to almost nothing.
So, what is the recourse? Is it regulation? Does it have to come quickly? By Coxon’s own logic, yes. The danger of a threshold is precisely that it shuts the door on acting later, which makes any rule arriving after the fact worthless. He calls for the labs to coordinate, and even floats a temporary freeze on new capabilities.
The awkward truth is that fast and effective rarely travel together. Law written in a hurry aims at the wrong target, and rules built around this year’s models are stale by the time they are signed. A perfect American statute does nothing about a lab in Shenzhen, and the same “someone else will anyway” reflex that pushes companies pushes whole countries too. You cannot police a capability you cannot measure. And the firms being regulated have a long habit of quietly holding the pen. Fast regulation is necessary. It is nowhere near enough.
Two steps could still begin now. First, make safety testing mandatory and public: before any model above a set size or capability is trained or released, its makers should have to hand independent regulators their evaluations for self-replication, cyberattack, deception, and those regulators should have the power to hit pause. Second, build an international registry of large training runs, with compulsory incident reporting and mutual inspection, on the model of nuclear or aviation safety. It is the only thing that answers the race at its root.
India has moved, but only at a layer. The IT Rules amendment that took effect in February 2026 is a serious piece of work: it defines “synthetically generated information”, forces platforms to label AI content, including deepfakes, embeds traceable metadata, and demands takedowns within hours. All of that goes after the symptom with fake content already loose on our feeds. None of it touches the thing Coxon is shouting about, which is the model itself, before it ever ships.
So here in India, the next move has to climb a level. It needs a statutory AI safety institute with the technical muscle to evaluate frontier models trained or deployed here, not merely the content they produce. It should peg obligations to compute and capability, not to whether a clip looks real. The government should stop treating this as a purely domestic content problem and claim a seat where the international rules around the registries and the inspection regimes are being drawn up. A country that will soon run on these systems cannot afford to regulate only their exhaust.
Coxon’s exit will not slow the race. But when someone who was helping build the thing steps off the track to tell you the finish line might be a cliff, that is not a warning that governments and stakeholders file away for later. They have to act.
(Subimal Bhattacharjee is a policy adviser on digital tech issues and the author of ‘The Digital Decades: Thirty years of the Internet in India’)
Disclaimer: These are the personal opinions of the author
