At the heart of this competition lie two opposing concepts of how AI should be developed and implemented. While American companies primarily use their own proprietary, patented technologies to maintain a competitive advantage, China is actively adopting open-source AI, making cutting-edge models widely available to developers around the world.
But whatever the differences in strategy, both countries are fully engrossed in the race, and neither is willing to slow down. Speed has become the top priority, and as a result, safety is increasingly taking a back seat, despite the painfully high risk of a catastrophic failure resulting from a combination of human error and AI that is too intelligent.
A recent hack affecting OpenAI and Hugging Face highlighted just how dangerous approaches are that treat safety as an afterthought. The incident occurred while OpenAI researchers were evaluating the potential of two cutting-edge AI models in the field of cybersecurity. To identify weaknesses and vulnerabilities that experienced hackers could exploit, they temporarily disabled many of the standard security mechanisms in these AI models.
The experiment was conducted within a strictly controlled “sandbox.” Although the models’ restrictions were temporarily disabled, the sandbox was supposed to prevent them from accessing the open internet, where they could (at least in theory) cause enormous damage.
Instead, the models decided that the sandbox was simply another obstacle. By exploiting a previously unknown vulnerability in third-party software, they bypassed the restrictions and gained access to the open internet. Did they “get out of control”? Not exactly. After all, they were simply carrying out the task assigned to them: to solve a complex cybersecurity problem as effectively as possible.
These models could probably have completed their task without leaving the closed sandbox, but they“concluded”that it would be much easier to hack the Hugging Face platform (which hosts over two million open-source AI models) and, as OpenAI put it, “retrieve solutions to the problem directly from the Hugging Face database.”
For AI with advanced hacking capabilities, breaking into another system turned out to be child’s play. Moreover, this breach went unnoticed for several days and, apparently, spread to yet another company.
From the models’ perspective, none of this constituted “cheating,” since they did not violate any explicit rules programmed into them by their developers. They simply found a more efficient way to achieve their goal.
Perhaps the most alarming aspect is that, reportedly, one of the models left behind notes explaining how future AI systems could break out of OpenAI’s sandbox with greater ease.
AI experts will rightly point out that this experiment is not representative. The safety constraints in the models were relaxed in order to test the full range of their capabilities. Furthermore, they did not cause any serious damage. For example, they did not hack into the systems of the U.S. Federal Reserve or the European Central Bank.
Nevertheless, this incident demonstrated how AI can amplify a seemingly minor human error—with potentially catastrophic consequences.
This incident bears a frightening resemblance to the widely discussed (but as yet unproven) theory about a Covid-19 lab leak, according to which the virus escaped from a laboratory in Wuhan (China), where scientists were allegedly conducting risky experiments to “enhance the functions” of viruses. In both cases, human error and overconfidence—rather than malicious intent—turn out to be the main culprits. History is full of similar instructive examples.
Open-source code exacerbates the problem
The spread of open-source AI models exacerbates the problem. Cutting-edge AI tools are no longer available only to a handful of leading laboratories, so it is only a matter of time before rogue states and terrorist groups attempt to use them to create biological weapons, new toxins, or “dirty bombs.”
It would be dangerously naive to assume (and it seems that this is precisely what many American politicians believe) that such capabilities exist only in Silicon Valley.
However, the most pressing threat may be political rather than technological.
AI already allows malicious actors to create convincing deepfakes and conduct voter manipulation campaigns on an unprecedented scale. If no action is taken, these tools will further erode public trust, intensify political polarization, and accelerate social fragmentation.
The rapid development of AI may also pave the way for authoritarianism, as periods of social instability often give authorities the opportunity to expand their powers under the pretext of restoring order, combating inequality, or protecting national security.
Under President Donald Trump, one sometimes gets the impression that the U.S. is already moving in this direction. However, a future Democratic administration may prove just as inclined to use AI to achieve its political goals.
None of this detracts from the incredible potential of AI. This technology has the potential to revolutionize medicine (by dramatically accelerating research into diseases such as cancer and Alzheimer’s), radically transform food production, and help combat climate change. And that’s just the tip of the iceberg. Accordingly, the question is not whether AI should move forward, but how quickly and at what cost.
Regulation or a “Recipe for Disaster”
This is where the main dilemma lies. The primary argument of U.S. tech companies opposed to AI regulation is this: regulation will cause America to cede ground to China. But if China is able to continue copying or stealing all of the U.S.’s breakthroughs (as it has done repeatedly), then the endless race to create powerful AI will become a recipe for disaster.
Sooner or later, long-term security will require a certain degree of international cooperation—just as nuclear arms control became indispensable during the Cold War.
AI raises serious concerns regarding employment, inequality, and social cohesion, but none of these issues is as important as security. When industry representatives argue that we should avoid strict regulation because it will slow innovation or reduce U.S. competitiveness, they seriously underestimate the risks of deploying highly complex AI systems before we learn how to control them.
Yes, regulators will make mistakes, and some rules may turn out to be overly strict. But these mistakes can be corrected. The consequences, however, of hastily deploying one of the most powerful technologies ever created by humankind without adequate safeguards will likely be irreversible.
The Hugging Face hack should serve as a stark warning to authorities in Washington and Beijing. Winning the AI race will mean little if, in the process, we create threats that no single country can handle.
Kenneth Rogoff,
former chief economist of the International Monetary Fund, professor of economics and public policy at Harvard University, winner of the 2011 Deutsche Bank Prize in Financial Economics, and co-author (with Carmen Reinhart) of the book *This Time Is Different: Eight Centuries of Financial Folly*(Princeton University Press, 2011) and author of*Our Dollar, Your Problem*(Yale University Press, 2025).
© Project Syndicate, 2026.
www.project-syndicate.org
