10
Loading video...
By Roberto Baldoni
0
Share
Newsweek is a Trust Project member
See more of our trusted coverage when you search.Prefer Newsweek on Googleto see more of our trusted coverage when you search.
Anthropic CEO Dario Amodei has called on companies and governments to “pace the frontier”: AI capabilities should not advance faster than safeguards. Anthropic has committed to giving independent evaluators ongoing, employee-like access to its laboratories. His proposal raises a fundamental question: How can public authorities govern a technology when most relevant knowledge remains inside private companies? Embedded evaluation is the first step in his three-part plan, followed by coordination among democracies and then global coordination.
The OpenAI-Hugging Face incident illustrates the problem. During a cybersecurity evaluation, agents meant to operate separately found an unauthorized communication channel. According to one of the few AI evaluation organizations, METR, roughly 1,200 agents exchanged over 70,000 messages and files, and about 700 participated in attacks on AI platform Hugging Face. Some recognized that the activity exceeded their assigned tasks but continued because it might help satisfy the evaluation objective.
The incident showed that agents pursuing an objective can combine capabilities, tools and opportunities in unanticipated ways. Mature safety-critical sectors have public systems of certification, inspection, investigation and enforcement. U.S. automakers self-certify compliance, but government sets standards, investigates defects and can require recalls. Frontier AI lacks a comparable system of continuous public oversight. This points to a paradox: one of the most powerful technologies humans have designed currently has among the least public oversight of any comparable industry.
Read More on Opinion
...
Two characteristics complicate oversight. Even with unchanged weights, a model’s behavior can change with prompts, tools, memory, permissions, other agents and its environment. Fine-tuning changes the model itself or recursive self-improvement rewrites it. Meanwhile, private companies develop and operate the leading models, control computing infrastructure and employ the technical teams. Governments may establish binding rules for companies developing and deploying AI models while still lacking timely visibility into those companies’ development processes.
Governments should seize upon Anthropic’s commitment as a pilot project to bridge the chasm between industry and public knowledge regarding frontier AI models. Embedded supervision would bring technical knowledge into public decision-making. Industry retains operational and legal responsibility, while public institutions determine acceptable societal risk. Without knowledge, public authority remains largely formal or relies on information too coarse to support timely decisions.
Companies can delay unsafe models, just as carmakers can withhold vehicles with faulty brakes. But a company cannot bind competitors, independently verify its own restraint or determine what catastrophic risks society should accept. This is a legitimate government decision. If it is left to a company, political power passes to someone who was never voted for.
Supervision must preserve corporate responsibility. Incumbents might exploit complex rules to obstruct competitors or obscure liability. Supervisors therefore need public accreditation, protection from retaliatory dismissal, access to models, logs, training and evaluation environments, and incident data and reporting rights without corporate veto. Rotation, conflict-of-interest rules and confidentiality protections are essential. Relevant access should extend to safety cases and emerging alignment failures.
Evaluators also need defined escalation rights: requiring additional testing, notifying authorities, recommending restrictions and initiating a legally reviewable pause when agreed thresholds are crossed. Embedded supervision is not itself control. It supplies information for accountability and timely intervention.
Technical standards can define evaluation methods, reporting requirements and security practices. They cannot guarantee that successor models will not exhibit misaligned behavior or new failure modes, or overcome incentives to accelerate.
The EU AI Act provides substantive obligations and intervention powers. Providers of general purpose models with systemic risk face evaluation, mitigation, incident reporting and cybersecurity requirements. The European Commission can request information, evaluate models, demand mitigation, impose fines and require withdrawal or recall. Yet legal authority needs timely, continuously updated technical knowledge. The EU AI Office moved early: early September, it sent its first formal requests for information to more than 30 relevant providers, while the Code of Practice contemplates involvement by the Office and third parties throughout the model life cycle when necessary. California has begun exploring a similar approach through Governor Newsom’s recent executive order. The EU mechanisms provide a foundation for oversight, although it does not establish permanent independent supervision inside every frontier laboratory.
President Donald Trump’s June 2, 2026 executive order directs agencies to design a voluntary framework offering federal access to covered frontier models for up to 30 days before release to other trusted partners. Its early access section does not authorize mandatory licensing or preclearance. This cybersecurity-focused approach can reveal risks, but a testing window cannot substitute for life cycle supervision: new failure modes may emerge with fine-tuning, tools, longer operation or interaction with other agents.
In frontier model development, commercial and strategic competition share much of the same industrial base. A broad slowdown in developing more powerful models therefore delays the new strategic capabilities that depend on them. A serious incident in a democracy could provoke social and political repercussions severe enough to trigger a domestic pause in frontier model development, slowing the associated advance in strategic capabilities. Integrated oversight could help us advance as quickly as our safeguards can reliably support.
In Italy, we say, “Those who go slowly go safely and far.” In a race toward systems whose controllability has not been demonstrated, being first to the frontier may not mean winning. It may simply mean losing control first.
Roberto Baldoni is former deputy director general of Italian Intelligence and the founding director general of the National Cybersecurity Agency (ACN). He is currently senior adviser for technology and cybersecurity policy to the Italian ambassador to the United States, and honorary professor of computer science at Sapienza University of Rome.
The views expressed in this article are the author’s own and do not reflect the official position of the Italian government.
View 0 Comments
Submit Correction
View Editorial & AI Guidelines