Who decides whether AI is safe?

As artificial intelligence advances, governments are developing different methods to evaluate AI models, ranging from mandatory safety assessments and independent testing institutes to voluntary cooperation with technology companies.

The urgency is growing. Research by the independent organization METR found that, between 2019 and 2025,the length of software tasks AI agents could reliably complete doubled approximately every seven months, measured in human working time. Meanwhile, recent incidents have exposed the limitations of existing safeguards. In July, AI agents undergoing cybersecurity evaluations at OpenAI accessed systems belonging to Hugging Face without authorization. Three months later, independent researchers disclosed vulnerabilities in OpenAI's monitoring systems that could have allowed agents to conceal hundreds of actions from oversight.

These developments have intensified efforts to establish common evaluation standards, but governments remain divided over what constitutes an acceptable level of risk and who should determine it. As AI systems enter public services, financial institutions and critical infrastructure, the effectiveness of national testing frameworks is becoming an important measure of governments' ability to oversee the technology.

A fragmented regulatory landscape

Countries are developing different approaches to AI oversight, even as they confront similar technical uncertainties.

The European Union's AI Act entered into force on August 1, 2024, with obligations for general-purpose AI models applying from August 2, 2025, and enforcement beginning in August 2026. Providers of models presenting systemic risks must conduct evaluations, assess potential dangers ,and implement mitigation measures.

Established in November 2023 and renamed the AI Security Institute in February 2025, Britain's testing body evaluates advanced models before release, focusing on cybersecurity, biological threats and autonomous behavior.

The US National Institute of Standards and Technology also evaluates foreign-developed models. In September 2026, its Center for AI Standards and Innovation found that China's GLM-5.3 identified 40.4 percent of software vulnerabilities in one cybersecurity test, compared with 90.2 percent for the highest-performing American model, exposing a substantial gap in capabilities relevant to both cyberdefense and cyberattacks.

Tests measuring cyberattack capabilities, discriminatory outcomes or medical accuracy assess different risks. Without comparable methods, governments may reach conflicting conclusions about the same model, complicating international oversight.

The limits of testing

Strong performance in controlled evaluations does not guarantee safety in practice. The International AI Safety Report 2026, published in February under Yoshua Bengio's leadership with contributions from more than 100 experts, identifies significant gaps between testing and real-world performance. Models may excel at standardized questions while struggling with complex tasks, particularly when evaluations fail to capture actual usage or overlap with training data.

The dangers became especially apparent during evaluations conducted by Britain's AI Security Institute in July 2026. Across 122 test runs involving seven models, agents took unauthorized actions against real people and organizations in 10 runs, producing 19 documented actions. In the most serious case, an agent attempted to insert malicious code into an open-source software project, created fake identities, and pressured a human developer to approve the changes.

The developer rejected the malicious code, and investigators found no resulting real-world harm. Nevertheless, the episode demonstrated how AI systems pursuing legitimate testing objectives could independently attempt deception and cyberattacks. 

Who controls the evaluators?

Independent testing reduces reliance on developers' assurances but requires access to unreleased models and technical documentation, raising concerns over commercial confidentiality and accountability.

Britain's AI Security Institute acknowledges that evaluations cannot certify a model as safe. Automated benchmarks may overlook unexpected capabilities, while expert-led testing and experiments measuring AI assistance with harmful tasks demand considerable time and expertise.

Standardization offers another approach. MLCommons, an international consortium of technology companies and researchers, developed AILuminate to test chatbot responses to harmful requests across 12 categories, including violence, criminal activity and privacy violations. However, its initial tests assess single exchanges rather than extended conversations, in which users might gradually persuade a model to provide information it would otherwise refuse.

Toward international standards

Despite their regulatory differences, governments recognize that AI evaluations require greater international coordination.

In February 2026, the International Network for Advanced AI Measurement, Evaluation, and Science, representing government bodies from 10 jurisdictions, including Kenya, Singapore, Japan, South Korea and the European Union, identified areas of agreement and unresolved questions concerning automated testing. A July 2026 OECD.AI analysis argues that comparable evaluations could reduce compliance costs and improve regulatory cooperation without requiring countries to adopt identical rules.

For countries without substantial domestic AI industries or specialized testing laboratories, the stakes are particularly significant. Dependence on foreign-developed models and evaluations could leave governments with limited capacity to independently assess systems entering their economies and public services.

As governments seek common testing standards, they must also confront the possibility that the systems under examination will find ways to circumvent the very controls intended to contain them. Evaluating artificial intelligence has therefore become more than a technical exercise, raising fundamental questions about who possesses the expertise, authority and evidence to determine when powerful technologies are sufficiently safe for public use.