← All posts
Governance· · 4 min read

Everyone Wants Monitoring. Few Can Prove They Have It.

Three very different voices, one word. A departing Anthropic researcher, OpenAI's chief scientist and Bill Gates all called for monitoring in September. For assurance professionals, what does it mean?

Something unusual happened in September. Within three weeks, three very different voices arrived at the same word.

On 6 September, OpenAI’s chief scientist Jakub Pachocki published an essay, An Alien Mind, arguing that no lab has solved alignment and monitoring well enough to keep scaling at maximum speed for much longer. Two days later, Jacob Coxon, a pretraining researcher who had spent three years at OpenAI and then Anthropic, resigned from Anthropic in a public thread, writing that “neither company is acting responsibly.” Within hours, a current Anthropic alignment researcher publicly backed him. Then, on 27 September, Bill Gates told Meet the Press that self-regulation is not enough, that a kill switch on its own will not do, and that advanced AI needs mandatory safeguards, monitoring and records of how it is being used.

One builder who stayed, one who left, and one outsider with a long memory of how technology goes wrong. Different motives, different audiences, same diagnosis.

The word underneath the alarm

Most of the coverage read this as a debate about catastrophe, and the timelines and probabilities made that easy. Read it as an auditor, and it becomes something far more familiar.

Pachocki is saying the people closest to the models cannot yet observe them reliably. Coxon is saying people inside the labs do not trust the labs’ own commitments. Gates says voluntary commitments need to become mandatory controls, backed by records someone outside the company can inspect.

Any internal audit function would recognise those as three findings: a monitoring control that is not operating effectively, a governance culture where management’s assertions are not trusted by the people doing the work, and no independent assurance over either. Nobody in our profession would need to be told what comes next. We would ask for evidence.

Self-regulation is management asserting its own controls

Gates’s rejection of self-regulation is not a radical position for anyone who works within the three lines. Management owning and operating its controls is the first line. It is necessary, but never sufficient on its own. The value of the second and third lines comes from independence: someone who did not design the control tests whether it works.

A frontier lab grading its own safety case is the first line signing off its own testing. When the lab’s own chief scientist says publicly that the monitoring is not good enough, that is, in our terms, a management self-assessment that has come back adverse.

The kill switch point lands just as cleanly. A single shutdown mechanism is a corrective control with no preventive or detective layer around it, and it only fires after something has already gone wrong. No control environment we would sign off rests on one control of that kind. Gates was describing defence in depth without using the phrase.

Records are where this becomes our problem

To be clear, none of these warnings were addressed to us. They were about the labs. But they carry a consequence the labs did not spell out. If the builders cannot yet fully observe their own models, a deployer cannot treat the vendor's assurances as its own. Relying on the provider's safety claims is self-regulation by proxy, and that is precisely what Gates said is not enough.

This is where the debate stops being about frontier labs. Most of our organisations will never train a frontier model. Almost all of them now deploy one, through a vendor platform, a Copilot licence, an agent wired into a workflow, or a team that signed up for something without telling anyone.

The question Gates is putting to Congress is the question a regulator, a board or an investigator will eventually put to us: show me what this system did, who used it, and what it could reach.

For high-risk systems in the EU, that question already has a date attached. Article 26 of the AI Act requires deployers to monitor operation and to keep the logs those systems generate for at least six months, and since the Digital Omnibus, the Annex III obligations apply from 2 December 2027. ISO/IEC 42001 sets event logging expectations in Annex A. The MEASURE and MANAGE functions of the NIST AI RMF assume there is ongoing monitoring to measure against.

None of that is new. What is new is that the people building the most capable systems are now saying, in public, that monitoring is the weak point.

What I would test first

If I were scoping an engagement after this month’s news, I would not start with policy. Most organisations now have an AI policy of some kind, and a policy tells you what management intends, not what the systems are doing. I would start with three requests, and I would expect at least one of them to fail.

The first is the inventory. Monitoring presupposes knowing what to monitor. An organisation that cannot produce a complete register of the AI it runs, including features embedded in vendor products and agents built by individual teams, cannot credibly claim to monitor it. This is why I have written a series on building and classifying an AI asset register: it is the precondition for everything else in this article.

The second is the logs. For a sample of systems from that register, ask for the actual records of use: inputs, outputs, tool calls, the identity behind each action, and how long all of it is kept. Not a screenshot of the logging configuration. The records. Many deployments rely on whatever the vendor retains by default, and customers often discover how little only when they need it.

The third is the response. Pick one system and ask who would notice if it started behaving outside its intended purpose, how quickly, and who has the authority to turn it off. That last question is the enterprise version of the kill switch debate. It usually reveals that the switch exists technically, but nobody owns the decision to use it.

What I would test first

The warnings raised this month are about systems far more capable than anything most of us audit today. It would be easy to file them as a matter for Congress, Brussels, or the labs themselves.

But the controls being demanded at the frontier are controls we already know how to test, and the gap the builders are admitting to is the gap we keep finding in ordinary deployments. Monitoring that was never built cannot be legislated into existence after the fact, and records that were never kept cannot be produced on request.

Gates wants records. A researcher walked out because he did not trust the commitments. OpenAI’s chief scientist says the monitoring is not good enough yet. If someone asked your organisation tomorrow for the records of what its AI did last month, what would you hand them?

Najwan Hudaihed
Najwan Hudaihed

25+ years in IT audit and cybersecurity. Writes on auditing AI systems for technology risk, internal audit, and governance practitioners.