The Classification You Don't Do, a Court Will

You can delete a bad output. You cannot delete a bad action. A draft can be rewritten, a recommendation ignored. An email that has already gone, a payment that has cleared, cannot be recalled. That is the line between software that proposes and software that decides and acts, that has agency. Organisations are stepping over it now without quite noticing they have done so, which is what makes most of the governance advice on offer in 2026, sensible as it sounds, beside the point.

So what is that advice? The advice is to classify your AI by risk before a regulator or a court does it for you. Tier every system, map each tier to the relevant rule, and have the register ready when you are asked for it. It is good advice. Boards have made it a duty, the European Commission published draft guidelines on classifying high-risk systems in May and opened them for consultation, and every firm with a letterhead is telling clients to label their systems ahead of enforcement. A company that cannot say which of its systems are high-stakes is not ready to govern any of them. Classification is a precondition.

It is also, on its own, a way to produce a precise description of your exposure and file it as though it were governance. The register answers the question everyone thought to ask, which is who signed off on these systems and under which rule. It is silent on the question that actually matters, which is what is watching each of them now, quickly enough to catch one the moment it begins to go wrong, and able to do something about it before the divergence becomes a decision the company has to defend. A risk tier is a label, and a label is an instrument, not the capability it points to. I wrote earlier this year about an audit trail that records a timestamp and calls it oversight, a record of a human presence standing in for the structure that presence was supposed to provide. A risk classification can be the same move, one level up. We classified the risk, and there we are.

Governance is what stays reversible

What makes such a label hollow, at best, is the asymmetry we started with. Classification does nothing about which of your systems can take actions you cannot take back. It sorts them by how much they might cost you, not whether their mistakes are recoverable. And that, not the tier, is what governance is actually for. Governance, the real thing rather than the record of it, is the capacity to keep a system’s failures reversible while it runs. The register describes. It does not reverse.

Oversight at three speeds

Once you see governance that way, the problem everyone trips over, that an agent acts faster than any human can intervene, stops being a paradox. You cannot oversee an agent at the moment it acts. You can only have already overseen it, in what you built before you let it run. Oversight of a fast system breaks into three jobs running at three different speeds, and only one of them is fast.

The first is human-paced and happens before anything runs: deciding the policies, the thresholds, the conditions under which a system must stop. This is where decision rights live, the question of who is allowed to set what a system may do, and it is where accountability stays. The second is machine-paced and happens while the system runs: every action checked against those pre-committed rules before it executes, and blocked the instant it breaches one. This is the only part that moves at the agent’s speed, and it has no judgement in it at all. It is a circuit breaker, not a second decision-maker. The third is human-paced again and comes after: reviewing what was stopped, testing a sample of the decisions against the standard the system was set, and tightening the thresholds where the review finds drift.

That division answers the first thing a sceptical engineer says, which is that runtime oversight is just monitoring with a new name. It is not. Monitoring observes: it collects a record of what the system did and tells you, accurately and too late, that it went off-policy at 2:14 am. Enforcement evaluates each action against policy before it executes and can refuse it, so the off-policy action never finishes. A dashboard tells you what happened. A runtime control means it did not.

It also answers the harder objection, the one that should worry you more. If the thing watching the agent runs at machine speed, what governs the thing watching the agent? The answer is that it has no discretion to govern. It carries out rules that named people wrote and review, so the question of who oversees it resolves to who set the thresholds and who audits the flags and the halts: a question with names attached. It is governable exactly where the agent is not. And its failure runs the safe way. A wrong stop costs you a few minutes and a second look at human pace. A wrong action costs you the thing the courts are now pricing. You automate the stop because the stop is the one move that is safe to get wrong.

The honest edge of this is that some teams are now building the enforcing layer itself out of AI, one model judging another’s actions in real time, which quietly puts the discretion back. The version that survives is the plain one: fixed rules and thresholds and halts that decide nothing themselves. The narrowness is the point.

We already run this architecture

None of this is hypothetical. We already trust the same architecture in another setting entirely: the financial markets. On the sixth of May 2010, the Dow fell roughly a thousand points in minutes and recovered almost as fast, a market moving far quicker than any regulator’s reflex. The response was not to hire faster humans. It was to write halt conditions into the market itself: circuit breakers that trigger automatically at defined thresholds, pause trading, and hand the question back to people to sort out afterward. Decide the rule in advance, enforce it at machine speed with no discretion, review it after the fact. That is runtime oversight, built more than fifteen years ago, in a domain that clears trillions a day. The agentic version is the same architecture pointed at a different kind of machine.

What a court actually asks

Which brings the argument to the place it has been heading, and it is a courtroom. Read this year’s AI cases not as the bias stories their headlines promise but as fights about classification, about who gets to say what a system is. In Mobley v Workday, a federal court in California has let an age-discrimination claim proceed as a nationwide collective covering applicants screened through the vendor’s hiring AI, and in March it rejected the argument that the law does not reach applicants at all. These are contested allegations and procedural rulings, not findings that anyone discriminated. But the shape is the point: a company’s description of its own product as a neutral tool is precisely the self-classification a court is now declining to take on trust. In Kistler v Eightfold, filed at the start of the year and still at the pleading stage, the unproven allegation is that a hiring platform scored applicants in secret and discarded the low-ranked among them with no way to see or challenge the score, conduct the plaintiffs say made it a consumer reporting agency that never registered as one. Strip away the statutes and it is the same dispute: a company called a system one thing, and a court is deciding whether it was something else.

Now set the register beside that courtroom. Picture the strongest thing a defendant could put on the table. It is not the tier they assigned the system themselves. That is the weakest exhibit in the room, because it was only ever written down, and nothing was built to honour it. The strongest exhibit would be a record of what was watching the system while it worked, what that watcher was allowed to stop, and whether anyone tested its decisions against the standard it was set: a halt log and an audit that was actually run. One of those documents is a statement of intent. The other is evidence that the system was governed while it ran. The classification tells the court what you meant to do. Only the second thing tells the court what was true while the software was making decisions in your name.

The same pressure is now building in regulation and the courts, and from three directions at once. In the United States it comes through the courts, which are declining to take a company’s account of its own system on trust. In the European Union it comes through statute: the AI Act defines a deployer as the organisation using a system under its own authority, and then, in the same law, requires that deployer to put named people with the competence, the training and the authority to intervene in charge of oversight. Read closely, it is not asking for a register. It assumes oversight that can act, and the criteria for what counts as high-risk are being drafted in Brussels as I write. In the United Kingdom it comes through the regulators themselves: the competition, data, finance and communications authorities have been working jointly on where accountability sits when an agentic system acts across a chain of suppliers. Three legal traditions, three mechanisms, one direction of travel. The label is the part everyone is selling. The thing the label is supposed to sit on is the part nobody is.

Three questions for the highest-tier system

So, before you are ever in that room, put three questions to the single highest-tier system on the register you have just finished, in front of the business, the risk people and the technologists at once.

1. Detection. If this system departed from its policy at two in the morning, how would we notice, and how fast? A person reading a dashboard the next day is the same answer as nothing.
2. Interruption. What can stop it in real time, before it happens, who wrote that rule, and when was it last tested with a deliberate breach? If nothing can stop it without a person in the loop, you do not have oversight. You have monitoring and a hope.
3. Review. When did someone last pull a sample of its decisions, test them against the standard you set, and feed the result back to continuously monitor and improve the system?

If the honest answers are nothing, nothing and never, then the tier you gave that system describes your exposure, not your governance. None of the three is a purchase. The market will sell you a layer that blocks an action before it executes, and it is worth having. But nobody can sell you the decision about what to enforce, or the discipline to read what it stops. Tool and institution together, or it is theatre at a faster clock.

The register is accurate. That was never the question. What a court or a regulator actually asks is not the tier you assigned. It is what you can produce: what was watching, how fast, and what it could do when the system moved. A classification you really do is one with something running underneath it. That is the difference the word has carried all along: the label was never the governance. The governance is what can still be reversed while the system runs. The classification you do not do, in that sense, is the one a court will do for you, against criteria you did not write, in a room you do not control.