System truth is not model output

Every system that puts a language model near an operational workflow has to answer one question early, because it is expensive to answer late: what is the model allowed to author?

The answer that holds is none of it. The case state, the stock level, the booking, the SLA clock — these are owned by the application. The model can read them, summarise them, explain them and propose changes to them. It does not get to be the place they are stored, and it does not get to be the reason they changed.

Systems that blur this are pleasant to build and produce states nobody can explain afterwards. Someone asks why a case is marked complete, and the honest answer is that a model said so in a paragraph that has since been overwritten.

The cross-border recruitment system Tomyest built tracks cases across partner offices in seven source countries, each with its own paperwork and reporting habit. The interesting engineering there is not the AI. It is a case model that survives contact with seven countries' processes, and a board that stays legible with hundreds of cases open.

Stage, delay detection and SLA state are computed by the application from recorded facts. That is what makes them auditable, and it is what makes it possible to say — and defend — that a case is late. A generated summary sitting on top is useful; a generated summary that *is* the state would make every counter on the board unciteable.

The question is not whether a person can intervene. It is which decisions require one, decided at design time and enforced in the workflow.

An automation with no defined escalation path does not fail safely. It fails silently and is discovered downstream, usually by whoever inherits the consequence. The escalation path is part of the specification, not a setting somebody adds after the first incident.

The recruitment board makes this concrete with an exception centre: rather than reporting how much work exists, it lists the cases that need a human decision today. A coordinator starts the day with that list instead of assembling it. The value is in the filtering, and the filtering is the hard part.

A board that reports a total is a number. A board that says which cases are late, which need a decision today, and which are about to breach is a tool.

This is the most common place operational AI is aimed wrongly. It is easy to generate a summary of everything and hard to decide what deserves attention — so a great deal of software generates the summary. The people using it keep their spreadsheet, because the spreadsheet answers the question they actually have.

For any workflow with a model in it, ask: if this system produced a wrong state right now, could someone reconstruct why? If the answer depends on text a model generated and nothing else retained, the line has been drawn in the wrong place.