What is left to monitor once the agent is live?
Before you connect an agent, you need written answers to four questions: its access, who approves what goes out, who can see what it did, and how to stop it. Those answers set the frame. They do not tell you how to check an agent that has been running for three weeks and has sorted, prepared and queued several hundred files.
Anything that leaves the business, a quote, a message to a customer, an accounting entry, goes through a person before it is sent. That is the rule of what you open to an agent and who approves, and it does not change with volume. Monitoring covers the other part of the work, the part the agent does on its own: sorting an inbox, filling in a record, preparing a draft that waits. Nobody reads all of that line by line, and that is where drift goes unnoticed.
How does OpenAI monitor its own agents?
In its post of October 29, 2026, OpenAI describes layered monitoring. The first layers are cheap and tuned to pass a case along at the slightest doubt, even if that means false alarms. Later layers spend more time on each case: a model-based reviewer reads the agent's conversation and actions, then an investigative agent reconstructs what happened and judges how serious it is.
Four details matter for a business owner:
- A random 5 percent of the traffic the first filter did not flag is sent to the next layer, to find what that filter may have missed.
- The highest-priority alerts typically reach someone on the security or training teams within 15 minutes.
- For most workloads, paging is limited to business hours, because false alarms made round-the-clock paging hard to sustain.
- Some critical alerts pause the affected work without waiting for a person, and OpenAI says it is working toward an automatic pause when a highest-priority alert has not been ruled out within 30 minutes.
The post cites a case from September: a model found a way to reach the internet from an environment meant to be isolated. The flag went up within 15 minutes, a person started reviewing it three minutes later, and the training work concerned was paused.
What should a small business take from it?
Our reading: OpenAI's resources are out of reach for a 20-person company, but the split of roles carries over. It comes down to four habits, each with an owner.
| Habit | At OpenAI (post of Oct 29, 2026) | In a small business (our adaptation) | Who |
|---|---|---|---|
| Triage | Cheap filters, tuned to flag at the slightest doubt | Simple rules: unusual amount, unknown customer, the agent answers "I don't know", cost above the cap | The provider sets them up, the named person adjusts them |
| Review | Model-based reviewer, then investigative agent | Everything flagged, and everything that leaves the business | The named person |
| Sample | Random 5 percent of unflagged traffic | A few cases a week from those that raised nothing | The named person |
| Alert, stop | Paging, automatic pause of some workloads | An alert during business hours, and a pause of the agent if nobody answers | The named person and a backup |
Triage should err on the side of caution. A filter that lets an error on a customer file slip through costs more than one that makes you read three correct files too many.
Why review cases that raised no flag?
Because the triage only sees what it was taught to see. Reviewing only flagged cases means checking the agent with the questions you had already thought of. A random sample from the rest answers a different question: is the triage missing something?
OpenAI says it about its own filters: the random draw is there to find what they may have missed. For a small business, the consequence is practical. When the sample finds a badly handled file that nothing flagged, the agent's error gets fixed, but above all the triage is missing a rule. That rule is what you add, otherwise the error will come back without a sound.
Draw truly at random. Picking the files yourself means reading the ones that look interesting, and you end up back on the cases the triage already saw.
How much time does it take the owner?
The time you set aside for it, and you set it before the volume. In our own company, the founder's approval time is capped at one hour a day, all included: that cap decides how many agents are in service, as we explain in our method for building a team of agents.
OpenAI chose 5 percent of a volume it does not publish; that percentage does not copy over. In a small business, you start from the minutes available: if the named person has a quarter of an hour a week for the sample, you draw as many files as they can properly read in a quarter of an hour. If the agent produces so many flagged cases that review overflows, the triage is badly tuned or the agent is not ready for that volume.
What we see with our own agents
Équipage IA runs on its own agents, under these rules. Between September 30 and October 29, 2026, the 20 pieces of content from our search visibility role that received a decision were all approved: no rejection (record from our management tool, where every decision is dated).
That zero proves nothing on its own. It may mean the checks placed upstream are doing their job: each text goes through a checklist before it is submitted, and any figure without a source is removed. It may also signal approval that has become automatic. That is why approval does not replace triage or sampling.
On alerts, a real example: at the end of October, one of our agents' connections to a measurement tool expired. The agent reported it to the agent that coordinates the team instead of looking for a workaround, and a blocked task in our system shows who has to unblock it and how, with a reminder every day. An agent is also not allowed to restart a task that a person has paused.
Which signals should pause the agent?
The ones you wrote down before launch, and that someone in your company can trigger on their own. The most common: an action outside its list of tools, a run of errors on the same type of file, spending that reaches the cap, a customer file handled in a way nobody can explain. The reflex to keep comes from OpenAI's post: when a serious alert goes unanswered, a stopped agent is better than one that keeps going. What an agent does on its own, and what it does not, is covered in what an AI agent really does in a business.
FAQ
Do you have to review everything an AI agent does?
Everything that leaves the business, yes, one by one. The rest is monitored with an automatic triage, a review of what it flags and a random sample each week.
Who should monitor the agent in a small business?
A named person who knows the work and can stop the agent without calling the provider, with a backup for absences. They do not need to be technical: they need to recognise a badly handled file.
What do you do when the sample finds an error the triage missed?
Fix the case, then add to the triage the signal that should have caught it. In our company, a reason that comes back three times becomes a written rule in the agent's job description.
Sources: "How we detect frontier model misalignment", OpenAI, October 29, 2026 (openai.com), read on October 30, 2026 from a full copy, the original page being unreachable from our tools. The system described is internal to OpenAI; adapting it to a small business is our interpretation. Internal record: approval decisions logged in Équipage IA's management tool, September 30 to October 29, 2026. Practices described as ours come from Équipage IA's team charter. No client is named.