In short
- An AI agent in service can drift: the AI model changes, your software changes, your data changes too. Supervision is there to spot these drifts early.
- The agent, its tools and its connectors will have a frozen version. An update will always go through a short acceptance test, that is, a trial before going into service.
- The agent will report the problems it detects itself: it will open a ticket, we will be alerted, and you will follow everything in your future client area.
Our method comes down to one rule: the agent prepares, your team approves at the planned steps, and the agent stops rather than guess in the cases written in advance.
One question comes up quickly, though: once the agent is installed, who watches it? And what happens if it gets something wrong? The answer lies in the supervision subscription, now in preparation. Here is what it will include, and why.
What can drift once the agent is in service
An AI agent is not installed to be forgotten. Three things can change around it without anyone touching its settings.
The AI model changes. The model is the agent's engine: the program that reads an email or an invoice and writes a proposal. Its provider publishes new versions and retires old ones. Anthropic, which develops the Claude models, writes in its documentation that it regularly retires its older models. For models made public, it warns the customers concerned at least 60 days before retirement. A request sent to a retired model fails. For example, on 30 September 2026, Anthropic warned developers using Claude Sonnet 4.5 that this model would be retired from its API on 30 November 2026. Claude models are also offered by Amazon (the Amazon Bedrock service) and by Google (Google Cloud). These two platforms set their own retirement schedules. So you have to follow every schedule that concerns the agent.
Connected software changes. The agent plugs into your tools (email, invoicing, bank) through their official interfaces (APIs): the entry points that this software opens to other programs. The small program that makes this link is called a connector. These entry points change. The accounting software Pennylane announced that the first version (V1) of its API would be fully obsolete at the end of 2025, V2 having become its official stable version. The management software Odoo plans, according to its developer documentation, to remove two old APIs (called XML-RPC and JSON-RPC) in its version 22, expected in autumn 2028.
Your data changes. A new type of client, a new invoice template, a seasonal peak: the agent meets cases that nobody foresaw at set-up.
Frozen versions for the agent, its tools and its connectors
Faced with these changes, our rule is simple: on our side, nothing moves without a decision.
Each agent deployed with you will carry a version number. So will its tools and its connectors. The AI model will be designated by a precise identifier: at Anthropic, each model identifier corresponds to a frozen version. The agent that passed acceptance testing with you will therefore be the one running the next day: same version, same tools, same connectors, same model.
Why insist on it? Freezing versions does not make the agent infallible. But the agent does not change without your knowledge. If a result gets worse, we look at your data or a connected piece of software, not at an update that went unnoticed.
An update sometimes becomes necessary: a model announced as retired, an API that changes, a fix. It will then go through a short acceptance test. We will run the new version on the agent's test sets, cases taken from your business and anonymised, then compare its proposals with those of the version in place. If the results hold, the new version will replace the old one. If not, we will fix it before putting it into service.
When a model is announced as retired, we cannot wait indefinitely: after the retirement date, the old model no longer responds. So we will start the acceptance test well before that date. This is also what Anthropic recommends: test your applications with the new models well before the old one's retirement date.
Automatic reporting: the agent will open a ticket itself
Your team will not have to watch for breakdowns. The agent will report the problems it detects itself.
It will open a ticket, that is, a tracking record in our support tool, now in preparation, when:
- the same error repeats;
- it runs into a bug, that is, a malfunction;
- a change affects the AI model it uses.
The ticket will be linked to your company and your project. It will contain the technical detail we need to understand, and we will receive an alert. If the same problem comes back several times, it will remain a single ticket: no shower of messages.
And if the agent gets something wrong without realising? Two safeguards then come into play.
First, each agent has stopping cases written in its rulebook: a client in dispute, an amount outside the rules, missing data, doubt about the sender of an email. In these cases, it does not guess. It stops and hands over to a person.
Second, each agent moves forward by autonomy levels, from 0 to 3:
- level 0: it proposes, your team does the work itself;
- level 1: your team approves, corrects or refuses each action before it goes out;
- level 2: it acts alone on safe cases, defined by written rules, and your team approves the others and checks a sample;
- level 3: it acts alone, and your team reads the report and receives the alerts.
At levels 0 and 1, every proposal passes in front of a person before it has any effect. After a serious alert, the agent will automatically go back down a level. Some actions never go beyond level 1: a payment, sending a third party a document that commits you, deleting data.
The dashboard and the visual diagram of each agent
On our side, each agent will appear in a supervision dashboard, now in preparation. We will track there:
- the agent's status and its alerts;
- the agreement rate, that is, the share of proposals your team approves;
- errors and open tickets;
- the indicators written in the agent's rulebook, for example the time saved by your team;
- the cost of AI.
This cost is counted in tokens. A token is a small piece of text: a word, part of a word or a few characters. It is the unit in which Anthropic bills the use of its models, with one price for the text sent to the model and another for the text it produces. Each agent will report its own figures: this way we will track the AI cost of your project, month by month.
Every action will also be entered in a log: what the agent read, what it proposed, what the person decided, who, when, and with which version of the agent.
The agent will run where you have chosen: in your cloud account (servers rented from a major provider, in a European region), on your own servers or on a dedicated server with us. Only the indicators, the alerts and the tickets will go up to our dashboard, not your files. The technical detail of a ticket will be filtered to remove personal data.
One clarification: to do its work, the agent will send the AI model the text it has to process. The personal data sent is limited to what is written in the agent's rulebook, which you approve before building. The path to the model, and so the place where this text is processed, depends on the deployment chosen. Depending on the provider, the model may be operated outside the EU, under GDPR safeguards.
Finally, each agent will have its visual diagram: a drawing of its path, step by step, with the points where a person approves. It can be read without technical knowledge. We will keep it up to date. Like the agent, it will carry a version number: if the path changes, a new version of the diagram will show it.
Monthly report and quarterly review of autonomy levels
Every month, you will receive a report. It will give the indicators for each agent: volume handled, agreement rate, errors, tickets opened and resolved. It will also announce what is coming: a model soon to be retired, an update to plan.
Every quarter, we will hold a broader review, on two subjects.
- The autonomy levels. Can an agent whose measurements are good move up to the next level? It moves up only on the strength of measurements (agreement rate, errors) and with your written agreement. If the measurements do not follow, it stays at its level.
- The versions. Which models, tools and connectors will change in the coming months? Which acceptance tests should we plan, and when?
What you will see in your future client area
Your client area, now in preparation, will bring together everything about your agents:
- tickets, including those your agents opened themselves, with their progress;
- your own support or change requests;
- your invoices.
Technical monitoring is our job. You keep the decision, and you will see where each agent stands.
The subscription is on quote. The response time for tickets is specified in your quote.
Going further
- Supervision and support: what the subscription will include, and the response time.
- Our method: from audit to going live, with the autonomy levels.
- Data and security: where your agents run and what comes back up to us.
- Request an audit: an audit at €490 excl. VAT; its price is deducted from the quote if you sign it within 30 days.