Skip to content
Menu

Method7 min read

Why so many AI agent projects disappoint, and how to avoid it

Autonomy too early, no measurement, a poorly connected tool, opacity, forgotten supervision: five mistakes that make AI agent projects fail, and how to counter them.

On a windowsill, two potted plants side by side: one droops, yellowed, and loses its leaves; the other, staked, stands upright and healthy green, next to a blue-green watering can.
AI-generated image

In short

  • Five mistakes explain many disappointments with AI agents: too much autonomy too early, no baseline measurement, a poorly connected tool, opacity, forgotten supervision.
  • Several cases reported in the press (Builder.ai, 11x, Artisan) show these flaws: promises of autonomy, disputed figures, an unclear split between machine and humans.
  • Each mistake has its counter: a rulebook approved by you, a “shadow” trial where the agent proposes without acting, autonomy levels and continuous supervision.

An AI agent is a program that relies on an artificial intelligence model to prepare or carry out tasks in your software: sorting emails, preparing a reminder, filling in a quote. We speak of “agentic” AI when the AI acts, and not only when it answers. Yet some of these projects stop along the way.

Several cases reported in the press show why. Here are five mistakes, and how our method counters them.

Mistake 1: aiming for full autonomy from the start

An agent that works alone, day and night: the promise is tempting. It remains risky.

The head of Artisan, a young company selling sales prospecting agents, admitted it: “We had extremely severe hallucinations at our launch” (TechCrunch, 9 April 2025). The company says it has since fixed most of the problem. A hallucination is information invented by the AI but presented as true. When an agent acts alone, an invention can go straight to your customer.

The counter: autonomy levels. Each agent starts at level 0 (it proposes, your team does) then moves to level 1 (it prepares, you approve every action). It only moves to level 2 (it acts alone on safe cases, written in its rulebook) or 3 (autonomous, monitored) if the measurements justify it and, as our method provides, with your written agreement. After a serious alert, it can drop back a level. The method also provides that some actions never go beyond level 1: a payment, sending a third party a document that binds you, deleting data. Its stop cases are written in its rulebook: in those cases, it stops and hands over instead of guessing.

Mistake 2: measuring nothing before you begin

Without starting figures, nobody can say whether the agent helped. The project becomes a budget line to cut.

Take a typical scenario: an SME with 30 employees that chases its unpaid invoices by hand. Before connecting anything, you need to know how many reminders go out each week, how long they take and the average payment time.

The counter: the audit and the shadow trial. The audit (paid, deducted from the quote if you sign it within 30 days) lists your tools, your volumes and the expected gains. Each agent page sets its indicators: for example the average payment time before and after, or the share of drafts your team approves. Then comes the shadow trial, what IT people call acceptance testing: for a few weeks, the agent proposes and your team works as usual. Agreement rate, errors and time saved are measured before anything goes live.

Mistake 3: a generic tool that does not talk to your software

Used without connections, a general-purpose chat assistant does not see your invoicing software and does not know your rules. You copy, you paste, and the tool may fill the gaps by guessing. And not everything needs an agent: for a repetitive task with no decision to make, a simple automation is often enough.

The counter: a rulebook per agent and chosen connectors. A connector is the small program that links the agent to one of your software tools. The agent's rulebook, which you approve before anything is built, states what it reads, what it produces and where a human approves. Access will be cut to the strict minimum, read-only where possible. Connectors will be chosen to suit your software, each with a frozen version.

Mistake 4: opacity about what the AI does and what humans do

Builder.ai promised to build apps using AI. Backed by Microsoft and Qatar's sovereign wealth fund, its valuation approached one billion dollars. In May 2025, its head announced that the company was declaring bankruptcy, for lack of funds (The Register, 21 May 2025, quoting the Financial Times). The Register recalls that the Wall Street Journal had reported as early as 2019 that it left most of the code to human engineers rather than to AI. In March 2025, Bloomberg revealed that, from summer 2024, it had revised downwards the sales figures given to its investors, and hired auditors to examine its accounts (Bloomberg, 31 March 2025).

A rumour followed: its “AI” was supposedly in fact 700 engineers. Former engineers dispute this: according to them, a real code generator existed, and subcontracted developers built the clients' apps (The Pragmatic Engineer, 12 June 2025, a secondary source based on testimonies). Even after the bankruptcy, people still argue about the share of the machine and the share of the humans. That is exactly the problem.

The counter: write everything down and make it visible. Every action of an agent will be entered in a log: what it read, what it proposed, the human decision, who took it and when. Each agent will also have a diagram of its path, readable by you, showing its steps and the points where a human approves. The agent prepares, your team approves, and you will know who did what.

Mistake 5: forgetting supervision and maintenance

Sales prospecting agents, sold as able to work alone, had a difficult start. At 11x, a young company in this sector, an employee quoted anonymously by TechCrunch said: “We were losing 70 to 80% of the customers who came in”. The company disputes this: it places its biggest customer losses in late 2023 and reports 79% of customers retained (TechCrunch, 24 March 2025). Artisan's head, for his part, describes the first generation of these tools: a “fairly low” response rate and a “relatively high” customer loss (TechCrunch, 9 April 2025).

These articles do not say where the losses come from. One account of experience sheds light on one factor: human attention. On the SaaStr blog, Jason Lemkin reviews six months of AI prospecting agents; his team runs five of them. He reports results, but at the cost of 15 to 20 hours of human steering per week. His line: “Performance goes up and down directly with human attention” (SaaStr). A review published in April 2026 covers 11x, Artisan and this SaaStr account (deathtocoldemails.com, a secondary source).

An agent is not installed to be forgotten. AI models change, your software gets updated, your rules evolve.

The counter: supervision by subscription. We will follow your agents in a dashboard, in preparation: alerts, reports, review of autonomy levels and versions, at a pace set with you. The agents will also report their own problems (a bug, a recurring error, a change of AI model) in your client area, also in preparation. Each agent will have a frozen version, and every update will go through a short trial. The subscription price is stated in your quote.

The method on one page: agent rulebook, shadow trial, autonomy levels, supervision

  1. Audit (paid, deducted from the quote if you sign it within 30 days): tools, volumes, expected gains, chosen agents.
  2. Rulebook per agent, approved by you: mission, human approval points, stop cases, indicators, starting autonomy level.
  3. Plan and build: connectors, tests on your anonymised data, security review.
  4. Shadow trial of a few weeks: the agent proposes, your team does, we measure. You decide on going live.
  5. Gradual go-live, from level 1 to level 2, according to the measurements.
  6. Continuous supervision, and training for your team: reading the proposals, approving, correcting, reporting.

You decide twice: at the rulebook and at go-live. In between, we do the work, and every step will be traced.

Going further

Sources

Consulted on 12 and 13 October 2026.

Sources

Accessed on 13 October 2026.

  1. TechCrunch, “Artisan, the 'stop hiring humans' AI agent startup, raises $25M and is still hiring humans” (9 April 2025)
  2. TechCrunch, “a16z- and Benchmark-backed 11x has been claiming customers it doesn't have” (24 March 2025)
  3. The Register, “Builder.ai coded itself into a corner – now it's bankrupt” (21 May 2025)
  4. Bloomberg (republished by Bloomberg Law), “Microsoft-Backed AI Firm Hires Auditors After Overstating Sales” (31 March 2025)
  5. The Pragmatic Engineer, Gergely Orosz, “Builder.ai did not 'fake AI with 700 engineers'” (12 June 2025), SECONDARY SOURCE
  6. SaaStr, Jason Lemkin, “6 Months of AI SDRs: What's Worked, How They Brought In $1M+ in 90 Days, and the Real Data Everyone's Asking For”
  7. deathtocoldemails.com, Peter Korpak, “AI SDR Post-Mortem: What 11x and Artisan Cost the Category” (April 2026), SECONDARY SOURCE

← Back to the blog

Next step

Tell us where your team loses time.

Everything starts with an audit: your tools, your ways of working, your volumes. You then receive a quote.

Book my audit