Agents that call the software you already run: reading first, and changing data only where you decide a person or a switch should allow it. Whoever builds yours should write down each tool's read-or-change boundary before it touches your data. On the MCP server we shipped, the 20 released tools only read, and the 2 write tools, finished but not yet released, stay off until an operator switches them on. Bring one workflow to a call. Ask for a senior software engineer and you get straight answers on architecture and stack. If we are not the right team for what you are building, we will tell you that.
An agent that works on the happy path can still fail in three plain ways. When it does, the cost lands on whoever commissioned it.
A loop nobody capped
One person on a practitioner forum described an agent that "called the LLM thousands of times sequentially before anyone noticed." Another found that "prompting against it rarely works reliably." A loop has to be stopped by code that counts the calls.
A write that should not have fired
As one Hacker News commenter put it, "any hallucinated tool call can fire off messages to an entire org while the agent is doing something unrelated." The danger sits in the tools that change things, so those tools need a rule or a person in front of them.
A bill that grows with every task
Another Hacker News commenter wrote: "Bills are going up because AI agents use way more tokens per task, not because the models cost more." The meter runs on model calls per task. If nobody counts them before launch, the first count arrives on the invoice.
Your name on the project
The agent sits on your budget line. If it loops, misfires or overspends, you are the one asked to explain it.
Two figures to take to whoever signs
Gartner names escalating costs and inadequate risk controls among the reasons it predicts agent projects will be canceled. Our reading: both are set by how the agent is built.
Over 40%of agentic AI projects will be canceled by the end of 2027, Gartner predicts, "due to escalating costs, unclear business value or inadequate risk controls".
Gartner press release, 25 June 2025. A prediction, not a measured result.
CAD 812.02awarded against Air Canada after a tribunal held the company responsible for what its website chatbot told a customer: "It makes no difference whether the information comes from a static page or a chatbot."
Moffatt v. Air Canada, 2024 BCCRT 149, BC Civil Resolution Tribunal (small claims), 14 February 2024.
What actually stops a run
Ask any vendor which line of code ends a run. A call-count limit is what both frameworks document; a cap in money is something your build adds. Your own software engineer can check every name below in the public docs.
What actually stops a run
Stop control
LangGraph
Agno
Fixed workflow with a model step
Hard stop on steps
LangGraphrecursion_limit raises GraphRecursionError. Set it per graph; do not rely on the default.
Agnotool_call_limit, off by default. It covers one response loop, not a budget across pauses.
Fixed workflow with a model stepNo loop to cap. Each order or item gets one run.
Call limits
LangGraphModelCallLimitMiddleware and ToolCallLimitMiddleware in LangChain agents. No limit by default.
AgnoStopAgentRun, raised from a tool, ends the run after the current batch of tool calls.
Fixed workflow with a model stepEach step is a short function with its own failure handler.
Wind down before the limit
LangGraphRemainingSteps tells the graph how many steps are left.
Agnostop_after_tool_call returns a tool result without another model reply.
Fixed workflow with a model stepNot needed. The steps are fixed.
Pause for a person
LangGraphinterrupt() saves state and waits until resumed. Anything it does before the pause should be safe to repeat.
Agnorequires_confirmation=True pauses the run until a person confirms or rejects.
Fixed workflow with a model stepA person reviews and sends.
Cancel mid-run
LangGraphstep_timeout limits one step, not the whole run.
Agnocancel_run() stops at the next checkpoint and does not undo changes already made.
Fixed workflow with a model stepEach step fails on its own handler.
Spend cap in money
LangGraphNot found in the documentation we checked (October 2026). Whoever builds it adds one.
AgnoNot found in the documentation we checked (October 2026). Whoever builds it adds one.
Fixed workflow with a model stepBounded by the fixed number of model steps per run.
Call limits only bound spend indirectly, so a cap in money is something
whoever builds your agent adds. If you run on AWS, you may also be
weighing Strands; we compare it against your AWS estate in discovery.
The framework follows your workflow and your cloud, not our preference.
One good run is not a working agent
A demo shows one run. On tau-bench, a public tool-calling benchmark, the chance that an agent gets the same task right on every try drops each time you repeat it. These are public figures, not our measurement.
Show data table
Chance that every one of k tries at the same task succeeds (pass^k), tool-calling claude-3-5-sonnet-20241022 on tau-bench. Retail falls from 0.692 at one try to 0.462 at four. Airline falls from 0.460 to 0.225. In the original paper, gpt-4o function calling on retail fell under 25% by eight tries. The tau-bench README now marks these tasks as outdated and points to tau3-bench. The shape, reliability falling with each repeat, is the point, not the level.
Before you sign off on a demo, ask for the same tasks run several times,
with each step and the whole path checked: the right tool, the right
inputs, the right place to stop, not only the final answer. A failure
found in production is worth adding to the evaluation set. After launch,
under an Annual Maintenance Contract or a change note, that is scoped
work, not free. How we approach evaluation for AI features is on
AI-augmented development.
What to require of whoever builds your agent
01
A written tool list
Every tool the agent can call, marked read or change, written down before the build.
Our shipped work
Nearest instance: the MCP server we shipped declares 24 typed tools, and the 20 released ones only read.
02
Read tools live first
Tools that only look things up go live first. Tools that change data stay off until someone switches them on.
Our shipped work
Where we did it: on the MCP server, actions that change data stay off unless an operator switches them on. Our pricing assistant in production uses only tools that read, and code, not the model, picks the next question.
03
An evaluation set agreed before launch
The tasks to test, how many times each one runs, and the bar it must clear, agreed with you before launch.
Our shipped work
We have no shipped instance of a signed evaluation set to show you.
04
A person in front of high-impact changes
Each tool that changes data goes live behind a person's approval, and its changes are made safe to repeat. OWASP lists this control for agents: "require a human to approve high-impact actions before they are taken." Before go-live, ask whoever builds it for calls per task measured on your own tasks.
Our shipped work
Nearest instance, not an agent: on the purchase-order product, a fixed workflow, a person sends every order.
05
Handover
The code, with its prompts and evaluation sets inside it, so your team can run and change the agent without the builder.
Our shipped work
With us, you own every line of code from day one, and prompts and evaluation sets kept in the repository are part of that code. Run traces sit in your accounts when the system runs in your accounts, which is the default.
Five things to require, in order, each shown against our shipped work where
we did it. Product Discovery is where architecture, scope and roadmap get
defined before development, and you see progress in a demo every week.
Have one workflow in mind?
Book a discovery call of 15 to 45 minutes. Ask, and a senior software engineer joins to answer architecture and stack questions. If we are not the right team for what you are building, we will tell you that.
What we have shipped, at its real size
A commerce agency running an integration platform for online merchants
An MCP server with 24 typed tools. The 20 released tools only read. Two finished write tools are not released: they stay off until an operator switches them on, and are marked so a client can ask a person first. 46 tests with 246 assertions pass on the finished version.
20 of 24 tools released, all read-only
24 toolsTyped tools built for AI clients, 20 of them released
60 a minuteRequests per token, counted before the token check in the finished version
12 fieldsThe most a customer lookup can return, from a fixed allowlist
A B2B company whose business customers order through a signed-in portal
An assistant that narrows a large catalogue and returns the customer's own price. Its tools only read: product search, and a price tool that releases a price after the backend verifies who is asking. Code, not the model, picks the next question. Our software engineer joined after the first eight-step workflow existed.
Read-only tools, in production
SecondsA price for a business buyer, where it once took 4 business hours to 2 working days
Plain languageBuyers ask in their own words and see their own account's price
Faster quotesBuyers finish the quote for their own customer sooner
An early-stage software company selling order automation to B2B sellers
An AI service reads each emailed PDF purchase order and rules match each line. A person checks the flagged fields and sends the order. We turned down an agent loop here and built a fixed workflow, one run per order. This product is not an agent.
A person sends every order
3 ways inA purchase order arrives by the seller's own inbox address, a dashboard upload or an API call
0.8The title similarity a line needs to match a product without an exact SKU; below it, a person picks
6,000 charactersThe point past which an order is split, read in parallel and merged, with repeated lines dropped by rule
We have no measured handoff or escalation rate for any of the three, so
none is shown.
What it costs to build, and what it costs to run
Run cost is model calls per task, times tokens per call, times the provider's price. On our own shipped assistant, a narrowing reply takes up to six calls. We have not measured that in money on shipped work; ask anyone who quotes you a cost per task how they measured it. By default the cloud accounts are yours. Set the model provider account up the same way and its bill comes to you from the provider.
Show data table
Most model calls one reply can take in a narrowing conversation. The shipped pricing assistant takes up to six. A redesign that merges steps takes at most two; it has been designed and run on a local machine and is not in production.
Item
Model calls per reply, at most
Shipped assistant, in production
6
Redesign, designed and run locally, not in production
2
FigureMost model calls one reply can take in a narrowing conversation. The shipped pricing assistant takes up to six. A redesign that merges steps takes at most two; it has been designed and run on a local machine and is not in production.Pricing assistant case study, /case-studies/b2b-pricing-assistant/
The build cost moves with five things: how many tools the agent has,
how many of them can change data, how many of your systems they touch,
how large the evaluation set must be before you sign off, and how many
approval points a person staffs. Older systems add cost. Gartner notes
that connecting agents to legacy systems can be technically complex and
often needs costly changes.
Product Discovery is where architecture, scope and roadmap get defined
before development. A dedicated team or sprint-based build starts once
confidence and budget are locked. A build against a full spec needs that
spec approved before work begins. After launch you choose an Annual
Maintenance Contract or a lean-mode arrangement. Any new feature comes
with a change note that sets out time, effort and cost before you
approve it. A written estimate follows discovery.
When you should not hire us for this
Where we fit
If one workflow needs to call your systems, and you want to own what the agent does after launch, book a discovery call.
Where we are the wrong call
Your steps are fixed and known. A workflow with a model step is easier to test and to audit. We built exactly that for a purchase-order product instead of an agent.
You want AI inside your own delivery process, not an agent in your product. AI-augmented development covers that.AI-augmented development
You need a partner with shipped AWS Strands work. We have none to show you.
You want a demo in days, not a system your team will run and own.
Questions to settle before you sign off
Who is responsible if the agent tells a customer something wrong?
A tribunal in British Columbia held Air Canada responsible for what its chatbot told a customer (Moffatt v. Air Canada, 2024). The decision reads: "I find Air Canada did not take reasonable care to ensure its chatbot was accurate." So answers about policy, price or eligibility should come from your systems through a tool, not from the model's memory. Whether an action is allowed should be checked in your systems too. OWASP puts it plainly: "Implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not." This is not legal advice. Have your own counsel review any agent that talks to customers.
Who owns the code, prompts, evaluation sets and traces?
You own every line of code from day one, and prompts and evaluation sets kept in the repository are part of that code. Run traces sit in your accounts when the system runs in your accounts, which is the default.
What happens when the model version changes?
Require that a new model version runs against the same evaluation set before anything switches over. On our side, after launch the work runs under an Annual Maintenance Contract or a lean-mode arrangement, and any change comes with a change note that sets out time, effort and cost before you approve it.
How do you test an agent without burning tokens on every run?
Size the evaluation set in discovery. Repeated runs cost model calls, so test on a small set of your hardest tasks first and grow it from there.
How is an agent in our product different from coding agents on our team?
An agent in your product acts for your customers or staff inside your systems. A coding agent helps your software engineers write code. The risks and controls differ. We explain the second in our post on the agentic software delivery lifecycle.
Sometimes. MCP lets outside agents call your tools, which widens who can act on your data. On the MCP server we shipped, the 20 released tools only read, and the 2 write tools, finished but not yet released, stay off until an operator switches them on. Should you build an MCP server walks through the decision.
How does a long task survive a pause or a restart?
The run's state has to be saved outside the process. In LangGraph, interrupt() saves state through a checkpointer and waits until the run is resumed, and the docs advise a durable checkpointer in production. Our post on agent runtimes covers the wider picture.
The one that fits your workflow and your cloud. LangGraph and Agno each document the controls shown above. If you are on AWS and weighing Strands, we compare it against your AWS estate in discovery. Sometimes the answer is no framework at all, just a fixed workflow.
Bring one workflow
Tell us the one workflow you want an agent to run. Someone from business development replies and sets up a call of 15 to 45 minutes. Ask, and a senior software engineer joins to answer architecture and stack questions. If we are not the right team for what you are building, we will tell you that. We have declined engagements where taking them would have benefited us but not the project.
