AI Agents from Demo to Production: Who Turns Reliable Execution into Profit?

Put success on the ledger

A customer-service agent finds an order, explains the policy and calls the refund tool. The demo ends there. In production, that is often where the story begins: the payment API times out—was the refund processed? The customer then asks for an exchange—can the earlier action be reversed? The same complaint moves from WeChat to a phone call—will the system compensate twice?

These cases first determine who pays. Model providers charge for calls, software vendors may charge for a resolution, and merchants absorb refunds, complaints and lost customers. Three bills correspond to three different definitions of success.

The thesis here is that agent profits are more likely to accrue to the layer that can constrain tasks, control business state and demonstrate delivery. Cheaper models help, but expensive verification and exception handling can consume the savings. This thesis has limits: low-risk tasks that are easy to verify automatically and reverse may suit a cheap model with simple rules, without an expensive platform.

Why one refund needs a whole system

Imagine an online retailer allowing automatic refunds for orders that have not shipped. This hypothetical workflow helps decompose costs; it does not describe a verified implementation at a particular company.

First, establish state. The agent must read order, warehouse and payment records and know which data is authoritative now. The warehouse may ship after the order query but before the refund. Putting a query result into a prompt does not close that time gap. Rechecking before execution adds a query and can reduce incorrect refunds. Critical state still needs an atomic conditional update or equivalent constraint to prevent changes after the check.

Second, make retries safe from duplicate payments. A network timeout only tells you that no response arrived. Bind a unique identifier to the refund request and check the existing operation when retrying. This keeps another attempt from becoming another refund. The value of idempotency appears in lower compensation costs and fewer investigation tickets.

Third, constrain permissions. Tracking a shipment, issuing a coupon and refunding CNY 10,000 should not share the same authorization. Amount caps, order scope and required human review limit exposure from a single mistake. Aggregate limits, rate limits and monitoring also matter, because repeated mistakes can accumulate. These controls may lower the apparent automation rate: some blocked tasks should never have been autonomous.

Fourth, recover. Recording which steps completed, queuing payments awaiting confirmation and letting a human resume at the failure point is usually cheaper than rerunning the whole conversation. Financial remediation cannot assume every action can be undone. Define in advance which cases require escalation.

Only then comes acceptance. τ-bench evaluates completion using the database state after a conversation, a more concrete measurement approach than checking a reply alone. We draw on that method, without extrapolating 2024 benchmark scores to current capability.E01 A customer going silent does not prove that a refund arrived. More meaningful acceptance criteria are policy compliance, confirmed payment state, consistent order state and observation of repeat complaints within an agreed window. Satisfaction and completion time still need separate measurement.

Anthropic's evaluation methodology also separates a claim of completion from the actual final state of the environment, and emphasizes different grading methods and human calibration.E05 These engineering steps turn uncontrolled losses into manageable costs. But API integration, permission maintenance, exception handling and quality sampling still require people. Their absence from a demo does not remove the expense.

What does an accepted task actually cost?

The following assumptions are a scenario in Chinese yuan (CNY), not industry averages or vendor quotes. They deliberately retain a conservative early-deployment policy: every task receives human review.

Each month, 10,000 similar tasks enter the system. The agent completes 80% and passes initial review. The remaining 20% are escalated, and 95% of those ultimately pass acceptance. Final accepted tasks therefore total 10,000 × [80% + 20% × 95%] = 9,900.

There are four cost items:

• Model, tool and retry costs average CNY 0.60 per incoming task. Retries are already included; do not count them again.
• Initial review costs CNY 1.20 per task and covers all 10,000 tasks, not just failures.
• A task that fails initial review incurs CNY 8 in additional takeover costs, including escalated processing and final verification. This cost applies whether escalation succeeds or fails; initial review is not counted again.
• Monthly engineering maintenance and system costs are CNY 20,000, assumed constant within this volume range.

Total cost is 10,000 × (0.60 + 1.20 + 20% × 8) + 20,000 = CNY 54,000. Cost per finally accepted task is 54,000 ÷ 9,900 ≈ CNY 5.45. Failed tasks still consume resources, so their costs remain in the numerator.

Suppose a comparable manual process costs CNY 6 per incoming task, including processing, quality control and maintenance, and achieves 99% final acceptance. That is about CNY 6.06 per accepted task. The agent scenario saves only about 10%. This also assumes accurate initial review, no missed errors, compensation or later repeat contacts, and no additional losses from failed escalation. It is a cost model that requires quality auditing, not a basis for approving deployment from accounting savings alone.

The most useful output is the threshold. Let p be the share that passes initial review after autonomous execution, while all tasks still receive human review. With other parameters unchanged, cost per accepted task is:

(11.8 − 8p) ÷ (0.95 + 0.05p)。

To beat the manual comparator, p must exceed about 72.8%. At p = 60%, cost rises to about CNY 7.14. At the baseline 80% pass rate, monthly volume must exceed roughly 7,692 tasks to spread fixed costs sufficiently. This threshold does not apply at larger scales that require a bigger maintenance team.

Even halving model and tool costs saves only about CNY 0.30 per accepted task. A small change in quality or process can matter more than an API price cut. If review is fallible, measure the takeover costs of false rejections and losses from missed errors, then recalculate successes confirmed by independent sampling. A vendor dashboard's resolved count cannot simply replace the denominator.

Who captures the time customers save?

Evidence from real work provides support and limits the claim. The 2024 revised version of Generative AI at Work covers 5,172 customer-service agents. AI assistance increased issues resolved per hour by 15% on average. This supports higher output from an assistive tool in that setting; it does not prove that autonomous customer service cuts costs by the same amount, or that profits rise 15%.E06

Saved staff time becomes profit and cash only when it reduces outsourcing, overtime or replacement hiring after attrition, or when the same resources serve more paying demand. Otherwise, the immediate result is spare capacity. Deployment costs usually arrive first, while wages and long-term contracts may not change immediately.

The vendor's economics differ from the customer's. Salesforce's May 2025 Flex Credits announcement priced a standard action at the equivalent of USD 0.10. That was the announcement price, not a current contract quote. Action-based charging can generate more consumption when one issue needs five retries, while the customer still wants one resolution. Outcome-based charging can reduce the mismatch, but shifts negotiation to how resolution is defined and who bears errors.E09

The operating variables to track are net new revenue from a comparable task, what remains after model, implementation, support and exception costs, and willingness to pay at renewal. Each of the following three companies has a plausible position from which to charge—and an unresolved question.

Salesforce: Can access to business state create net new revenue?

Customer records, service workflows and operational permissions place Salesforce close to acceptance outcomes. This is an inference about a potential mechanism of advantage.

The company's August 2026 FY2027 Q2 disclosure reported Agentforce ARR above USD 1.5 billion, but the definition began including Slackbot and Headless 360 that quarter. AWU also combines resolved cases, record updates and triggered workflows. ARR is an annualized contract metric; AWU aggregates unlike activities. Neither directly establishes profit from autonomous execution.E10

Salesforce captures incremental revenue if customers add budget for execution and reusable product revenue covers model and delivery costs. If credit budgets mostly replace existing seats, a new label may grow faster than new revenue. Track renewals on a consistent definition, actual consumption and expansion, while separating contract scale, revenue recognition and cash collection.

Microsoft: Two revenue layers, two cost layers

Microsoft can charge for application subscriptions and cloud usage, with existing identity, distribution and office entry points reducing adoption friction. Its FY2026 Q4 earnings call reported more than 30 million paid M365 Copilot seats and emphasized a seat-plus-consumption approach. The same material reported Microsoft Cloud gross margin of 65%, down year over year, with management citing business mix, AI infrastructure and usage growth among the factors.E11

This supports commercialization at scale, not high standalone agent profitability. Copilot seats are not active autonomous-agent users, and cloud margin changes cannot all be attributed to agents. Infrastructure also requires cash, while depreciation and capital expenditure occur at different times. Assess usage revenue, cloud costs, capital expenditure and retention together to understand cash conversion.

ServiceNow: Can governance become product revenue?

ServiceNow sits close to tickets, approvals and enterprise workflows, providing a basis for reusable governance and execution products. Its 2026 Q2 disclosure reported AI ACV above USD 1 billion, subscription revenue of USD 3.877 billion and subscription GAAP gross margin of 73.5%. AI ACV is a contract metric. Overall subscription revenue and margin do not isolate the contribution of autonomous agents.E12

Reuse is the key variable. Selling the same permissions and recovery system to more customers can grow revenue faster than delivery costs. Rewriting workflows for every customer looks closer to a labor-intensive service. If cloud providers bundle governance, ServiceNow may still improve retention without being able to charge a separate premium. Test net additional contract value on the same base product, renewals and implementation costs, rather than the number of products bearing an AI label.

What evidence would change this thesis?

The first counterexample is measurement failure. METR's small 2025 randomized experiment found that developers familiar with their own projects took 19% longer with the AI tools then available. Its 2026 follow-up, however, identified serious selection bias, changed the design and reported new estimates whose uncertainty intervals crossed zero. Together, these sources warn that subjective impressions, old tools and specific populations cannot substitute for a matched comparison today.E07E08

The second counterexample directly challenges the profit-allocation thesis. In April 2026, Anthropic introduced Managed Agents with persistent session logs, a runtime harness and isolated environments as separate components, including recovery design. Model platforms are entering the reliable-execution layer. Log recovery alone does not make external refunds happen exactly once. But standardizing these capabilities could narrow the engineering differences that application vendors can charge for.E04

Watch for three types of evidence, rather than a grand date for the agent era:

1. A deployment comparison on the same task distribution: full costs fall without worse repeat contacts, compensation or timeouts. If inexpensive general-purpose models with simple rules keep winning, reduce the expected premium for complex execution platforms.
2. Actual renewals: additional payment under a consistent definition tracks real usage, with incremental profit after implementation and runtime costs. If adoption mainly reallocates existing budgets, reduce the expected expansion of the revenue pool.
3. Delivery across customers: implementation time and exception-handling costs fall for the second and fifth customer. If every deployment starts from scratch, supposed platform advantage may be project experience.

For deployment teams, begin with a well-bounded task whose final state can be verified, and calculate full costs. For company researchers, follow the money through that same task: who bears failure, defines acceptance, receives new budget and pays for compute and delivery?

Reliable execution has economic value. Whether a company keeps that value on its own ledger still depends on contracts, renewals and cash flow.

Evidence and definitions

Sources were checked through 12 Oct 2026. Engineering recommendations and value-allocation arguments are the author's inferences; the model is hypothetical. The table indexes original sources. Vendor disclosures are not independent causal verification. This article contains no valuation or trading recommendation.