Three quotes are open on your desk. One says $8,000 and four weeks. One says $46,000 and ten weeks. One is a rate card offering an “AI agent engineer” at an hourly number that would make a corporate lawyer blush. All three describe the same scope line: an agent that reads incoming requests, looks things up in your systems, and acts on them. Nobody has explained the factor of six.
When you hire AI agent developers, the hourly rate is the smaller and by far the most predictable part of what the project costs you. What blows the budget is the evaluation harness that tells you whether the thing works, the integration into the systems the agent must actually touch, and the per-run inference cost that keeps arriving every month after the build is finished. The $8,000 quote is usually not cheaper. It has just left those three out.
So stop comparing rates. Work out which components each quote has actually priced.
The three numbers in every agent project
- Build. One-off. Prompt and tool design, the control loop, retrieval if you need it, the interface. It looks like ordinary software work, maps cleanly onto an hourly rate, and is the part vendors estimate best. On honestly scoped projects it is also the smallest of the three.
- Integration. One-off in theory, routinely underestimated. Every system the agent reads from or writes to is a negotiation with someone else’s authentication model, permission model, rate limits and error behaviour.
- Run cost. Recurring, scaling with usage rather than with the build. Inference per step, retries, caching, trace storage. A build you paid for once, running ten thousand times a month, carries a bill unrelated to what the build cost.
These do not trade off against each other, with one exception: cutting the evaluation work makes everything downstream worse. A cheap build with no evals is not a cheap agent. It is a demo that worked on the six cases the developer tried and no evidence about the seventh, which means a system nobody can responsibly switch on. You then either turn it on anyway and meet the failure modes in production, or leave it off and write off the build. The second is cheaper.
What sets the rate when you hire AI agent developers
“AI agent developer” is not an occupation. It is three skill sets sold under one title.
A software engineer fluent with model APIs. Understands retries, idempotency and queues, and treats a model call as one more unreliable network dependency. Most agent projects need this profile most. The nearest published US benchmark is the BLS profile covering software developers (SOC 15-1252, published with QA analysts and testers, 15-1253): median annual wage $135,980 in May 2025.
An ML or applied-AI engineer. Can fine-tune, compare models, reason about retrieval quality below the API surface. BLS publishes data scientists at SOC 15-2051, median $120,230 in May 2025, and computer and information research scientists at SOC 15-1221, median $140,300, master’s degree typical for entry. That last category is tiny: 38,600 jobs nationally against 1,905,400 in the software developer profile.
A data-literate person who can build the eval set. Pulls three hundred real cases from your ticket system, decides the correct answer for each, argues with a subject expert about the ambiguous forty. BLS publishes no occupation for this. The honest adjacency at the clerical end is general office clerks (SOC 43-9061), median $45,010 in May 2025, and it understates the role, because the hard part is domain judgement rather than typing. Treat it as a floor, not a rate.
All four figures are US employee wages, not agency rates; a vendor bills overhead, management and margin on top. Use them to compare seniority between profiles, not to predict an invoice. And the opinion I will defend: most projects need profile one and profile three far more than profile two. If your agent calls a frontier model through an API, nobody is training anything. The work is engineering plus evidence. Paying research-scientist rates for prompt iteration is an expensive mistake, and so is cutting the eval-set person.
The evaluation harness is a deliverable, not a phase
Ask for it as a named, priced, scheduled deliverable with an acceptance test. Three parts.
A labelled test set of real cases. Not synthetic examples the developer wrote. Real inputs from your own history, each with the correct outcome recorded, including the messy ones and the ones where the right answer was “escalate to a human”. Thirty cases is a smoke test. A few hundred is a test set. The labelling consumes your domain expert’s time, and that cost belongs in the budget.
An automated scoring method. Takes a run, returns pass or fail, no human reading required. For structured output, field-level comparison. For free text, usually a rubric applied by a model, spot-checked against human judgement so you know how far to trust it. If scoring needs a person to read every run, you do not have a harness. You have a habit.
A regression run before every change. Every prompt edit, model version bump and tool change reruns the set and you watch the score move.
If budget is tight, shrink the test set rather than dropping the harness. Eighty labelled cases with automated scoring beats a thousand nobody scores.
Researchers got to the same place from the other direction. In AI Agents That Matter (Kapoor, Stroebl, Siegel, Nadgir and Narayanan, July 2024), the authors find that “many agent benchmarks have inadequate holdout sets, and sometimes none at all,” which lets agents become “fragile because they take shortcuts and overfit to the benchmark in various ways.” They also argue for optimising cost and accuracy together rather than accuracy alone, which yields agents that are “needlessly complex and costly.” That is the buyer’s problem in academic dress.
So say it plainly before you sign. An agent delivered without a labelled test set and an automated scoring method cannot be maintained by anyone, including the people who built it, because nobody can tell whether tomorrow’s change made it better or worse.
Integration is the real cost centre
“Connect it to our CRM” is not a task. It is a project, and this is why.
- Authentication. Which flow, whose credentials, where the secret lives, what happens when it expires at 2am. Service accounts need provisioning by someone with authority to grant them, and that person has a queue.
- Permissions. An agent acting as a super-user is a liability you cannot explain to an auditor. Scoping it to the exact objects and fields it needs means someone must know your data model well enough to say no.
- Rate limits. Real APIs throttle. Retry logic has to back off rather than hammer, and a loop fanning out across records finds the ceiling faster than a human user ever did.
- Idempotency. The agent creates a record, times out before seeing the confirmation, retries, and now there are two. The fix is a client-generated idempotency key on every write, honoured downstream or faked with a deduplication check where the vendor API has no such concept.
- Partial failure. It updated the CRM, sent the email, then failed on the ledger write. What is the state of the world, who knows, and what undoes it? The compensation path for each multi-write sequence is the most common omission from a cheap quote.
None of that is AI. All of it is why the integration line is larger than the build line on honest estimates. A vendor quoting “CRM integration, two days” has either done it against that exact system before, in which case ask them to name it, or has not thought about it.
How to model run cost without quoting token prices
Nobody should hand you a per-token rate as a fixed input; published rates change and models get superseded. Model it from your own step count: runs per month × calls per run × (input plus output tokens per call), adjusted for retries and caching, then multiplied by the provider’s current published rate.
- Loop step count is what people get wrong by an order of magnitude. One classification is one call. An agent that plans, calls three tools, reads the results and writes a summary is five or six, each carrying the accumulated transcript, so cost grows faster than linearly with loop length.
- Retries multiply everything, and they peak exactly when things are going badly.
- Caching is the biggest lever you control. Anthropic’s pricing documentation prices cache reads at 0.1x the base input rate, with cache writes at 1.25x for the five-minute cache and 2x for the one-hour cache. Put the stable prefix first and you pay a fraction for the bulk of your input.
- Batching helps anything not user-facing. The Message Batches API is charged at “50% of the standard API prices,” most batches finish inside an hour, and there is a 24-hour ceiling. An overnight queue can halve your inference bill through a scheduling decision.
- Tool charges sit outside token pricing. Server-side web search bills per thousand searches on top of the tokens it produces.
- Observability storage. You want the full input, output and tool trace for every run, because that is how you debug and how you build next quarter’s eval set. Traces are verbose, and at volume they need a retention policy set by someone who knows your data-protection obligations.
Keep the rate in one spreadsheet cell you update from the provider’s own pricing page, and take the per-run token count from an instrumented pilot rather than a guess. Then divide by successful outcomes, not runs. That is the number that says whether the agent is worth running.
Four engagement shapes, and what each suits
Fixed-scope proof of concept. Right when you do not know whether the task is tractable and the answer is worth paying for alone. Insist the eval set is in the deliverable, or you finish with a demo and no evidence. Wrong for anything open-ended: fixed price plus vague scope means the vendor ships the thinnest thing that satisfies the sentence.
Time and materials with a named engineer. Right for most first real builds, because the integration surface is unknowable until someone is inside your systems. Protections: a named individual rather than a pooled resource, a weekly demo, the ability to stop. The risk is drift; the cure is the eval score as your progress metric instead of a status narrative.
Staff augmentation into your own team. Right when you have engineers and a backlog but no agent experience. Cheapest over two years, slowest to start, and it works only if your people have capacity to absorb the knowledge.
Managed service. Someone else runs it, monitors it, fixes the drift. Right when the agent matters but is peripheral to what your company is for. You are renting; read the exit terms before signing.
A strong freelancer is usually the cheapest route through the build and the worst route through the next two years, because one person cannot cover an on-call rota and an undocumented handover costs you later.
Whichever you pick, price maintenance from day one, because agents decay in a way ordinary software does not. Model versions get deprecated and their replacements behave differently on your edge cases. Prompts tuned for one model degrade on the next. Upstream APIs change fields, retire endpoints and quietly alter error semantics. Your business rules change and nobody tells the agent. Expect a standing allocation and a regression run on every model change. A vendor with no maintenance line is not being generous.
The cost buyers forget: the human in the loop
Almost every agent worth deploying has a fraction of runs a person must review: low-confidence classifications, high-value actions, anything a regulator or customer might ask you to justify. Buyers price the build and the tokens, then act surprised that the reviewing needs a rota.
The arithmetic takes a minute and almost nobody does it. If 12% of 4,000 monthly runs need review, that is 480 reviews; at four minutes each, 32 hours a month. Most of a working week, from somewhere. Whose week? What covers their leave? At what queue depth should the agent stop accepting work rather than build a backlog nobody will clear?
This is an operating cost forever, not a launch-phase cost. The review fraction should fall as you fix the failure modes your traces expose, and it will not reach zero. Reviewers are also your best source of new eval cases, but only if someone builds the path from “reviewer corrected this” to “this case is in the test set”.
A cost model you can fill in this afternoon
Substitute your own numbers. The figures below are arbitrary placeholders showing the shape of the arithmetic, not quoted market rates.
- Build: engineer days × day rate, the day count split into prompt and loop design, tool implementation, and interface.
- Integration: distinct systems × days per system, higher for custom auth flows and higher again for anything the agent writes to. Three systems at five days is fifteen days. If this line is smaller than the build line, ask the vendor to walk you through partial-failure handling on the riskiest write.
- Eval harness: labelling hours (cases × minutes per case ÷ 60, mostly expert time) plus engineer days for scoring and the regression runner. Three hundred cases at six minutes is 30 expert hours, which gets spent either way, so put it in the plan.
- Run cost per month: runs × steps × tokens per step × retry factor × (1 − cache saving) × current published rate, plus per-use tool charges, plus trace storage.
- Human review: runs × review fraction × minutes per review, converted into a named person’s time.
- Maintenance: a monthly allocation for regression runs, model migrations and upstream breakage.
Total the recurring lines, multiply by twelve, compare with the one-off lines. At real volume the twelve-month recurring figure is usually the larger one, and that is the whole point. Do the division before you sign, because it changes which engagement shape to pick.
Then ask any vendor these three. They separate people who have shipped an agent to production from people who have shipped demos.
- “Show me the eval set from something you shipped: how many cases, who labelled them, the pass rate at handover, and the last change that pushed the score down.” Anyone who has done this has opinions about labeller disagreement. Anyone who has not talks about accuracy abstractly.
- “A run updates our CRM, sends an email, then fails on the third write. Walk me through what happens.” Listen for idempotency keys, a compensation path, and an alert with a human at the end. Vagueness here is the most reliable warning sign.
- “What did your last production agent cost per successful outcome, and what happened to that when the model version changed?” A real answer has a number, a caching strategy and a migration story. “It’s very cheap” is not an answer.
If that model convinced you the evidence layer and the integration work are where you need help, that is the right conclusion, and it is the work AB7 Solutions does. We build agentic AI in this order: the labelled eval set and automated regression harness first, then the CRM, ERP and internal-API integration priced as its own line with idempotency and partial-failure paths designed rather than discovered, then the data annotation and human-in-the-loop operation that keeps the review queue moving and feeds corrections back into the test set. We can also place engineers into your own team under contract staffing or staff augmentation if you would rather own the capability. And if the honest answer after we look at your process is that you want a deterministic workflow with two model calls instead of an agent, we will say so. Call +1 321 341 7733, or email ab@ab7solutions.com or director@ab7solutions.com. There is more on our agentic AI, automation and AI data operations work at www.ab7solutions.com.
Sources: BLS Occupational Outlook Handbook: Software Developers, Quality Assurance Analysts, and Testers (median $135,980, May 2025); BLS OOH: Data Scientists, SOC 15-2051 (median $120,230, May 2025); BLS OOH: Computer and Information Research Scientists, SOC 15-1221 (median $140,300, May 2025); BLS OOH: General Office Clerks (median $45,010, May 2025); Kapoor, Stroebl, Siegel, Nadgir and Narayanan, “AI Agents That Matter” (arXiv:2407.01502, July 2024); Anthropic docs: pricing and prompt-caching multipliers; Anthropic docs: Message Batches API.