How to Build Your First AI Agent Without It Falling Over

You read the step-by-step post, spent a Saturday on it, and now there is a thing on your laptop that reads an email, works out what it is about, and writes a reply. It worked six times in a row. The obvious next move is to point it at the real inbox, and something sensible in you is saying don’t.

That instinct is right, and not because your code is bad. When you build your first AI agent, the hard part is not the model or the framework. It is scoping the task narrowly enough that the agent can be reliable, and designing what happens when it is wrong. No tutorial covers that, because it is specific to your business and not fun to write about. One consequence before you start: most business problems you are about to point an agent at do not want an agent.

Workflow with LLM steps, or an actual agent?

Anthropic’s engineering write-up on building effective agents (December 2024) draws the line cleanly. Workflows are “systems where LLMs and tools are orchestrated through predefined code paths.” Agents are “systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.”

In a workflow, you decided the steps and the order, and the model does the bits needing judgement. In an agent, the model decides what to do next, possibly for many turns, and you find out afterwards. Take invoices. The workflow version: a PDF lands, you extract fields with a model, validate them against a schema, look the vendor up with ordinary code, apply the approval rule with an if-statement, post to accounting. Four deterministic steps, one model call. The agentic version: hand the model the PDF plus tools for lookup, validation and posting, say “process this invoice,” and let it work out the order.

The second demos better. The first is what you want in production, because every failure is localised: if extraction goes wrong you know which step to open, you have its input and output, and you fix it with a schema change rather than a prompt change. That same document gives advice most people skip: add agentic complexity “only when simpler solutions fall short.”

Real agency earns its place when the number of steps genuinely cannot be known in advance: researching across sources until you have enough, debugging, triaging a case that might need one lookup or nine. If you can draw the flowchart, do not build an agent. Build the flowchart and put the model in the boxes that need a brain.

Picking a first use case that will actually survive

Most failed first agents failed at selection, not engineering. Five properties make a task winnable:

• High volume. Dozens of real cases a week, or you never accumulate enough evidence to know whether it works. Ten a month tells you nothing.
• Low stakes per instance. One wrong output should cost an apology and two minutes, not a client or a regulatory filing.
• A success definition you can write in one sentence. “The correct SKU and quantity were written to the order” is checkable. “The customer felt heard” is not.
• An existing human process to copy. If Priya does this forty times a day, watch her, ask what she checks, steal her decision rules. If nobody does it today, you are designing a process and automating it at the same time, and you will get both wrong.
• A cheap way to check the output. A deterministic check (does this reference a real order number?), or a human glancing for five seconds. If verifying costs as much as producing, automation buys you nothing.

The reliable failures: unsupervised external communication; anything touching money movement, contracts or medical decisions; anything where the input is a vague request with no fixed shape (“handle whatever comes into this inbox”); anything needing knowledge scattered across six systems nobody has joined up. And anything where the current process is undocumented because the person doing it is improvising. That last one is the most common and the least obvious.

Good first projects, roughly by hit rate: classify and route inbound messages; extract structured fields from recurring documents; draft a first-pass reply a human sends; reconcile two lists and flag the exceptions.

The anatomy of a simple agent, minus the jargon

Strip the framework vocabulary and there are five parts. Build all five in one file at least once before adopting anybody’s abstraction, so you know what the framework is hiding.

Instructions. What it is for, what it must never do, what “done” looks like, the edge cases you know about, the shape of the answer. Write it like an SOP for a new hire, because that is what it is.

Tools. Functions the model can call: look up a customer, search the knowledge base, create a ticket. Each is a name, a description, a typed input schema, and your code behind it. The model never touches your systems. It asks; your code decides whether to comply.

State. What the model can see this run. In most business agents “memory” is just the transcript of this one task plus whatever you deliberately loaded. Persistent cross-run memory is a later problem and usually an early mistake.

A stopping condition. Two, in fact: success (the required output exists and validates) and a hard cap (six tool calls, ninety seconds, or $0.40, whichever hits first). Without the cap you eventually get a loop calling the same search with slightly different words until your bill notices.

An evaluation step. Something, code or human, that decides whether the output was right and records the verdict where you can count it later. An agent with no evaluation step is not a system. It is a demo with a cron job.

Tool design is where most of the quality lives

When an agent misbehaves the instinct is to rewrite the prompt. Usually the prompt is fine and the tools are wrong.

Anthropic’s tool documentation is blunt: “Provide extremely detailed descriptions. This is by far the most important factor in tool performance.” It recommends at least three to four sentences per description, more for complex tools, covering what the tool does, when it should and should not be used, what each parameter means, and the limitations. On naming, prefix by service (github_list_prs, slack_send_message) so selection stays unambiguous as your library grows, and consolidate related operations into fewer tools rather than proliferating near-identical ones.

• Narrow beats general. get_open_invoices_for_customer(customer_id) is a better tool than query_database(sql). The second is more powerful and will hurt you more.
• Validate inside the tool, not in the prompt. If the date must be within 90 days, check it in code and return an error the model can act on. Asking politely in the system prompt is not validation.
• Make errors instructive. “Customer not found. Try search_customers_by_name first” gets recovered from. “Error 500” gets retried identically four times.
• Return small payloads. Dumping a 200-field API response into context costs money on every later turn and buries the three fields that mattered.
• Force the output shape. OpenAI’s Structured Outputs documentation says schema-constrained generation “ensures the model will always generate responses that adhere to your supplied JSON Schema, so you don’t need to worry about the model omitting a required key, or hallucinating an invalid enum value,” and recommends it over plain JSON mode. Use the equivalent on your provider, then validate again on your side. Schema adherence is not the same as being correct.

Grounding it in your own data

Your agent needs your refund policy, your catalogue, your account history. Two routes get confused constantly, so: retrieval means fetching the relevant text at question time and putting it into the prompt; fine-tuning means adjusting the model’s weights on examples so it behaves differently by default.

Retrieval is the right first move for nearly every business case. Policies change, and updating a document is free where retraining is not. You can show a human exactly which paragraph produced an answer, which matters the first time someone disputes an output. And it fails visibly: if the right chunk was not retrieved, the logs say so. OpenAI’s guidance on optimising accuracy frames the choice as diagnosis. Retrieval solves a missing-context problem, where “the model lacks contextual knowledge because it wasn’t in its training set, its knowledge is out of date, or it requires knowledge of proprietary information.” Fine-tuning solves a learned-behaviour problem: consistency on a task, or efficiency. Their own case study found adding retrieval on top of a fine-tuned model reduced performance, so the two are not additive.

Retrieval quality is mostly unglamorous document work: chunk along real boundaries like sections and clauses rather than fixed character counts, keep the source and date on every chunk, and delete the obsolete policy versions still on your shared drive. Test retrieval on its own, by asking twenty real questions and checking whether the right document came back. If retrieval is wrong, the model is answering from the wrong page, and no prompt work fixes that.

Guardrails, in the order they matter

Build these before features. Retrofitting means a live, unguarded period, which is exactly when the story you will be telling for years happens.

• Input validation. Reject what you were not designed for at the door: wrong document type, missing account number, a language you have not tested. Most spectacular failures start with an input nobody imagined.
• Output schema validation. Nothing leaves without parsing against a schema and passing business rules. Quantity must be positive. The order ID must exist in your database. Check the second part, not just the first.
• A hard step limit. Pick what the task actually needs, add two. Hitting it is a failure: logged and escalated, never a silent retry.
• A cost ceiling, per run and per day, that pages someone rather than just stopping.
• Permissions. Its own service account, narrowest access that works, read-only wherever possible, never a human’s credentials, and an audit trail that names the agent. If your answer to “what could it touch if the prompt were manipulated?” is “everything that account can,” fix that first.
• A human approval gate on anything irreversible: external mail, money, deletions, contract changes, publishing. The agent prepares the action and stops. A person clicks.

People talk themselves out of that last one because it looks like it defeats the purpose. It does not. The value was never the click. It was the fifteen minutes of gathering, reading and drafting before it.

Evidence: test sets, logs and the bill

Build the test set before the agent

Collect 20 to 50 real cases from the last few months with their known-correct outcomes, including the weird ones on purpose: the malformed input, the duplicate, the customer with two accounts, the request that should be refused. An afternoon of work, and the highest-value afternoon in the project. Run the whole set after every change and record the pass rate. “It worked when I tried it” is not evidence, because these systems are non-deterministic: the same input can succeed and fail on different runs. One case run once tells you the task is possible, not that it is reliable.

Plan around a real reliability ceiling. The τ²-bench paper (arXiv:2506.07982, June 2025) evaluates agents where both the agent and a simulated user act on a shared environment, closer to real support work than a single-actor benchmark. Two findings matter for your design. Difficulty is sharply domain-specific: gpt-4.1’s pass^1 “drops from 74%/56% for retail and airline respectively to 34% for telecom.” And performance fell as the number of actions rose, “reaching close to zero for > 7 actions” in their default mode. Those figures are tied to mid-2025 models and benchmarks age fast, so read them as a snapshot, not current capability. The shape is the durable part: more autonomous steps, sharply worse odds. Arithmetic agrees. A step right 98% of the time, twelve deep, gives a clean run about 78% of the time. That is the real argument for narrow scope, and it is empirical rather than a matter of taste.

Log enough to debug a run

Per run, store: run ID, input, every model call with the tool requested and its arguments, raw tool results, token counts, total cost, elapsed time, final output, exit reason (success, schema failure, step limit, error, escalated), and the verdict once someone checks. An hour’s work before launch, impossible to reconstruct after a bad week. When someone asks why it said that, you open one run and read it top to bottom.

Where the money goes

Loop cost does not grow the way you expect. Each turn resends the accumulated conversation, so spend rises roughly with the square of the step count, not linearly. If the first call sends 2,000 tokens and each tool result adds 1,500, then by step ten you are sending about 15,000 tokens in that one call, and total input across all ten is around 85,000 rather than 15,000. A verbose tool returning a full API object instead of four fields is not a small inefficiency. It multiplies across every remaining turn.

So: trim tool outputs, and use prompt caching if your provider offers it. Anthropic’s documentation prices cache reads at 0.1 times the base input token price, with five-minute cache writes at 1.25 times and one-hour writes at 2 times, subject to a minimum cacheable prefix that varies by model. A large static instruction block read every turn is exactly that shape. Run your test set with cost logging on and you have a real per-run number before committing.

A realistic plan to build your first AI agent

Week one, no code. Pick the task against the five criteria. Sit with the person who does it now and write down every check they make, including the ones they do not think of as checks. Collect 30 real cases with correct answers. Get the process owner to agree the one-sentence success definition in writing.

Week two, build the boring version. Deterministic pipeline, model only where judgement is needed, output schema enforced, every run logged, no loop yet. Run the 30 cases. Below roughly 80% here, your problem is scoping or data, and adding autonomy will make it worse.

Week three, shadow mode. Live inputs, real outputs, nothing shipped. A human works as normal and you compare. This is where the input shapes you never imagined show up, and it is non-negotiable.

Week four, supervised live. Agent drafts, human approves, every approval and edit recorded. After a few hundred cases you have an accuracy number and a list of what it gets wrong. Only then consider letting the highest-confidence, lowest-risk slice through unreviewed, with sampling on top.

Design the handoff while you are there. When the agent escalates, the human should get the original input, what it tried, what it found, its draft and why it stopped, all in one place. An escalation that just says “couldn’t handle this” makes the human redo the work, and then they stop trusting the queue. Escalate on low confidence, unhandled input types, step or cost limits, validation failures, and anything the process owner flagged as sensitive. Track the escalation rate: if it climbs, something upstream changed.

Questions that come up next

Should I use the biggest model? Start on a strong one, so you learn whether the task is possible at all. Once the test set passes, try a cheaper model and compare pass rates. A smaller model with better tools and tighter scope frequently beats a bigger one doing loose work.

What if our data cannot go to a model provider? That constraint is a design input, not an afterthought: enterprise terms with zero retention, a hosted model in your own cloud tenancy, or self-hosted open weights with more engineering attached. Settle it before you build, because it changes the architecture.

If you would rather build this alongside someone than learn it by breaking things, AB7 Solutions does exactly this work: scoping the first use case honestly, assembling the real-case test set, building the deterministic pipeline with tool validation, retrieval over your own documents, and the logging and approval gates that let you see what happened. We also take the part most teams underestimate, which is running it after launch. Call +1 321 341 7733, or email ab@ab7solutions.com or director@ab7solutions.com, and there is more on our agentic AI and automation work at ab7solutions.com.

Sources: Anthropic Engineering, “Building effective agents,” 19 Dec 2024; Anthropic docs: defining tools; Anthropic docs: prompt caching; OpenAI docs: Structured Outputs; OpenAI docs: optimizing LLM accuracy; τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment (arXiv:2506.07982, June 2025).

Related reading

• How to Choose an AI Agent Framework in 2026 Without Regretting It
• How Much It Costs to Hire AI Agent Developers
• You Shipped an AI Agent. Have You Tried to Break It Before Someone Else Does?

Related AB7 service: AI automation and AI agent services.