The AI Agent That Assumes: Why Automation Needs to Ask, Not Guess

A developer posted recently about building an AI agent workflow on AWS, and the hardest part wasn't the agent itself. It was getting it to stop assuming things. Give it a task with a gap in it, and it doesn't stop and ask. It fills the gap with a guess and carries on, confidently, as if the guess was fact.

I see this pattern constantly when I'm asked to build or fix automation that involves AI agents making decisions inside a business process. The agent isn't broken. It's doing exactly what it was designed to do: produce an output. The problem is that producing an output and producing the correct output are two different things, and nothing in most agent setups forces a distinction between them.

Where the assumptions actually creep in

It's rarely dramatic. It's small, boring gaps in the data.

• An invoice matching agent gets a supplier name that's slightly different from the one on file (a trailing "Ltd", a missing space) and decides they're the same supplier rather than flagging a mismatch.
• A booking assistant gets "next Tuesday" with no year and no timezone context, and picks one rather than confirming it.
• A lead-routing agent gets a form submission with an empty "budget" field and assigns it to the smallest package rather than marking it as unqualified for a human to check.
• A support triage agent gets an ambiguous request that could be a refund or a complaint, and picks whichever category it processed most recently in training, without asking which one applies.

None of these look like failures when you glance at the output. The invoice gets matched. The booking gets made. The lead gets a reply. Everything appears to be working. The trouble only becomes visible weeks later, when someone notices the wrong supplier got paid, or a customer turns up on the wrong day, or a big lead sat in the wrong queue for a fortnight.

Testing doesn't catch this, because testing uses clean data

When I build automation, I test it against realistic scenarios, but "realistic" and "messy" aren't the same thing. Test data tends to be complete. Real business data has missing fields, inconsistent formatting, duplicate entries and the occasional entry someone typed while distracted. An agent that performs well against a demo dataset can still make confident, wrong assumptions the first week it touches production data, simply because production data is where the gaps actually live.

This is why a demo that looks impressive isn't proof that a workflow is safe to run unattended. The demo shows you what happens when everything is present and correct. It tells you nothing about what happens when it isn't, and in a live business, something is always missing or inconsistent.

Designing automation that stops and asks

The fix isn't to avoid AI agents in automation. It's to build the process so that ambiguity produces a pause, not a guess. In practice that means:

• Defining what "unknown" looks like for each field. If a date, an amount, or an identifier is missing or doesn't match an existing record with high confidence, the workflow should route to a person rather than pick the nearest plausible value.
• Making low-confidence matches visible, not silent. A fuzzy supplier match or a guessed category should be logged and flagged, so someone can review it in minutes rather than discover it in an audit.
• Setting a threshold, not a default. Instead of "if in doubt, do X", the rule should be "if in doubt, stop and show me the doubt". That one change removes most of the silent failure risk.
• Keeping a record of what was assumed, when it did happen. If an agent does have to make a judgement call, log the reasoning alongside the outcome, so it can be checked later without reverse-engineering the decision.

None of this is exotic. It's the same discipline as validating a web form, applied to a process that now includes an AI step. The difference is that a form rejects bad input immediately and visibly. An agent will often accept it and move on, which is precisely why the checkpoints have to be built in deliberately rather than assumed to exist.

What this means if you're commissioning this kind of work

If you're paying someone to connect an AI agent to a real business system, ask directly what happens when the input is incomplete or ambiguous. Not "does it work", but "what does it do when it doesn't have enough information". A vague answer is a warning sign. A specific one, describing where it stops, what gets flagged, and who sees the flag, tells you the person building it has actually thought about failure, not just success.

I build this kind of systems integration and automation for a living, and the projects that go wrong are almost never the ones where the AI got something obviously stupid wrong. They're the ones where it got something quietly, plausibly wrong, and nobody noticed until the pattern had repeated fifty times. Building in the pause is less impressive in a demo. It's the thing that keeps the automation trustworthy once it's running unattended in your actual business, on your actual, messy data.