Your AI Agent Can Draft a Form. Should It Also Submit It?

An AI model recently tried to fill out a visa application form on a US government website on its own. Another one submitted a tip to police about an unsolved murder, based on information it had no real way of verifying. Neither story is really about AI being clever or stupid. They're about the moment an AI system stops suggesting something and starts doing it.

That moment matters a lot more than most businesses treat it. A tool that drafts an email is low risk. The same tool that sends the email is a different thing entirely. The gap between those two is usually one unchecked step, and it's the step most automation projects skip because it feels like the obvious next upgrade.

Drafting and acting aren't the same risk

When an AI tool produces a draft, a person reads it before anything happens. Mistakes get caught in that gap. A wrong figure, a made-up detail, a tone that's off: someone notices, because they're looking at it before it goes anywhere.

Once you let the same tool submit the form, send the reply or file the report itself, that gap disappears. The AI isn't more reliable than it was when it was only drafting. You've just removed the thing that used to catch its mistakes. The output quality hasn't changed. The consequences have.

This is why "the AI can now do X automatically" is a sentence that deserves a second question every time: what happens the first time it does X wrong? Not if. When.

Where this shows up in ordinary businesses

You don't need a visa form or a police tip line to hit this. The same pattern turns up in plenty of smaller, duller places:

• A support agent that drafts replies to customers, then gets upgraded to send them automatically once it's "proven itself" for a few weeks.
• An AI tool that summarises supplier invoices and someone decides it may as well push the approved ones straight into the accounting system.
• A compliance assistant that drafts a report from your data, then gets connected to submit it to a regulator or a client portal directly.
• An agent that reads incoming leads and automatically books them into a calendar or fires off a quote, with no one checking the numbers it used.

Each of these started as something useful and low risk. Each one became a different kind of system the day it got permission to act rather than suggest, often without anyone deciding that on purpose. It usually happens gradually: someone notices the drafts are good, then one day flips a setting that lets it go straight through.

The checkpoint doesn't have to be a bottleneck

The fix isn't "never let AI act, always have a human approve everything." That just rebuilds the manual process you were trying to get rid of. The useful middle ground is deciding, deliberately, which actions need a person in the loop and which genuinely don't.

A few questions tend to sort this out quickly:

• Is the action reversible? An email that goes to the wrong person is hard to take back. A draft sitting in a queue costs nothing to wait on.
• Who outside your business sees the output? Internal notes and customer-facing or regulator-facing documents don't carry the same risk if they're wrong.
• Does the AI have a way to check its own facts? If it's generating a figure or a claim from its training rather than pulling it from your actual data, that's a flag for review, not automatic trust.
• What does it cost to be wrong once? A badly worded internal reminder is a shrug. An incorrect report filed with a client or a regulator is a different conversation.

Actions that score high on reversibility and low on external exposure can run without a person watching every one. Actions that don't should sit in an approval queue, even a very fast one, until you've genuinely seen it get things wrong and right enough times to trust the pattern.

Build the trail, not just the gate

Even for the actions you do let run automatically, you need a record of what the AI actually did, with what input, and why. Not a log line that says "email sent", but something that shows what it was responding to and what it decided to send. When something goes wrong, and eventually something will, that trail is the difference between a five-minute fix and a week spent trying to reconstruct what happened. I've written before about what actually makes an audit trail trustworthy, and it applies directly here: a timestamp and a status flag aren't enough once a system is acting on its own.

None of this is an argument against giving AI tools real permissions. A tool that only ever drafts things nobody reads is barely worth building. But the step from "it writes something" to "it does something" is a decision, not a feature flag, and it's worth treating it as one before an agent files something on your behalf that you'd never have sent yourself.