Guardrails your agents can’t talk their way around

The prompt is not a policy

Most teams write their first guardrail as a sentence in the system prompt: “Never refund more than $200 without approval.” It works in testing. Then a customer writes a long, reasonable message explaining why their case is special, and the agent agrees with them.

That is not a model bug. A prompt is guidance, and a model weighs guidance against everything else it reads. If a rule must hold every time, it has to live somewhere the agent cannot argue with. In Kairon, that place is the policy layer, which checks every action after the agent decides and before anything executes.

Tell the agent the rule so it plans well. Enforce the rule somewhere it cannot reach.

Three controls that cover most cases

In about 200 customer deployments, we see the same three controls do most of the work.

Approvals

An approval pauses a run and asks a named person or group to confirm an action. Use them for anything that moves money, contacts a customer for the first time or changes access. Good approval rules are narrow. “All Payly refunds over $200” is clear. “Anything risky” is not, and will either block everything or nothing.

Spend limits

Limits cap what an agent can do over time, not just per action. A refund agent might be allowed $200 per refund, $2,000 per day and 10 refunds per customer per month. When a limit is hit, the run pauses and routes to a person with the totals attached. Limits catch the failure approvals miss: many small actions that add up.

Tool allow-lists

Every agent gets an explicit list of the tools and operations it may call. A support agent might read from Payly and Relay, write notes to Relay and draft refunds, but never delete records or change billing plans. If a tool is not on the list, the call fails before it leaves Kairon, whatever the agent intended.

Setting it up

Here is the order we recommend for a new agent:

1. Start with a read-only allow-list. Let the agent investigate and draft, nothing more.
2. Add write operations one at a time, each with an approval rule.
3. Set daily and per-customer spend limits before raising any per-action limit.
4. Run in shadow mode and count how often each rule would have fired.
5. Loosen approvals only where reviewers approve more than 98% of requests unchanged for two weeks.

Step five is the one teams skip. Approvals that never change anything slow everyone down and teach reviewers to click without reading. Approvals that catch real mistakes are worth keeping forever.

What to watch

• Policy blocks per hundred runs. A sudden rise usually means a prompt or tool change made the agent bolder.
• Approval wait time. If it creeps past an hour, reviewers are overloaded or the rules are too broad.
• Edit rate on approvals. High edit rates point to a missing tool or a vague policy.

For a real example of these controls in a regulated setting, read how Halcyon Health keeps a human in every refund.