A customer-support agent is investigating a disputed payment. It has the conversation, an account summary, and the result of an internal billing lookup in context. The primary model times out, so the router sends the request to a fallback model.
The workflow recovers. The security decision may not.
The fallback could run through another provider, process data in another region, retain prompts under different terms, or expose a different set of tools. A retry policy built only for availability can quietly move customer data and delegated authority across a boundary that nobody approved.
Treat the switch as a fresh security decision. If the fallback cannot meet the policy attached to the original run, stop the run rather than weakening the policy to keep it alive.
a model route carries more than a prompt
A model call inside an agent is rarely just a block of user text. The request may include retrieved documents, tool results, memory, system instructions, customer identifiers, or a summary of earlier actions. The model may also decide which tool to call next.
That means a route change can alter two boundaries at once:
Data boundary
Which provider and region receive the current context?
Authority boundary
Which tools and actions can the fallback influence?
Suppose the primary route is approved for internal customer data in a named region. Its fallback is approved only for public documentation. Both models may be technically capable of answering the request. Only one is allowed to receive the account summary.
The router should not discover that difference after sending the payload.
This is the same reason I recommend testing the full tool path rather than only the agent’s final answer. A correct answer does not repair an unapproved data transfer or an overpowered tool call.
bind policy to the run before failover
The original request should carry a compact policy envelope. The router evaluates every candidate against it, including the primary model.
run_policy:
run_id: support-1842
purpose: investigate_disputed_payment
requester: support-user-381
data_classes:
- customer_confidential
- payment_metadata
required_region: eu
retention: no_provider_training
allowed_actions:
- billing.invoice.read
- support.case.read
forbidden_actions:
- billing.refund.create
- support.case.export
prompt_logging: redacted
policy_version: support-agent-12
expires_at: 2026-07-27T11:15:00Z
This envelope describes the run, not the preferences of a particular model. A fallback candidate must prove that it can obey the same constraints before it receives context.
Do not reduce the check to an allowlist of model names. The same model can be offered through different providers, regions, account configurations, and retention settings. A route that passed last month may no longer match today’s deployment or contract.
The candidate record should name the actual route:
candidate_route:
model: fallback-model-v3
provider: provider-b
endpoint: eu-production-2
processing_region: eu
approved_data_classes:
- public
- internal
retention_mode: thirty_day_logging
allowed_tools:
- support.case.read
policy_bundle: provider-b-eu-7
In this example, the candidate fails. It does not allow customer_confidential or payment_metadata, and its retention mode does not satisfy no_provider_training. The safe result is not a clever redaction attempted at the last second. The run stops and reports which constraint blocked recovery.
make the gate mechanical
The failover decision belongs in code and policy, not in an instruction asking the model to behave carefully.
candidate provider is approved
AND candidate region matches the run
AND every data class is allowed
AND retention terms satisfy the run
AND candidate tools are a subset of allowed actions
AND candidate policy version is current
AND run approval has not expired
If any mandatory check fails, deny the route before payload assembly. This ordering matters. The router should evaluate metadata first, then build the provider request only after approval.
It also needs a fail-closed response:
{
"outcome": "fallback_denied",
"run_id": "support-1842",
"candidate": "provider-b/eu-production-2",
"failed_checks": [
"data_class:customer_confidential",
"retention:no_provider_training"
],
"context_sent": false,
"tools_invoked": [],
"next_action": "request an approved route or human review"
}
context_sent: false is the important evidence. A policy denial after transmission is an incident record, not a successful control.
The same gate should narrow tool authority. Even when a fallback may receive the data, it should not inherit every capability available to the primary route by default. Tool grants need to match both the task and the candidate’s approved profile. My article on permission receipts shows the wider pattern: authority should be visible, scoped, and tied to a specific run.
record why the route changed
A failover event needs enough evidence for an operator to reconstruct the decision. “Primary unavailable” is too vague.
Keep the trigger, candidate evaluation, data classes, policy versions, and outcome:
fallback_receipt:
run_id: support-1842
triggered_at: 2026-07-27T10:51:12Z
trigger: primary_timeout_after_8s
primary_route: provider-a/eu-production-1
candidate_route: provider-b/eu-production-2
run_policy: support-agent-12
candidate_policy: provider-b-eu-7
data_classes_checked:
- customer_confidential
- payment_metadata
decision: denied
reason:
- data_class_not_approved
- retention_mismatch
context_sent: false
approval_override: none
Do not allow an override to become a reusable escape hatch. If an exceptional route is necessary, bind approval to one run, one candidate, a precise data scope, and a short expiry. Record who approved it and why. An operator should not be able to click “allow fallback” and widen every later request.
A useful dashboard should separate availability failure from security denial. Otherwise an operations team may see falling completion rates and loosen the router without understanding that the gate is preventing an unapproved transfer.
test the failure path before production does
A model-routing eval should force the router through mismatched candidates. Test at least these cases:
Primary EU route times out; fallback is US only.
Fallback accepts the region but not the data class.
Fallback has correct data terms but broader write tools.
Provider policy changed after the route was approved.
The run approval expired while the primary model was retrying.
A partial prompt was sent before the policy check completed.
The expected outcome is not always completion. The correct pass may be a clean stop with no context sent, no tool called, and a useful message for the operator.
Also test downgrade pressure. Give the router several unavailable approved routes and one fast unapproved route. It should continue to refuse the unapproved option. Reliability code should not rewrite the security policy because the queue is growing.
This connects directly to stop conditions as agent security controls. A production agent needs a known answer for “what happens when the safe route is unavailable?” If the answer is improvised during an outage, availability pressure will usually win.
design recovery around the original promise
A good fallback preserves the original promise to the user and the organization. It does not merely produce an answer by another route.
Before enabling model failover, write down:
Which data classes can enter the run?
Which providers and regions may process them?
What retention terms must hold?
Which tools may each route influence?
What policy version approved the route?
When does that approval expire?
What evidence proves that denied context was never sent?
Who handles the task when every safe route is unavailable?
If those questions have no owner, the system does not yet have a fallback design. It has a list of alternative endpoints.
where the books fit
I wrote Securing Enterprise AI Agents for teams responsible for this control model. The book covers bounded authority, identity, MCP and RAG governance, policy enforcement, evidence, revocation, and incident response. Get the book on LeanPub.
If you also own the engineering loop around coding agents, Claude Code: Building Production Agents That Actually Scale covers evals, observability, review packets, rollback, cost controls, and release gates. The Enterprise AI Agents in Production bundle joins the reliability and security sides without treating either as an afterthought.
A fallback is useful only when it recovers the run without abandoning the rules that made the original route acceptable.