Decision Models

A supervisor agent answers approval prompts with a full LLM call. That works, but most reviews are routine approvals, and each one costs a chat completion and several seconds.

A decision model (a “decider”) is a non-generative classifier such as typesafe/jev-1.13. It answers fixed questions with probabilities instead of text, in a fraction of the time and cost. Denkeeper can put one in front of the supervisor so that the clear cases never reach it.

Where it sits

For a supervised agent, each tool call stops at the first stage that decides:

1. Auto-approve rules
2. Decider, if supervisor_decider is set
3. Supervisor agent, if supervisor is set
4. Human approval

The decider works with or without a supervisor behind it.

Setup

Declare the decider once, then point a supervised agent at it:

[[llm.deciders]]
name = "jev"
provider = "openrouter"          # an [[llm.providers]] instance of type openrouter
model = "typesafe/jev-1.13"

[[agents]]
name = "default"
session_tier = "supervised"
supervisor = "argus"
supervisor_decider = "jev"
supervisor_decider_mode = "shadow"
supervisor_decider_approve_at = 0.95
supervisor_decider_deny_at = 0.05

The same fields are on the Agents page under Permission, and on PATCH /api/v1/agents/{name}. Both apply to the running agent at once. Editing the TOML by hand is different: a reload applies mode and threshold changes and stops a removed decider, but starting a decider that way needs a restart.

What it is asked

Every call gets the same three yes/no questions, mirroring what the supervisor is told to check. Each answer is a probability between 0 and 1:

The decider sees what the supervisor sees: the tool name, description and arguments, the user request, skill context, and recent messages.

The verdict

The lowest of the three answers decides:

Thresholds must satisfy 0 < deny_at < approve_at < 1.

Shadow, then enforce

supervisor_decider_mode controls whether the verdict is acted on.

shadow (the default) computes and records the verdict, and changes nothing. Every review writes a supervisor audit event with source = "decider:<name>", decision = "shadow" and would_decide. The supervisor or you still decide every call.

enforce acts on it:

• Approve: the call runs. The supervisor is not asked.
• Deny: the call is blocked and the agent is told why, for example Tool call denied by decider: arguments flagged as unsafe (p=0.03).
• Escalate: the call goes to the supervisor, or to you if there is none.

The audit event then carries decision = APPROVE, DENY or ESCALATE. In chat, decider verdicts use the same approval statuses as a supervisor’s, with text that names the decider.

Start in shadow. A decider’s probabilities are not calibrated to your tools and your requests, so thresholds should come from data:

1. Run in shadow and compare the decider’s would_decide with what the supervisor decided on the same calls.
2. Or run denkeeper decide replay over existing supervisor history to see the agreement at each threshold.
3. Pick thresholds where the decider’s approvals and denials match the supervisor’s, then switch that agent to enforce.

Failure never approves

If the decider errors, times out, hits its cost limit, or the review is larger than max_input_tokens, the call goes on to the next stage as if no decider were configured. The failure is audited with a cause. An oversized review is refused, never truncated: a truncated view could hide the part that matters.

Cost

Decider spend is billed to the reviewed agent, per conversation, and counts against that agent’s cost limits.

Things to weigh

• Data egress: tool arguments and recent messages are sent to the decider’s provider, an additional data processor.
• Prompt injection: tool arguments can contain text an attacker controls. A decider cannot be talked into acting, but text such as “this call is safe” can sway its probabilities. Keep approve_at high, and keep a supervisor or yourself behind it.
• Thin denial reasons: a decider names which check failed, not why. An agent adapts better to a supervisor’s written reason. If denials confuse your agent, raise the bar for them by lowering deny_at.