Architecture pattern · independent
Decision models in agent workflows
An agent loop makes many small decisions for every piece of text it writes: which tool, which model, whether a step succeeded, whether a human should approve. Using a frontier LLM for each one multiplies latency and cost, and every answer has to be parsed out of prose.
A decision model handles that hot path. It returns a typed value with a probability, so the agent runtime branches in code. This page covers where the model sits and what it does not replace.
The hot path problem
A single user request can trigger a dozen routing decisions before any answer is written. If each is a chat completion, the cost is a dozen generation round trips plus a parser and a retry path for each. If each is a typed decision call, it is one evaluation over a fixed answer space, often with several questions sharing the same state.
Cloudflare and AWS both frame the category this way: small, fast models handle routing and gating; large models handle generation; humans handle high-impact uncertainty. That split is an architectural choice, not a claim that decision models are smarter.
Where the model sits
- →Before a tool call: approve, deny, or request confirmation based on the action and its arguments.
- →Before a model call: choose the tier — a cheap small model or a frontier one — from the request shape.
- →After a step: check whether the result satisfies a completion criterion instead of trusting an agent to say "done".
- →On memory and context: decide what to keep, drop, or compact.
- →On output: grade against a rubric, detect hallucination risk, or route to human review.
Agent frameworks people are using in 2026
Public coverage describes Nous Research's Hermes Agent as an open-source autonomous agent with persistent memory and self-generated skills; AWS Strands as an agent SDK and harness with an intervention system; OpenCode as an open-source coding agent; and LangGraph as a graph-based orchestration framework. They are runtimes and orchestration layers, not decision models.
A decision model can sit in front of any of them as a gate, in the same way a policy check does. No first-party integration between Jev and these frameworks is documented in the sources this directory tracks; the pattern below is conceptual and framework-agnostic.
A minimal gate
- →Set the threshold from measured behaviour on your own labels, not from a vendor example.
- →Log the state, the answer, the probability, and the threshold together for audit.
- →Keep a human approval path for destructive actions even when confidence is high.
{
"state": "Tool: delete_records · arguments: { "scope": "all" }",
"questions": {
"safe_to_execute": {
"type": "noul",
"instructions": "Is this action within the approved policy?"
}
}
}
// confidence >= 0.8 → execute
// confidence < 0.8 → request human approvalFrequently asked questions
Do I need a decision model to build an agent?
+
No. Most agents route with plain code or an LLM instruction and work fine. A decision model earns its place when routing decisions are frequent, latency-sensitive, or must be logged and thresholded as values rather than parsed from text.
Is Hermes Agent a decision model?
+
No. Public coverage describes it as an open-source autonomous agent runtime with persistent memory and self-generated skills. It belongs to the orchestration layer. A decision model could act as a gate in front of its actions, but no first-party integration is documented here.
How should I handle low confidence?
+
Make it an explicit branch: escalate to a human, fall back to a slower generative model, or take a safe default. Do not treat a probability as a guarantee — calibration claims in this category are vendor-reported and should be measured on your own data.
Can this actually reduce LLM cost?
+
It can, by removing small classification calls from the generative path. Whether it does depends on your volume and on the price of the decision call. The cheapest gate is still a rule; a model has to beat it on cost and maintenance.
Where do I see concrete patterns?
+
The use cases section of this directory writes up routers, guards, triage, RAG verification, and CI triage as concrete arrangements of Choice, Score, and Noul.
Keep reading
Composition patterns
Routers, guards, triage, and verification written up concretely.
LangGraph + decision models
The same idea expressed as graph nodes and conditional edges.
Strands Decider 2B
A model built specifically for agent-loop interventions.
Decision models vs LLMs
Where each belongs in the same pipeline.
Sources
- Cloudflare — decision models for agentic workflows ↗
- AWS ML blog — Strands harness and Decider 2B ↗
- awesome-jev community catalog ↗
Reviewed 2026-11-13. Vendor- and project-reported figures are labelled as such; none were independently reproduced by this site.