Architecture pattern · independent

Decision models in agent workflows

An agent loop makes many small decisions for every piece of text it writes: which tool, which model, whether a step succeeded, whether a human should approve. Using a frontier LLM for each one multiplies latency and cost, and every answer has to be parsed out of prose.

A decision model handles that hot path. It returns a typed value with a probability, so the agent runtime branches in code. This page covers where the model sits and what it does not replace.

The hot path problem

A single user request can trigger a dozen routing decisions before any answer is written. If each is a chat completion, the cost is a dozen generation round trips plus a parser and a retry path for each. If each is a typed decision call, it is one evaluation over a fixed answer space, often with several questions sharing the same state.

Cloudflare and AWS both frame the category this way: small, fast models handle routing and gating; large models handle generation; humans handle high-impact uncertainty. That split is an architectural choice, not a claim that decision models are smarter.

Where the model sits

  • →Before a tool call: approve, deny, or request confirmation based on the action and its arguments.
  • →Before a model call: choose the tier — a cheap small model or a frontier one — from the request shape.
  • →After a step: check whether the result satisfies a completion criterion instead of trusting an agent to say "done".
  • →On memory and context: decide what to keep, drop, or compact.
  • →On output: grade against a rubric, detect hallucination risk, or route to human review.

Agent frameworks people are using in 2026

Public coverage describes Nous Research's Hermes Agent as an open-source autonomous agent with persistent memory and self-generated skills; AWS Strands as an agent SDK and harness with an intervention system; OpenCode as an open-source coding agent; and LangGraph as a graph-based orchestration framework. They are runtimes and orchestration layers, not decision models.

A decision model can sit in front of any of them as a gate, in the same way a policy check does. No first-party integration between Jev and these frameworks is documented in the sources this directory tracks; the pattern below is conceptual and framework-agnostic.

A minimal gate

  • →Set the threshold from measured behaviour on your own labels, not from a vendor example.
  • →Log the state, the answer, the probability, and the threshold together for audit.
  • →Keep a human approval path for destructive actions even when confidence is high.
gate.json
{
  "state": "Tool: delete_records · arguments: { "scope": "all" }",
  "questions": {
    "safe_to_execute": {
      "type": "noul",
      "instructions": "Is this action within the approved policy?"
    }
  }
}

// confidence >= 0.8  → execute
// confidence <  0.8  → request human approval

Frequently asked questions

Do I need a decision model to build an agent?

+

No. Most agents route with plain code or an LLM instruction and work fine. A decision model earns its place when routing decisions are frequent, latency-sensitive, or must be logged and thresholded as values rather than parsed from text.

Is Hermes Agent a decision model?

+

No. Public coverage describes it as an open-source autonomous agent runtime with persistent memory and self-generated skills. It belongs to the orchestration layer. A decision model could act as a gate in front of its actions, but no first-party integration is documented here.

How should I handle low confidence?

+

Make it an explicit branch: escalate to a human, fall back to a slower generative model, or take a safe default. Do not treat a probability as a guarantee — calibration claims in this category are vendor-reported and should be measured on your own data.

Can this actually reduce LLM cost?

+

It can, by removing small classification calls from the generative path. Whether it does depends on your volume and on the price of the decision call. The cheapest gate is still a rule; a model has to beat it on cost and maintenance.

Where do I see concrete patterns?

+

The use cases section of this directory writes up routers, guards, triage, RAG verification, and CI triage as concrete arrangements of Choice, Score, and Noul.

Keep reading

Sources

Reviewed 2026-11-13. Vendor- and project-reported figures are labelled as such; none were independently reproduced by this site.

Architecture pattern · reviewed 2026-11-13 · first published 2026-11-13