Most agent programs we review in 2026 look alike. Four to twenty pilots, one or two in production in a narrow lane such as ticket triage or code review, and nearly all of them able to propose an action: send an email, open a pull request, issue a refund, move funds, or call an internal API with elevated scope.
Almost none have a system that decides whether a proposed action should happen.
That gap is not a capability problem. The missing piece is a layer between “the agent wants to do this” and “this happens”. In most organizations it either does not exist or it exists as a prompt asking a model whether an action looks acceptable, which survives a demo and fails in production for structural reasons.
Governance sits in front of every agent action by definition, so its cost and latency multiply by the fleet’s traffic rather than by one workload’s. A governance layer that calls a frontier model on every policy evaluation becomes the slowest and most expensive dependency you own, and therefore the first thing carved out when a release slips or a bill lands. Teams building on the Shakudo Platform hit that wall in the same quarter as their first production agent.
We have watched the sequence often enough to recognize it from the outside. A prompt-based guardrail gets added, becomes the highest-traffic dependency in the critical path, and then cost and latency force a choice between weakening it and moving it out of the path. Those are the same option.
The rest of this post is an architectural argument: the decision layer is the missing control plane for agent governance, and unlike most 2026 AI categories, you can own it.
A decision model does not generate text. You send it a state plus a set of typed questions, and it returns typed answers with probabilities in a single forward pass. There is no decoding loop, no internal monologue, and no output token budget.
Jev, from TypeSafe AI, popularized the category, and its three question types work as a vocabulary for the whole class:
choice, which selects one of up to 255 declared options and returns a distribution over all of them.score, which places a value on a rubric such as 1 to 5, with a distribution rather than a point estimate.noul, which answers a single yes/no question with one probability.The set of possible outputs is exactly the declared option set. The distinction that matters is worth stating plainly: a schema is a request, constrained decoding is a fairly strong request, and a closed output head is a guarantee. With a closed output head, a malformed answer is not unlikely. It is unrepresentable.
That moves the failure mode instead of removing it. The model can still choose the wrong option. It cannot invent a field, return prose where a label was required, or carry prompt text into a downstream system. For anyone writing policy, the distance between “usually well formed” and “structurally cannot be malformed” is the distance between a control you monitor and a control you rely on. The Pydantic AI integration makes that concrete: TypeSafeModel subclasses an abstract DecisionModel, so a typed decision becomes one more callable in an application graph.
The work landing in this category shares one property: the set of acceptable answers is known in advance, and it is small. Decision models are the wrong instrument for open-ended synthesis and the right one for every judgement where you can write the answer key before you see the input.
| Workload | The question it answers | Answer type | Where it shows up |
|---|---|---|---|
| Classification | Which known category does this request belong to? | choice | Intake triage, ticket tagging, document type detection |
| Routing | Which handler, queue, or model should serve this? | choice | Model routing, queue selection, tool selection |
| Scoring | How does this state rate against a rubric? | score | Risk tiering, quality gates, lead qualification |
| Ranking | In what order should these candidates go? | score per candidate | Retrieval reranking, work prioritisation |
| Verification | Does this output or claim satisfy the rule? | noul | Output validation, claim checks |
| Policy evaluation | Should this action proceed, wait, or stop? | noul plus a policy mapping | Allow, block, or escalate an agent action |
We see classification and verification appear first, because they are cheap to author and hard to argue with. Routing and policy evaluation arrive when an agent gets connected to a system that can change state. Ranking shows up last, when retrieval returns too many plausible candidates and a scoring threshold is the only way to keep the context window honest.
The argument is arithmetic before it is philosophical. Governance evaluation runs in front of every action, so per-call cost and per-call latency multiply across the fleet. Put a frontier model there and enforcement becomes the most expensive and slowest part of the system, with the worst possible traffic profile.
A subtler problem is that the evaluator and the agent share a substrate. Research on LLM judges found that roughly 96% of judge verdicts repeated the decision model’s confident errors, at 16 to 325 times the cost and 28 to 350 times the latency, so agreement between a generative evaluator and the system it evaluates is weak evidence of correctness.
An LLM in the enforcement path is also an attack surface. It reads untrusted content, including what the agent is acting on, so it inherits prompt injection as a failure class. A closed output head removes format-level manipulation; it does not remove semantic manipulation, because a crafted input can still push a classification to the wrong option. That is why the escalation design below carries as much weight as the model choice.
Generative models remain genuinely useful for governance. They draft policy from an existing rulebook, summarize incidents, and explain a decision after the fact for a risk committee. They are poor choices for the synchronous decision itself, which has to return a stable answer at a price you can pay on every action. It is also why we route the enforcement leg through a governed inference path such as the Shakudo AI Gateway, where the model call, its access controls, and its audit trail are one artifact instead of three.
Jev is proprietary, from TypeSafe AI, a San Francisco company founded in 2024. It entered limited early access on 15 September 2026 alongside a $40M seed led by DCVC, and the current version is jev-1.13.0. The endpoint is POST https://api.typesafe.ai/v1/systemone, with GET /v1/models for discovery. TypeSafe quotes $0.042 per million input tokens with output free, and a 64k request budget of which roughly 32k is available for the state plus the longest question. Output tokens are billed at $0 because the model does not generate prose, though the API still reports output_tokens in its usage block. TypeSafe calls it the first “System One model”, borrowing the fast intuitive mode from Kahneman’s Thinking, Fast and Slow, and it is listed on the Vercel AI Gateway as typesafe-ai/jev. No published weights, no published parameter count, no self-hosting option.
The marketing is where a skeptical reader slows down.
Our read is that the category is real and the headline multiples are not.
Three tiers do the work, and keeping them distinct is the whole design.
In practice the policy tier returns one of three things: allow and execute, pause for human approval, or block. The decision tier never holds the rulebook, and the policy tier never asks a model what the rule should be. That separation is what lets you change a threshold without retraining anything, and what makes the audit record comprehensible to someone who was not in the room when the policy was written.
Policy as code is the emerging pattern here. Toolkits such as Microsoft’s agent governance tooling implement hash-chained approval records, so each decision commits to the previous one and tampering is detectable after the fact. Worth stating plainly for anyone writing a compliance narrative: there is no ratified standard yet for the format of a governance receipt, so you are choosing a data model that regulators may later specify.
A decision model that returns a distribution is only useful if you route on it. The confidence score feeds a threshold, and the threshold decides whether the system acts or asks a person.
Calibration is where this gets uncomfortable. A model that reports 93% confidence on a coin flip routes almost nothing to a human if the threshold sits at 0.9, which converts an oversight mechanism into a formality. The audit above carries the deeper warning: the same model on the same items produced expected calibration error of 0.091 or 0.020 depending only on whether the question was phrased as a yes/no or as a two-option choice. If phrasing moves calibration more than training does, then question authoring is the control, and it deserves review like code.
EU AI Act Article 14 requires high-risk systems to be effectively overseen by natural persons, and oversight that depends on a 70 to 500 millisecond model call per action, a vendor-published range that independent measurements place at roughly 90 to 500 ms, does not scale to the volume it was written to cover. Regulation (EU) 2026/1744 moved the stand-alone Annex III deadline to 2 December 2027, while Article 50 transparency duties applied from 2 August 2026. The EU AI Act reference site and the Commission’s regulatory framework pages are where to track those dates, and the NIST AI Risk Management Framework remains the most usable vocabulary for the oversight argument you have to make internally.
Four deployment shapes are on the table, and they differ less in accuracy than in who owns the question set and the record.
| Option | Unit cost | Data residency | Operating burden | Best fit |
|---|---|---|---|---|
| Closed hosted API | Lowest, usage based | Leaves your perimeter | None | Teams without residency constraints |
| Vendor-hosted open model | Higher, usage based | Leaves your perimeter | Low | Teams that need licence flexibility |
| Self-hosted open model | Infrastructure and staff | Full control | High | Regulated, air-gapped, sovereign environments |
| Open method on an existing self-hosted LLM | Marginal, reuses the current estate | Wherever that LLM runs | Moderate | Teams with a large self-hosted estate |
At small scale the self-hosting case is not about unit cost, because infrastructure and staffing dominate. It is about owning the question set and the audit record. When a regulator asks why an action proceeded, “the vendor’s hosted endpoint said 0.94” is a weaker answer than a versioned question artifact, a reproducible evaluation, and a hash-chained receipt in your own storage. This is also one of the few 2026 AI categories with real open alternatives.
| Project | Licence | Base | Mechanism | Notes |
|---|---|---|---|---|
| AWS Strands Decider 2B | Apache-2.0 | Qwen3.5-2B | Language model head replaced with a pointer head that scores typed options in a single forward pass | Our pick as the cleanest reference implementation of the pattern |
| Cloudflare Clef and Clef-flash | Apache-2.0 weights | Qwen3.8-27B and Qwen3.5-9B | Post-trained decision models, hosted on Workers AI | Open weights is not the same as OSI open source, and on the hosted Workers AI path the price is several times the closed alternative |
| Convai Laya | Apache-2.0 | ModernBERT-large bidirectional encoder with a decision head | Not a decoder at all | Roughly 35 ms, and architecturally the most distinct option |
| Nokia AnyJev | Apache-2.0 | Any open LLM | Restricts the next-token distribution to the declared option tokens, with no gradient steps and no parameter changes | Also publishes a self-distilled Tacit family (1.7B to 9B) |
| Independent starter reimplementation | MIT | n/a | Same input and output shape as Jev | Shows the pattern is reproducible outside the original vendor |
Two cautions before you pick from that table. There is a naming hazard: dozens of unrelated projects share the flagship model’s name, one renamed itself away from OpenJev within days, and a repo named autotrust/JEV now publishes an open-weights student of Jev. Every card we checked disclaims correctly, which is the point: the collision is loud enough that the disclaimers had to be written at all. None of this is an OSI open-source release either, and open weights is a weaker claim than open source. And “open” is doing a lot of work in vendor copy, so check the licence, check whether weights are actually published, and price the hosted path if that is how you intend to consume it. The Hugging Face blog and OpenRouter’s model directory are useful on the packaging question.
For teams that want the architecture without operating a model, this is where a governed platform earns its place, and it is why we ship the decision layer as a component of Kaji rather than as a standalone endpoint. Customers under residency constraints usually reach that conclusion with or without a deadline forcing it, and the longer argument, written for a review board, is in the whitepaper on decision models and the enterprise decision layer.
Vendor benchmarks are directional evidence about a vendor’s test set. They are not a procurement input. Here is what we ask for instead.
The one number we insist on before a pilot starts is the escalation rate on representative traffic, measured with a threshold set in advance. Everything else in that list is a filter for vendors, not a measurement of your system.
A decision layer is infrastructure, and it fails the way infrastructure fails: quietly.
The operating routine that keeps those four from compounding is short enough to actually run:
The metric that matters most is not accuracy. It is the escalation rate, because it is the only number that tells you simultaneously whether the decision model is competent for the traffic it is seeing and whether you are saving the money you built the layer for. A decision layer that escalates 40% of traffic is a slow approval queue wearing a model as a hat.
What is a decision model? A model that returns typed answers instead of text. You send it a state plus a set of typed questions, and it returns the answers with probabilities in a single forward pass. The declared options are the only possible outputs, so an answer arrives in a shape your systems can act on without parsing.
What is the difference between a decision model and an LLM? An LLM generates a token sequence and you extract an answer from it. A decision model does not generate text at all; its output head is closed to your declared options. The difference that matters operationally is the failure mode. An LLM can be right and unparseable. A decision model can be wrong, but it cannot be unparseable.
Is Jev really 193x faster than an LLM? Treat the headline as vendor marketing. An independent analysis measured roughly 1.7x for a single decision, reaching about 100x only when a six-step workflow collapses into one call. The multiple is a property of the comparison rather than of the model. Independent testers measured speedups from roughly 2x to 25x on real workloads, while the 20x to 200x band comes from vendor material.
Why does AI agent governance need a decision layer? Because governance evaluation runs in front of every agent action, so its latency and cost multiply across the entire fleet. If the enforcement decision needs a frontier model call, you have built the slowest and most expensive component in the stack, and it will be the first thing weakened when pressure arrives.
Can we self-host a decision model in our own infrastructure? Yes. Apache-2.0 options include AWS Strands Decider 2B, Cloudflare Clef and Clef-flash, Convai Laya, and Nokia’s AnyJev method, which converts an existing open LLM into a decision model with no gradient training and now also publishes a Tacit model family. Check the word “open” carefully: open weights are not the same as OSI open source, and a hosted open path can cost several times a closed API.
What is the escalation rate and why does it matter more than accuracy? The escalation rate is the share of decisions routed to a human. It is the only single number that tells you both whether the decision model is competent for the traffic it is seeing and whether you are saving the money you built the layer for. Accuracy against a static benchmark tells you neither.
If you are running agent pilots today and cannot describe, in one paragraph, how an action gets blocked, that is the gap worth closing first. Our team works through this architecture with regulated enterprises regularly, and you can start that conversation here.
