Auto, needs approval, manual: how we let an AI decide
We use Jev, a decision model, to classify incoming work and to choose between acting automatically, asking a person first, or leaving the item alone. Here is the setup, the code that holds the rules, and what we would do again.
Most AI features that fail in production do not fail because the model wrote a bad sentence. They fail because nobody decided, in advance, what the software is allowed to do with the model’s opinion.
On recent builds we have used Jev, a decision model from TypeSafe AI, for exactly that part: classifying what comes in and deciding whether the system acts on its own, asks a person first, or stays out of the way. This is how we set it up and what we learned.
A decision model is not a chat model
A chat model answers “should we refund this order?” with a paragraph. Your code then has to turn that paragraph back into a yes or a no, and hope the wording does not change next week.
Jev works differently. You send it a state (the thing being judged: a message, a record, a document, an image) and a set of typed questions about that state. It returns values your code can use directly, with probabilities, and it generates no text at all. The question types are:
- Choice: pick one label from a list you define. Returns the label, a probability for every label, and a confidence.
- Score: place the state on an ordered rubric, such as minor, moderate, severe.
- Noul: how true is this statement, as a number from 0 to 1.
- Box: where something is in an image.
Each question is evaluated on its own against the same state, so asking six questions does not make the fifth one worse. TypeSafe’s own advice, which matched our experience, is to keep questions atomic: break a complicated judgment into small questions and combine the answers in your code, where you control the weighting.
A request looks like this. The content is illustrative.
{
"model": "jev-latest",
"state": "Hi, I was charged twice for my March invoice. Please refund one of them.",
"questions": {
"category": {
"type": "choice",
"instructions": "What is this message about?",
"criteria": {
"billing": "Charges, invoices, refunds, payment methods",
"technical": "Something in the product is not working",
"account": "Login, access, profile or plan changes",
"other": "Anything that does not fit the labels above"
}
},
"urgency": {
"type": "score",
"instructions": "How urgent is this for the customer?",
"criteria": ["Can wait", "Should be handled today", "Blocking the customer now"]
},
"asks_for_money": {
"type": "noul",
"instructions": "The customer is asking for money to be returned or credited"
}
}
}
The answer comes back as a category with probabilities, a score, and a number between 0 and 1. Nothing to parse.
Three modes, and who chooses between them
Classification is the easy half. The harder half is what happens next, and that is where the three modes come in.
- Auto. The system classifies the item and carries out the action. Nobody is asked.
- Needs approval. The system prepares the action and shows it to a person with the model’s answer and its confidence. One click approves, edits or rejects.
- Manual. The system only labels the item. A person does the work. The model’s answer is a suggestion on screen, nothing more.
The important design choice: the model never picks the mode. Our code does. The model answers narrow questions. A plain, readable policy turns those answers into a mode.
That policy has three inputs.
- A ceiling set by people. Every kind of action has a highest mode it is allowed to reach, chosen by the business. Tagging a message can be Auto. Sending a reply might be capped at Needs approval. Moving money is Manual until someone with authority changes it.
- How sure the model is. Under a confidence threshold, an action drops one mode down.
- Risk flags. Separate noul questions ask about things that should always slow the system down: the customer is upset, the item mentions legal action, the amount is unusual. Any flag above its limit forces a person into the loop.
In code it is short on purpose:
type Mode = "auto" | "approval" | "manual";
function decideMode(action: ActionConfig, answer: Answers): Mode {
// People set the ceiling. The model can only lower it.
if (action.ceiling === "manual") return "manual";
// No answer, or a failed call, is never a reason to act.
if (!answer.ok) return "approval";
const flagged = action.riskFlags.some(
(flag) => answer.nouls[flag.id] >= flag.limit,
);
const unsure = answer.category.confidence < action.minConfidence;
if (flagged || unsure) return "approval";
return action.ceiling;
}
Anyone on the team can read that function and say what the system will do. That is the point. If the same rules lived inside a prompt, nobody could.
What we would do again
Start everything in Manual. For the first weeks the model only labels and people work as before. Every time a person agrees or disagrees with a label, we get a checked example for free. By the time anyone asks “can this be automatic?”, there is data to answer with.
Promote one action at a time. Modes are set per action, not for the whole product. The first thing to reach Auto should be something cheap to undo, such as routing or tagging. Actions that cannot be undone stay behind approval far longer, whatever the accuracy looks like.
Write criteria like instructions for a new colleague. Vague labels give vague probabilities. Each label gets a concrete description, the labels do not overlap, and there is always an “other” so the model is not forced into a wrong answer.
Do not trust confidence until you have checked it on your own data. A confidence of 0.9 is only useful if items scored 0.9 are right about nine times in ten for your task. Independent tests of Jev report that calibration differs from one task and one question form to another, so we measure it on our own checked examples and set each threshold from that, per action. A threshold copied from a blog post, including this one, is a guess.
A failed call means approval. Timeouts, missing answers and anything unexpected fall back to a person. The system should never act because the check did not run.
The model is not the lock. A decision model judges content. It does not enforce permissions. Who may approve, which records a user can touch and what the system can reach are enforced in ordinary application code, exactly as they would be with no AI involved.
Keep auditing Auto. Once an action is automatic, nobody looks at it, which is the moment mistakes become invisible. We send a random sample of automatic decisions back to a person every week and track the error rate. If it rises, the action drops a mode with one setting.
Log the whole decision. State, questions, answers, probabilities, the mode chosen and what the person did. When someone asks why the system did something three months ago, the answer is a row in a table.
Where this does not fit
A decision model does not write. If the task needs a drafted reply or an explanation in words, that is still a job for a language model, and the two work well together: one writes the draft, the other decides whether the draft may be sent without review.
It also does not remove the need for judgment about stakes. Where a wrong decision harms someone and cannot be reversed, the right mode is approval or manual, and a good confidence number does not change that.
The short version
Let the model answer small, typed questions. Keep the rules about what happens next in code that people can read. Give every action a ceiling that a person chose, start in Manual, and earn each step toward Auto with checked examples.
The model got better at classifying than we expected. What made the feature safe to ship was everything around it.
If you are planning something similar, this is the kind of work we do in AI and automation.
Working on something in this territory?
If any of this sounds like the problem in front of you, tell us where you are and where you'd like to be. We'll come back with a plan within two business days.