Every time an agent does a piece of work, something decides which model does the thinking. Until this month, in the platform we build our agents on, that decision was made once, when the agent was set up. An invoice check and a month-end report ran on the same model. A one-line fix and a full refactor went to the same coding engine.
That setup is simple, and it costs money in a way nobody notices on any single run. It also hides a question every operations leader should be able to answer: why did this job run on that model?
We just changed it. Vocion, the open-source platform underneath every Metacto agent, now asks a small decision model before each run: which agent should take this, which model should run it, and how much effort it deserves. The decision model is called Jev, from TypeSafe. It answers in well under a second, for a fraction of a cent.
Why one model per agent stops working
Model choice is the biggest lever on what agents cost to run, and it is usually set once and forgotten. The largest models earn their price on work that needs judgment. On work like reading a field off a document or sorting a request into the right queue, they cost more and add nothing. Putting the cheapest model on everything is the opposite mistake, and it shows up as quality problems instead of invoices.
The right answer changes job by job, many times a day, which is exactly the kind of decision nobody wants to make by hand. So the useful question is what should make it, and what should limit it.
What Jev decides, and what it can’t
Each agent gets a short list of models it is allowed to use, and each model on the list has a plain description of the work it suits and its relative cost. Before a run, Jev reads the request and picks the cheapest model on that list that covers it, plus an effort level. If an agent has no list, Jev only sets the effort.
It never picks outside the list. It can only choose models and engines the workspace already allows, and budgets, approvals and trust rules still apply to whatever it picks.
Today it makes that choice in four places:
- Every agent turn and chat, including missions and automations, which until now always used the agent’s single model.
- The software factory, where it picks the coding engine, model and effort for each task, so a typo fix and a refactor no longer go to the same place.
- The first message in a workspace with several agents, where it picks which agent answers.
- New records, where it can check each one against a short list of likely duplicates with one yes or no per record.
When it isn’t sure, the default runs
This is the part we care about most. Jev returns a choice and how sure it is. If it is less than 60% sure, or it errors, or it times out, the run uses the model it would have used anyway, and the reason is written down.
Vocion never depends on it. Calls to Jev have short timeouts and no retries, a breaker trips after three failures in a row, and removing the key puts everything back the way it was.
Last week I wrote about a production loop that a platform limit stopped almost four hours before any of us knew. The same idea applies here. The rules that matter are the ones the system enforces whether or not anyone is watching: a list it cannot step outside, a confidence floor, a default that always works.
Every choice is on the record
There is a routing view for every workspace. It shows how many turns Jev routed and how many ran the configured model, every fallback with its reason, the first messages it assigned to an agent, and spend by model, split by how each model was chosen.
That turns “why did this cost what it cost” into a question with an answer for every turn. It also tells you whether routing is earning its place, from your own numbers.
One decision stays with you
To make a choice, Jev reads the first 2,000 characters of the request and the descriptions of the options. That makes TypeSafe a subprocessor for that content. So turning Jev on for a client’s workspace is a data-sharing decision as well as a cost decision, and we make it with the client, workspace by workspace. Without a key, Jev is never called.
What to ask about the agents you run
If agents are doing work in your business, or you are about to put them there, these five questions are worth asking whoever runs them:
- Which model runs each job, and who chose it? “Whatever the agent was set up with” works in a demo. In production, each job should have a reason.
- What is the list it chooses from? Routing without a list of allowed models is a budget surprise waiting to happen.
- What happens when the router isn’t sure? There should be a default that always works, and a record of every time it was used.
- Can you see spend by model, and why each model was chosen? If not, you cannot tell whether routing saves money or just moves it.
- What data leaves to make the choice, and where does it go? Every new service in the path is a data decision.
Picking the right model for each job is a cost question. Being able to show why each job got its model is a control question. You need both before agents run work that matters.