Elyra
Blog
Elyra
Elyra · · 8 min read

Elyra Desktop 0.4.3: a decision model beside the coding model

0.4.3 adds Jev, a small calibrated decision model that answers the agent's yes/no questions: which model tier a task needs, whether a shell command is destructive, and whether a finished run actually did what was asked.

Elyra Desktop 0.4.3: a decision model beside the coding model

A coding agent asks itself a lot of small questions that have nothing to do with writing code. Is this shell command dangerous? How hard is this instruction — does it need the big model or will the cheap one do? Did I actually finish, or did I just say I did?

Until now Elyra answered those the way everything else gets answered: either with a rule — a regex, a keyword list — or by asking the coding model, which is like asking a surgeon whether the parking meter has expired. It'll answer. It'll take a second or two and cost a few cents, and it'll phrase the answer as a paragraph you then have to parse, and sometimes it'll be confidently wrong, because writing prose is what it's good at and calibrated judgement is not.

0.4.3 gives Elyra a second mind for exactly those questions. It's called Jev, it's made by TypeSafe, and it is deliberately not a chat model.

What a decision model is

You give Jev a piece of content and one or more typed questions, and it gives back probabilities. That's the entire interface.

decide({
  state: "rm -rf ./build && npm run build",
  questions: [
    { name: "destructive", type: "noul",
      instructions: "Would running this destroy data or make a change that is hard to reverse?" }
  ]
})
→ { destructive: 0.08 }

decide({
  state: "the checkout flow drops every third order when the cart has a discount code",
  questions: [
    { name: "tier", type: "choice",
      instructions: "How much model does this instruction need?",
      criteria: { fast: "lookup, rename, one-line answer",
                  balanced: "ordinary feature or fix",
                  powerful: "cross-cutting, subtle, or the cause is unknown" } }
  ]
})
→ { tier: { fast: 0.03, balanced: 0.21, powerful: 0.76 } }

Three question types: noul (yes/no with a probability), choice (one option, plus the whole distribution), score (an ordinal scale). Several questions in one call cost the same as one. An answer takes about a second and costs about $0.00002 — two thousandths of a cent — which is the number that makes the rest of this release possible. You can ask Jev something on every single shell command and never notice the bill.

And it never writes prose. There's no paragraph to parse, no "I think this is probably…" to interpret. A probability maps directly to a decision, a threshold, a button. That's what makes it usable inside an agent's loop rather than as another conversation.

Elyra reaches it directly with a TypeSafe key (TYPESAFE_API_KEY), or through the OpenRouter key you may already have, preferring the first. The decide tool only appears when a key is configured; /jev shows what's active. No key, nothing changes — every feature below stays exactly as it was in 0.4.2.

Where it's used

Routing stops guessing from keywords

Smart routing picks a model tier per turn — fast, balanced, powerful — and until now the tier was decided by matching words. "Refactor" and "architect" bought the expensive model. Short, plain sentences bought the cheap one.

Which meant:

  • "Refactor the spelling of this one variable" → powerful. Twelve dollars an hour of model for a rename.

  • "The checkout flow drops every third order when the cart has a discount code" → fast. The cheapest model for a bug whose cause nobody knows.

Keyword rules score the vocabulary; the question is about the task. With a Jev key and smart routing on, the instruction is classified instead — one call, cached per instruction, about a second before the turn starts — and the reason line in the footer says what it decided and how sure:

/route
  tier: powerful → claude-opus-5 · high
  because: Jev 0.76 powerful — cause unknown, cross-cutting (cart + checkout + discounts)

The mechanical signals that read the turn rather than the message — a tool error, many edits in one turn, a huge result — still escalate the tier mid-run. Those are evidence, not opinion, and they don't need a model. If Jev can't be reached, the keyword rules take over for that run, and the reason line says so. jev.routing: false keeps the old behaviour.

A second opinion before a destructive command

Since 0.4.1, a short list of shell commands is confirmed rather than just run — rm -rf outside the project, sudo, git push --force, a download piped into a shell. That's a rule-based guard, and it's good at what rules are good at: the patterns it knows.

What it can't do is notice a command that's destructive in a way nobody wrote a rule for. find . -name '*.orig' -delete in the wrong directory. git checkout -- . when you had uncommitted work. A curl -X DELETE against something that turns out to be production. Rules see syntax; those need judgement.

/jev gate on adds it. Every shell command the model proposes is first sent to Jev with one question — would running this destroy data or make a change that's hard to reverse? — and anything at or above jev.bashGateThreshold (0.7 by default) goes through the same confirmation dialog the rule-based guard uses:

Jev thinks this command is destructive (0.91):
  git checkout -- src/
Because: "discards every uncommitted change under src/; the working tree
shows 14 modified files there"

[ Run it ]   [ No ]

Two things about how this is built, both deliberate.

It's off by default, because it adds about a second to every shell command, and for most people most of the time that second isn't worth it. Turn it on for the afternoon you're letting an agent loose on a repository you care about.

And it fails open. No key, an expired key, a network error — the failure is reported and the command runs. That's not carelessness; it's the consequence of a sentence we want to be very clear about: Jev's gate is a second opinion, not a security boundary. The rule-based guard is the boundary. It runs in code, consults no one, and works when the network is down. Jev sits beside it, catching what rules can't see, and if Jev isn't there you still have the rules. An opt-in feature that could block your commands when a third-party API hiccups would be a worse tool than one that says "couldn't check, running it" — and we'd rather you know which kind you have.

Did the run actually do what was asked?

This is the one we're most pleased with, and it's about a blind spot in how Elyra measures itself.

Since 0.9.30-something, every turn writes to a routing ledger: stop reason, failed edits, retries, failovers, latency, cost. /routing-stats reads it; the router ranks models by it; Elyra-Cost: trailers come from it. It's a genuinely useful record, and every number in it is mechanical. It knows whether the call errored. It has no idea whether the answer was any good.

So a model that cheerfully answers the wrong question — reads three files, writes a confident summary, touches nothing, and says "Done!" — scores perfectly. No errors, no retries, cheap. It would rise in the rankings. And when you compare two system prompts with /routing-stats --by prompt, you're comparing their error rates, which is a fine proxy right up until one prompt makes the model better at finishing and the other makes it better at sounding finished.

jev.turnLabels: true closes the gap. After each run ends, Jev is asked — from the instruction, the model's final message, and the list of tools it called with their targets — a small set of questions:

  • Outcome: completed / partial / blocked / failed?

  • Unbacked claim: does the final message say something works without the run containing anything that would show it — no test run, no check, no build?

/routing-stats --labels
model              turns  completed  partial  unbacked-claim  $/completed
claude-sonnet-5      212      84%       11%        3%           0.048
gpt-5-mini           198      61%       24%       19%           0.031
claude-opus-5         77      91%        7%        1%           0.190

Look at that third column. The cheap model claims things work, without evidence, one time in five. Its per-turn cost is lowest; its cost per completed turn is not, once you stop counting the turns that only said they were done. That's the number that was missing, and it's the one that should drive the router — which, from this release, it does.

It runs after the turn, so it costs no waiting. It's a few thousandths of a cent per run.

Where this one leaves the machine

Turn labels are off by default, and they're the one place in this release where we need to say something carefully.

The ledger has always stayed on disk — ids, counters, timings, never prompts or code. A turn label can't be made from counters. To judge whether a run did what was asked, Jev has to see what was asked and what the model said — so with jev.turnLabels on, each finished run sends TypeSafe the instruction, the final message, and the tool names with their targets (read src/lib.rs, bash cargo test). File contents are never sent. Diffs are never sent. Tool output is never sent.

That's a real boundary crossing and it's stated in the setting's own description, in the guide, and here. It's off until you say so. We think the number it produces is worth it for most teams; we think you should get to decide that with the facts in front of you.

Why a second model, and not a better first one

It would be tempting to say the coding model should just be better at these things. Ask it "is this dangerous?" and trust the answer. But asking a large generative model a yes/no question is expensive (a full turn), slow (seconds), poorly calibrated (it says "probably" with the same confidence at 55% and 95%), and returns prose you have to parse. Asking it after every shell command would double the cost of a session. Asking it to grade its own runs would be asking the student to mark the exam.

A decision model is the right shape for the job. Small, fast, cheap enough to run constantly, calibrated by design, and — importantly for the turn labels — not the model being judged. The coding model does the coding. Jev does the judging. Neither is asked to be the other.

Get it

Elyra Desktop 0.4.3 is at elyracode.com/elyra — the Desktop tab. macOS on Apple silicon, Linux on x86_64 and ARM64, free to use, every feature. Already installed? /update. The desktop guide has the section A decision model beside the coding model with every setting and exactly what each one sends.

Add a TypeSafe key. Run /jev. Then run /route on a bug report written in plain English and watch it pick the tier for the problem rather than for the words.

Back to blog Raw markdown