An AI agent’s next command could find a vulnerability or cause an outage. Before it runs, a guardrail has to make a decision: allow or block.
We wanted to understand where decision models fit into that process.
Can they catch harmful actions quickly enough to check an agent’s next step?
We compared JEV with Grok 4.7 on 200 synthetic cases, repeated three times per model. That’s 1,200 checks covering data deletion, credential leaks, service disruption and actions outside the agreed scope.
We measured three things:
Accuracy: Grok made more accurate decisions on the checks it completed, although some requests timed out.
Speed: JEV’s median response was 324 milliseconds, compared with 16.44 seconds for Grok.
Cost: About $0.084 per 1,000 checks for JEV versus $3.79 for Grok, using published pricing and reported usage respectively. These are different calculations, not verified invoice totals.
The examples show why context matters. In one case, a routine status check would replace authentication keys and sign users out. Both models caught the risk when given the application’s behavior.
This is a research experiment with synthetic cases. It does not measure NoScope’s production guardrails.
At NoScope, we combine the strengths of different approaches to stop harmful actions as effectively and efficiently as possible. Research like this helps us evaluate what each approach can contribute.
Read Using JEV for Security Guardrails for the results, limitations and concrete examples:
noscope.com/blog/jev-agent-g…