Prompt Injection Defense: When to Allow, Block or Defer

A prompt injection defense can allow an input, block it or hold it for further review. Whether the third option helps depends on the second reviewer’s errors, too.

An agent is asked to summarize a customer email. The message also tells it to send internal documents to an unfamiliar address. A scanner returns a suspicious score. What should the application do next?

It can block the content or release it for processing. It can also ask another reviewer to decide. While that happens, the affected processing needs to wait. Otherwise, the agent may act on the injected instruction before the review finishes.

The appeal is straightforward: a small model handles clear cases, while a more expensive reviewer handles the rest. A paper submitted on October 14, 2026, From Detection to Decision: Calibration and Deferral for Prompt-Injection Detection, examines this arrangement. Its authors, Qianlong Lan, Anuj Kaul, Vinothini Pandurangan and Indranil Sanyal, show why the reviewer's mistakes matter as much as the initial score.

At a glance

• A model score needs calibration evidence on relevant data before it is treated as a probability.
• In the new study, more than 91.8% of inputs reach a reviewer before missed attacks fall to at most 1%. False alarms remain substantial.
• The finding describes the tested pipelines. Wolf Defender Jev was not evaluated.

Sources and product references checked October 16, 2026.

Three decisions before processing continues

A prompt injection uses external content to give the agent an unauthorized task. The customer email can supply facts for a summary. It cannot authorize the release of internal documents.

An application can respond in three ways:

The consequences of an error should shape the policy. Summarizing a paragraph requires different permissions from sending emails or deleting files. OWASP recommends layered defenses, restricted privileges and human approval for high-risk actions.

A score of 0.9 needs evidence behind it

A detector can rank attacks above benign inputs while producing poorly calibrated scores. Calibration asks whether predicted probabilities match observed frequencies.

For an illustrative example, a large group of comparable cases assigned a 90% attack probability should contain roughly that proportion of attacks. That relationship needs validation on appropriate data. A number between zero and one does not establish it.

Guo and colleagues studied post-hoc calibration of neural networks, including temperature scaling. This method adjusts score confidence while preserving ranking. If the model initially ranks an attack below a benign input, scaling will not reverse their order.

That matters when the application sends only uncertain cases to review. A confidently missed attack may bypass the second stage entirely.

What the new study tested

The authors compare TypeSafe's Jev, Llama Prompt Guard 2 with 22M and 86M parameters, and a GPT-5.5-based LLM judge. They query Jev through its official API with a fixed question and also test Prompt Guard after calibration on separate data. Prompts, calibration and decision policies are fixed before the final evaluation. Study, Sections 3 and 4

The confirmatory test contains 47,611 examples from TensorTrust, BIPIA, NotInject and PIDS-Bench v3: 41,981 attacks and 5,630 benign inputs. Attacks therefore account for approximately 88.2% of the mixture, with BIPIA supplying most of them. This deliberately constructed mixture does not represent normal production traffic.

For ranking attacks against benign inputs, the LLM judge achieves the highest AUROC at 0.897, followed by Jev at 0.871. Prompt Guard 2 scores 0.757 for 86M and 0.714 for 22M. However, Jev's native probabilities do not consistently outperform calibrated Prompt Guard scores on calibration metrics. Ranking and probability reliability measure different properties.

Where the 91.8% finding comes from

The second stage uses a fixed LLM reviewer. The authors examine curves that send progressively more cases to it. The observed system false-negative rate first reaches at most 1% at these points:

Source: Table 4 of the study. The percentage columns use different denominators: all inputs, actual attacks and actual benign inputs.

In this setup, almost every input needs a second review. Large fractions of legitimate inputs are still blocked. The authors report these crossings as descriptive results from the test curves. They are not deployment thresholds validated on independent data. Differences below one percentage point between the gates also do not establish a reliable ranking without uncertainty intervals.

The reviewer shapes the result

The LLM reviewer's operating point prioritizes missing as few attacks as possible. Its fixed threshold is approximately 0.0527. On the separate reviewer-calibration split, this setting produces a 98.6% false-positive rate. The complete systems have lower false-positive rates on the final test, as shown above. These measurements use different datasets and describe different parts of the pipeline. Study, Sections 3.2 and 4.3

As more inputs reach this strict reviewer, its behavior increasingly determines the overall result. The high escalation requirement describes a particular combination of gate, routing rule, reviewer and data mixture.

For an application, the useful questions are how many attacks the entire pipeline stops, how much legitimate work it interrupts and how long inputs wait. A low missed-attack rate alone leaves most of that unanswered. The study also evaluates detection and routing rather than a complete agent workflow with measured downstream harm.

Evaluating the pipeline in your own workflow

Start with inputs from the channels your agent actually consumes: emails, documents, web pages and tool results. Include legitimate security material that quotes attacks. Otherwise, a detector can look effective while repeatedly blocking ordinary tasks.

Select thresholds and reviewer-routing rules on separate development data. Then evaluate the complete frozen pipeline on held-out cases. Record missed attacks, false alarms, escalation rate, waiting time and reviewer errors. Break results down by source to expose weaknesses hidden by an aggregate. Our Merlin evaluation illustrates another version of this problem: a high benchmark score did not survive changes to tool names.

Define what happens when review fails. A timeout is not a benign finding. For the internal-document transfer in the opening example, the action needs to stay on hold until valid authorization exists. Independently restrict recipients, data access and tool permissions, regardless of the content score.

Where Wolf Defender and the Scanner API fit

Wolf Defender is freely available for detecting prompt injection in individual text inputs. The free Patronus Scanner API offers another way to inspect content. Either can be a component in an integration whose processing and release rules you evaluate together.

The study did not evaluate Wolf Defender or Wolf Defender Jev. Its Jev model comes from TypeSafe. Our Wolf Defender Jev is a different model with its own evaluation conditions. The numbers reported here do not transfer to those models. Detecting injections in individual inputs also does not establish protection across separate chats.

Sources

1. Lan, Q., Kaul, A., Pandurangan, V. and Sanyal, I. (2026): From Detection to Decision: Calibration and Deferral for Prompt-Injection Detection, version 1, October 14, 2026. The table above reports the authors' measurements; we did not rerun their experiments.
2. Guo, C., Pleiss, G., Sun, Y. and Weinberger, K. Q. (2017): On Calibration of Modern Neural Networks, ICML 2017.
3. OWASP: LLM Prompt Injection Prevention Cheat Sheet, checked October 16, 2026.

If you spot a factual error, please let us know through our contact page.