Sub-cent AI security inference: Why only 25% needs the deep model path

As agents, tool calls, and context windows grow, so does the number of security decisions and the volume of data that must be inspected. AI security is becoming an inference problem: what matters is not only how fast a model runs, but how often the most expensive model path is needed at all.

Patronus Ark answers that question with a cascade. Inexpensive structural and semantic stages make early decisions. A chunk reaches deep L3 inference only when uncertainty, risk, or conflicting evidence remains.

About 25% deep inference instead of 100%

At the validated operating point, roughly 25% of chunks are promoted to L3. The cascade was evaluated against the full L3 path using more than 150,000 training examples, around 50,000 validation examples, independent holdouts, and out-of-distribution benchmarks. Its F1 and false-positive behavior remain approximately at the full-L3 level.

The efficiency gain does not come from inspecting less traffic. Deep inference runs selectively where it is expected to change the decision.

CPU routing, GPU acceleration

In the current benchmark, the optimized L2 path routes about 963,000 input tokens per second at peak on the CPU setup. At a roughly 25% promotion rate, that produces around 241,000 L3 tokens per second.

A full NVIDIA L4 runs Wolf Defender Small with TensorRT at around 387,000 L3 tokens per second using batch 16. One L4 can therefore absorb the promoted workload from the measured CPU peak with headroom.

The architectural consequence is the important part: GPU demand scales with the difficult share of traffic, not with the entire protected volume. CPU and GPU capacity can be expanded independently where the actual bottleneck appears.

What that means economically

Patronus Pro is planned at €69 per month with roughly 4,000 tokens per second of continuous throughput. At 24/7 utilization, that is about 10.368 billion protected tokens in a 30-day month, or approximately €0.0067 per million tokens.

This is a customer-price calculation, not a claim about infrastructure costs. The report therefore states the benchmark setup, pricing assumptions, and comparison data explicitly.

Download Patronus Insights, Issue 3

The issue includes the complete CPU, L4, and TensorRT measurements, the scaling model, and all methodological caveats and sources.