Joseph Attia

The Bouncer Before the Detective

One intrusion plotted against when an AI agent actually answered about it. The attacker is gone by the 52-second mark, data already stolen. The AI agent's verdict — threat confirmed — doesn't arrive until the 243-second mark, nearly three minutes later. ~3 minutes of silence Intrusion begins t = 0s Attacker gone t = 52s Verdict arrives THREAT CONFIRMED t = 243s seconds since the intrusion began → illustrative timeline
The data was already gone when the verdict arrived. Everything inside the shaded stretch is silence.

Ask an AI agent whether a log line is a real threat, and it will tell you. Eventually.

By the time it answers, the archive it should have flagged is already gone.

Why real-time threat detection breaks agentic AI

Handing a log line to a large language model isn't like a bouncer glancing at an ID. It's like calling in a detective for every single person who walks up to the door.

The detective is thorough. That's the problem. Reasoning about whether a log line is suspicious means pulling recent history for that user and host (context assembly), forming a hypothesis, often calling out to a threat-intelligence or reputation service (a tool call, a real network round trip), revising that hypothesis with what came back, and writing up a verdict in prose. Each step is seconds. Together, they're the better part of a minute.

One AI agent pass over one log line, broken into three parts: gathering evidence, reasoning, and writing the report, totalling about sixty seconds. ~60 seconds for one verdict Gather evidence 20s Reason about it 27s Write the report 13s meanwhile, logs can arrive thousands of times a second
Two of the three blocks are gathering and weighing evidence. None of it is optional.

None of that is wasted effort. A real investigation costs exactly that much.

The failure is applying it to every line. Logs don't arrive one at a time with a polite pause between them. A moderately busy service can emit thousands a second, and a sixty-second investigation run one at a time falls behind at the very first burst. The verdict in the first figure isn't slow because reasoning about that one event took four minutes. It's slow because roughly four ordinary sixty-second passes were already queued in front of it. Once an agent falls behind, it stays behind, and every later verdict inherits the wait of everything ahead of it in line.

What security teams already do with a flood of alerts

A hospital doesn't send every arrival straight to a specialist. A triage nurse takes one look — vitals, presenting complaint, maybe thirty seconds — and decides who waits and who doesn't.

Security operations centers run the same shape. Tier-one analysts, or increasingly correlation rules, close the overwhelming majority of alerts in seconds: a failed login here, a port scan there, ordinary noise from an ordinary network. What survives that first pass goes to someone who investigates properly. Only a small fraction of that goes further still, to whoever owns the incident.

The edge is what happens when the first pass is missing or badly calibrated. Analysts facing thousands of low-quality alerts a day stop reading them closely — the industry calls it alert fatigue — and a real intrusion gets the same half-second glance as the noise around it. Read faster was never the fix. Triage before you read was.

A bouncer at the front of the log pipeline

A bouncer doesn't investigate anyone. A glance at an ID, a read of the room, a decision in under a second: in, or step aside. Almost everyone gets waved through.

That's the shape to put in front of the logging system, literally. Not a smaller LLM — a small model of a completely different kind: a compact classifier, the sort that fits comfortably on a CPU, trained on structured features pulled straight out of each event. Event type. Authentication-failure rate in the last five minutes. Entropy of the command line, since obfuscated commands look statistically different from typed ones. Whether it's off-hours for this account. How critical the asset is. Reputation of the source network.

onLogLine(line => {
  const features = extract(line)         // ~15 structured fields
  const score = model.predict(features)  // under 2ms, no LLM involved
  ingest(line, score)

  if (score >= THRESHOLD) {
    const bundle = withIdentity(line, features)
    agentQueue.push(bundle)              // the only lines the agent sees
  }
})

Every event gets scored. Almost none of them cross the threshold. The ones that do get bundled with whose account and which asset they touched, then handed off. That handoff is the entire point of building the small model in the first place.

Ten bars of event volume by suspicion level, tallest to shortest, with a movable line separating events sent to the AI agent from events handled automatically. Every log event, scored automatically low risk → high risk Handled automatically: 1,749,000/day (93.1%) Sent to the AI agent: 129,000/day (6.9%)
Drag the slider. The line moves, and the two numbers below trade against each other.

What the small model can't do is write a paragraph explaining why any of this matters. It doesn't need to. Its only job is: does this deserve a closer look. That's a narrower question than "is this a threat," and narrower questions are exactly what small, cheap, fast models are good at.

How much faster is a small model than an LLM agent?

Tens of thousands of times faster, in the plainest sense: milliseconds against the better part of a minute.

A small trained classifier answers in about one millisecond. An AI agent's full investigative pass takes about sixty seconds — roughly sixty thousand times longer. ~60,000× slower Small model ~1 millisecond (plain rule-based detection is about this fast too) AI agent ~60 seconds
The small model isn't a faster language model. It's a peer of detection that never used one.

That gap isn't sloppy engineering on the agent's part. Reasoning across evidence and calling out to enrichment services costs what it costs, whichever vendor writes the agent. So the fix was never a faster LLM. Asking it less often was.

What happens after the bouncer says yes

A bouncer waves someone inside and that's the end of their involvement. A detective takes over from there: establishes who this person is, what they were doing, whether it connects to anything else, and writes it up so someone else can act on it.

An escalated event gets identity attached — which account, which asset, which team owns it — and, before it ever reaches the AI agent, it's grouped with other escalations from the same account and asset into one session. That grouping does most of the remaining work: the figure below shows roughly fifty-six thousand flagged events a day narrowing to about a thousand the agent actually investigates. Deduplication and prioritization are why that gap doesn't require fifty-six thousand sixty-second agent passes.

What the agent does with that bundle is the part a small classifier structurally can't: pull the broader story. Cross-reference other systems. Decide whether this fits a known technique. Draft a report a human can act on in one read, with a recommended next step. Not "risk score 87." "This account, from this network, escalated privileges seven seconds after a credential-stuffing burst succeeded; isolate the host and force a credential reset."

A funnel from all log events, through flagging by the small model and investigation by the AI agent, down to confirmed real incidents — narrowing across five orders of magnitude. All log events ~2,190,000/day Flagged for review ~56,000/day Investigated by the AI agent ~1,150/day Confirmed real incidents ~26/day illustrative volumes for a mid-size environment
Fewer than thirty events a day ever reach a human. Everything above that is the machine doing its job.

Where the two-tier design breaks

Every threshold is a bet. Set it too low and the AI agent drowns in the same flood it was supposed to be spared: recurring, harmless noise the small model hasn't yet learned to wave through. Set it too high and something real slips under it, silent, forever.

There's a sharper failure than miscalibration, though: an attacker who knows a threshold exists can stay under it on purpose.

Six login attempts from one account, spread across a day, each individually under the escalation threshold — only visible as one campaign when correlated by account. escalation threshold morning night one loud event caught instantly Six quiet events. Same account. One attack. none of the six crossed the threshold alone
Trace the dashed line. Six ordinary-looking moments, hours apart, from one account.

Six failed logins on the same account, spread across nine hours instead of nine seconds, each one scored well under threshold on its own. A per-event classifier has no memory of the last event, let alone the last nine hours of them. Correlating across time is exactly the kind of thing the LLM agent's context-gathering step was already built for. Which is why the escalation threshold is a filter, not the whole defense — and why periodic session-level audits of near-threshold accounts matter even when nothing individually escalates.

There's a quieter risk underneath both of these: the small model's own training labels usually come from the LLM agent's past verdicts. That's efficient, and it's also a feedback loop. Whatever pattern the agent has been consistently wrong about becomes a pattern the small model learns to wave through with confidence. Retraining on the agent's judgment without ever auditing that judgment is how an error compounds instead of getting caught.

Common questions about AI-driven threat triage

Does a small model replace the LLM agent entirely?

No. It replaces running the LLM agent on everything. The small model decides what deserves attention; the agent decides why it matters and what to do about it. Cutting the agent out entirely gets you a fast system that can't explain itself or correlate across systems, which is most of what an analyst actually needs from it.

Does this replace a SIEM or existing correlation rules?

No. The small model runs at roughly the same speed as plain rule-based detection — noted right on the latency figure above — and typically sits alongside it rather than instead of it. Rules are precise about patterns someone thought to write down; the small model adds statistical pattern recognition for the ones nobody wrote a rule for yet.

What happens if the small model misses a real threat?

It's a false negative, and in security that's the expensive direction to be wrong in. That's the entire reason the low-and-slow example above matters: some misses are threshold-tuning errors, fixable by moving the line, and some are structural blind spots — like campaigns spread across time — that no single threshold fixes, which is why correlation and periodic audits exist as a second layer.

How often does the small model need retraining?

Often enough to track a moving target. Attacker behavior shifts, the environment's normal baseline shifts, and — per the feedback-loop point above — the agent's own past judgments shift what "correct" looks like. Treat the model as a component with a maintenance schedule, not a one-time export.

Go back to the first figure. The gap between "attacker gone" and "threat confirmed" isn't a flaw in how the agent reasons. It's what happens when every event, urgent or not, waits in the same line.

Put a fast, narrow model at the door and that line disappears for almost everyone who never needed to be in it.

The bouncer doesn't investigate anything. It just decides, in about the time it takes to blink, who the detective needs to meet.

← All posts