The Last Mile of Threat Hunting: Using LLM Agents to Execute and Refine Hunts at Scale

Hypothesis-driven hunting is often presented as a process of generating hypotheses and translating them into hunt queries. In practice, the most difficult part begins after the query runs. A single hunt can return thousands or tens of thousands of events, leaving analysts to manually baseline activity, identify outliers, and determine which findings deserve investigation. This presentation shares lessons learned from building and deploying a production LLM-agent workflow focused on hunt execution and analysis. Rather than stopping at query generation, the system assists analysts throughout the investigation process by iteratively refining hunt results, building behavioral baselines, clustering similar activity, identifying anomalies, and surfacing the most promising leads for human review. The talk walks through a complete threat hunting case study, showing how an initial broad hunt query evolves through multiple stages of automated analysis. Attendees will see how specialized agents perform result triage, compare activity against environmental baselines, identify unusual patterns, generate investigative pivots, and summarize findings with supporting evidence. The workflow is demonstrated against real security telemetry and production hunting scenarios. The presentation also examines the challenges encountered in practice, including weak baselines, hallucinated conclusions, inconsistent results across model versions, and analyst trust. We discuss the safeguards implemented to ensure findings remain explainable, evidence-based, reproducible, and subject to analyst validation. Attendees will leave with a practical framework for using LLM agents to augment hunt execution, reduce manual triage effort, and help analysts focus on the most suspicious activity without sacrificing transparency or investigative rigor.