Confidently Wrong: Feedback Loops of False Positives in AI-Augmented Threat Hunting

Modern SOC and threat hunting workflows are increasingly augmented by large language models for alert triage and hypothesis generation, yet these systems remain unreliable in practice. They frequently produce confident but incorrect classifications, including overestimating routine activity as active compromise, mislabeling benign scheduled operations as command-and-control behavior, and incorrectly mapping post-exploitation or vulnerability exploitation stages. For example, legitimate enterprise behavior such as backup jobs running at 2 AM may be incorrectly interpreted as beaconing or data exfiltration patterns. These hallucinations increase analyst workload, inflate false positives, and introduce operational risk by obscuring real threats. This presentation introduces a self-correcting AI threat hunting pipeline designed to address these limitations through structured feedback, institutional context modeling, and persistent learning. A key extension of this work is the concept of *institutional benchmark engineering*, where organizations define their own environment-specific ground truth. This allows security teams to encode what “normal behavior” looks like in their own infrastructure, for example, scheduled backups, patch deployments, or batch ETL jobs so that these are not misclassified as malicious activity. The system is evaluated on a custom benchmark of 30 realistic security scenarios derived from publicly documented attack patterns and detection use cases.(CISA KEV and NVD CVE) Each scenario includes structured ground truth for severity, recommended response action, and MITRE-aligned attack categorization. A baseline evaluation using Gemini-2.0-Flash achieved 33.3% accuracy, revealing significant weaknesses across severity classification, response action selection, and attack categorization. The system then applies a persistent learning loop that captures failures, stores them as structured JSON memory, and generates targeted course corrections that evolve the prompt across iterations. Following a single learning cycle, performance improved to 96.7% accuracy, reducing failures from 20 scenarios to a single remaining error. The loop injected 43 targeted corrections addressing systematic issues such as DDoS over-classification, infiltration mislabeling, and incorrect remediation actions. The session walks through the full architecture, including benchmark design, institutional baseline modeling, failure-memory construction, automated correction generation, and iterative prompt evolution. It also demonstrates how organizations can continuously refine detection intelligence by embedding site-specific operational context directly into evaluation benchmarks, thereby reducing hallucinations without model retraining. Attendees will receive: •⁠  ⁠Full open-source Python implementation of the persistent learning loop •⁠  ⁠Framework for building institutional SOC benchmarks tailored to enterprise environments •⁠  ⁠Benchmark generation pipeline with KEV-derived scenarios •⁠  ⁠Failure-memory and correction injection patterns for self-improving SOC workflows •⁠  ⁠Prompt engineering strategies for deploying self-correcting AI in security operations This talk provides practical methods for detection engineers and threat hunters to systematically evaluate, contextualize, and improve the reliability of AI-driven security decisions before operational deployment.