Bits and Chunks of Evidence with AI

Digital forensic examiners are turning to locally run large language models (LLMs) for the data privacy and chain-of-custody guarantees that cloud AI cannot offer. However, data collected from even a single compromised host can easily exceed the small context windows and other parameters of locally run AI. To facilitate forensic analysis using AI, examiners must rely on a strategy to divide evidence into pieces that the model can process and reassemble.

This study addresses that gap through a controlled comparison using a known, examiner-authored intrusion that represents a common attack type. Forensic evidence was collected using standard triage tools and analyzed by a local LLM model under two strategies: one that carries findings forward through sequential chunks of evidence, and a second that summarizes each chunk independently before synthesizing a final report. The strategy that carries the findings through sequential chunks of evidence identified a mean of 1.0 attack artifact out of 5.

In contrast, the second strategy, which summarizes each chunk independently before synthesizing a report, identified all five attack artifacts. Neither strategy delivered a complete, accurate final report of the intrusion. Both failures are constrained by what the model could write rather than what it could read. The results demonstrate that a model's ability to recognize an intrusion is not the same as a tool's ability to deliver that recognition to the examiner and that the two can diverge completely.