Advancing SIEM Log Management Strategies through Vendor-Agnostic Measurement

The Security Information and Event Management (SIEM) system hit the market two decades ago, yet the initial promise of the tool remains largely unrealized for many organizations. Measuring the operational effectiveness of the SIEM continues to be a challenge due to the complexity of deployment, configuration of the numerous components, and the process of determining the logs the Incident Response team needs to perform a comprehensive security investigation. While security engineers may be able to deploy the necessary components according to the manual, they often lack the understanding of which logs provide valuable insight to answer the questions necessary to identify, contain, and eradicate malicious activity. Meanwhile, vendor recommendations often suggest sending all logs to the SIEM while simultaneously charging based on log ingestion rate. This creates a situation whereby vendors propose solutions to increase their bottom line budget versus the best interest of the customer. This paper explores a novel approach to quantifying the value of an individual log source sent to the SIEM. Through vendor-agnostic measurement, the algorithmic model utilized by the Log Quality Value (LQV) index enables security engineers and incident response teams to determine which logs provide the most value for security investigations. Two common attack patterns were assessed against the proof-of-concept tool, and a positive correlation was found between the LQV index and the critical logs used to investigate the attack. Future opportunities exist to evaluate the LQV algorithms against a more extensive dataset from live production environments and measure the tool effectiveness through periodically comparing the LQV index to logs used to detect security incidents. This research ultimately proposes a call-to-action for the security community to build more vendor-agnostic methods to independently measure the effectiveness of security products.