Go Jump in a Lake: How a Data Lake Can Help Your Security Program

Back in the olden days, security was done using a SIEM. You plunked down a large sum of money, poured all of your logs (or at least the ones you could afford) into this thing that patiently collected them, then searched it either manually or automatically for threats.

Over time, this concept evolved into EDR and XDR which provided a more targeted effort to identify threats—but the need to store raw logs never really went away. Instead, our applications grew more and more complex, involving on-prem servers, cloud servers, serverless functions, containers, container orchestrators, and complicated networking to wire these things together. As this mountain of data (which is useful for finding threats) grew, so did the challenge of storing it all within a SIEM.

In recent years, the concept of using Data Lakes to hold all of this data has started to become more prominent; but what the heck is a data lake? How is this different than a SIEM? Why is it that this “magical panacea” suddenly appeared as an option to the alternatives?

This talk intends to demystify the concept of a data lake—explaining what it is, how it works, and even how to build one! More importantly, however, it explains the usefulness of a data lake as a tool in your security operations tool belt and how it enables you to gain greater visibility across the new technologies that you’re adding into your system on a daily basis.