Focused vs Dump-Everything Ingestion for GCP and Kubernetes SIEM Pipelines

The Focused vs Dump-Everything Ingestion for GCP and Kubernetes SIEM Pipelines research, published by SANS Institute in March 2026, measured whether a narrow, allowlisted log export strategy can match a broad "dump-everything" strategy on investigative coverage while cutting SIEM ingestion volume and cost. The research drew on a controlled nine-day A/B lab experiment comparing two concurrent Google Cloud Platform (GCP) and Google Kubernetes Engine (GKE) logging pipelines exported into separate Splunk indexes, covering cloud control-plane audit activity, Kubernetes API server audit activity, and 77 simulated attacker test procedures.

Key findings:

• A focused, allowlisted export strategy reduced exported log volume from 383.53 GiB to 18.7 GiB over nine days, a 20.5x reduction, or 95.1% less data exported
• Indexed events in Splunk dropped from 134,765,353 to 14,640,637 under the focused strategy, a 9.2x reduction, or 89.1% fewer indexed events
• Despite the 95% volume cut, the focused pipeline still detected 47 of 48 executed GCP control-plane attack scenarios, matching the dump-everything pipeline on all but one
• The focused pipeline detected all 29 of 29 executed GKE and Kubernetes control-plane attack scenarios, identical coverage to the dump-everything pipeline
• The only missed scenario in the focused pipeline was Cloud SQL database local user creation, which had been excluded from the allowlist
• Under a list-price cost model, the focused strategy cut the estimated monthly cloud logging bill from $971.36 to $151.28, an 84.4% reduction
• Cloud Logging storage costs alone fell from $614.22 to $6.17 per month between the dump and focused strategies
• Pub/Sub throughput costs dropped from $99.53 to $4.49 per month when moving from dump-everything to focused ingestion
• Kubernetes workload stdout/stderr logs, not control-plane audit logs, were identified as the telemetry class most likely to dominate SIEM ingestion cost without a proportionate security benefit
• Control-plane audit sources (Cloud Audit Logs and Kubernetes API server audit logs) supplied the attribution, action, target, and outcome evidence needed for the large majority of executed attack scenarios

The research finds that ingestion volume and investigative coverage are not tightly coupled in cloud and Kubernetes environments: a well-defined evidence contract built around control-plane audit logs can preserve nearly all detection and investigation value while removing the vast majority of ingestion cost. The one coverage gap identified, Cloud SQL user creation, shows that focused allowlists are only as good as the scenario catalog behind them and require ongoing governance rather than a one-time filter. The experiment was conducted in a dedicated GCP/GKE lab project using parallel Pub/Sub to Dataflow to Splunk HTTP Event Collector pipelines with identical downstream configuration, varying only the upstream sink filter. Coverage was scored against a 48-scenario GCP control-plane catalog and a 29-scenario GKE/Kubernetes control-plane catalog, based on whether authoritative, attributable audit evidence appeared in each Splunk index.