Information Security - Data Retention

Information Security - Data Retention

March 15th, 2008
By Evan Wheeler

Especially with the recent increase in e-discovery concerns, retention policies have become an essential proactive step in any organization’s information security preparedness. There are many laws, regulations and contracts that may include obligations to maintain information for a given period, and each have their own time periods and criteria. Examples include:

• The Basel II Accord - Affects international banks. Activity logs should be retained 3-7 years
• Federal Financial Institutions Examination Council (FFIEC) - Affects financial institutions governed by the Federal Reserve, FDIC, etc. Specifies historical retention.
• Gramm-Leach-Bliley Act (GLBA) - Affects entities that participate in financial institution activities.
• The Health Insurance Portability and Accountability Act (HIPAA) - Affects healthcare industry. Logs should be retained up to 6 years.
• North American Electric Reliability Council (NERC) - Affects electric power providers. Specifies log retention for 6 months and audit record retention for 3 years.
• National Industrial Security Program Operating Manual (NISPOM) - Specifies log retention of at least one year.
• The Sarbanes-Oxley Act (SOX) - Affects US Corporations. Specifies retaining audit logs for up to seven years.
• VISA Cardholder Information Security Program (CISP) - Specifies retaining audit logs for at least six months.[1]

In most cases, the type of business will define the external requirements for information retention, and these periods range greatly. Legal counsel and audit staff should always be included in the development process for any data retention policies to ensure the business is complying with all contracts, local laws, industry regulations, and national or international laws.

Beyond these external forces, information retention in its various forms also has intrinsic value. Regular logging and auditing of user and administrator activities can support later troubleshooting, incident response, and forensic efforts and is essential for establishing accountability. In many cases, security incidents and legal issues may not present themselves immediately so it is important to keep data records long enough to facilitate investigations several months after the fact, and to include procedures to preserve records during the course of an investigation. For example, it has become a recommended standard to either take a forensic image of an employee’s system upon termination or quarantine the system drive in case of future investigation. This could facilitate a wrongful termination suit or allegations of data theft by that employee after they have left the company and their system (minus the storage drive) has been repurposed to another user.

Some examples of key data sources that may be useful when investigating an incident include:

• Firewall logs including denied and permitted traffic
• Internet gateway logs including web proxies, web filters, and network devices
• Name and address assignment/resolution history (such as dynamic DNS or DHCP records)
• Web, application, and file server logs
• Email and Instant Messenger communications from server and client logs
• Authentication and authorization service logs
• Workstation audit logs (account activity, system events, etc.)
• Server operating system audit logs (in addition to application software logging)
• Intrusion Detection / Prevention System logs
• Malware detection and removal logs
• Network infrastructure or monitoring devices

Any data retention provisions cannot be successful without first having a clear information classification standard and performing discovery efforts to determine what is stored in the environment and where. More detailed recommendations for what data should be logged for each system and how the logging infrastructure should be configured can also be critical to maintaining useful data for an investigation. For instance, having proper time synchronization is important for event correlation between devices or across locations.[2] Although centralized logging is critical to maintaining log entry integrity, having some facility for local logging can also be crucial when connectivity to the external log server has been interrupted. You never want a situation in which the critical system simply doesn’t log any information because it can’t reach the central log server. Having a small local store for emergencies should be a default configuration setting.

The operational challenge for most organizations has shifted from not having enough data to information overload. With all the systems - firewalls, IDS systems, email gateways, web filtering devices, workstations, servers, etc. - generating a steady stream of log entries the typical staff can’t possibly look at them all. This creates a complex problem of what data should be retained, how available does it need to be, and how long should it be kept? Additional examples of data that should be covered by a data retention policy but aren’t directly related to incident response can be found in resources such as the NARA General Records Schedule 24[3] or the NIST Special Publication 800-86[4].

"Organizations should establish policy for how long evidence from an incident should be retained. Most organizations choose to retain all evidence for months or years after the incident ends." The following three factors should be considered when determining appropriate evidence retention policies:

• Prosecution - Any cases that may involve legal action, civil or criminal, require all evidence (original if possible) to be maintained for the entire duration of the legal activities.
• Data Types - The categories of data contained in evidence (such as emails or sensitive client information) may affect how long the data is kept but should not supersede the need to preserve evidence.
• Cost - The cost of storing related hardware or the loss of a single drive may not seem consequential, but retention decisions need to account for the overhead involved in maintaining evidence stores and protecting them over time versus destroying them at the conclusion of an investigation.[5]

In all cases, the confidentiality, integrity, availability, and accountability of the evidence should be preserved in a documented and defendable manner. Any sensitive data needs to be protected at every stage of its lifecycle in the organization including handling, storage, and archival. Data retention policies need to address these issues as well as acceptable tools for use in the destruction of data. A common pitfall for many organizations can be either be keeping data for too long or not destroying it completely.

Collection of any communications or user data can also pose legal issues. Some of the most common issues include:

• Disclosure of information with privacy or security implications, such as passwords or the contents of e-mails, in network captures or log entries
• The long-term storage of information might violate an organization’s data retention policy for that category of information
• The lack of policies regarding the monitoring of networks and the lack of consistent warning banners on systems that indicate that activity may be monitored
• Communications data collection occurs as part of regular operations versus ad hoc troubleshooting or incident handling
• The lack of policies that clearly explain what types of monitoring can and cannot be performed without approval, and that describe or reference the procedures that detail the request and approval process
• The need to preserve original logs when many log aggregation systems will automatically normalize and filter log data
• Many ISPs now require a court order before providing any information related to suspicious network activity that might have passed through their infrastructure which can slow down the investigative process[6]

In all cases, legal counsel should be consulted along the way if a specific case is not covered by policy or there is any doubt. For example, as soon as a subpoena has been served, the organization is required to retain any and all data related to that incident for the duration specified. The universal imperative for all organizations is that they must establish an electronic data retention policy[7] and be held accountable for observing that policy in a consistent manner.

====
1. http://isc.sans.org/diary.html?date=2005-03-22
2. http://www.ucl.ac.uk/cert/log_retention_guidelines.pdf
3. http://www.archives.gov/records-mgmt/ardor/grs24.html
4. http://www.csrc.nist.gov/publications/nistpubs/800-86/SP800-86.pdf
5. http://csrc.nist.gov/publications/nistpubs/800-61/sp800-61.pdf
6. http://www.csrc.nist.gov/publications/nistpubs/800-86/SP800-86.pdf
7. http://www.sans.org/reading_room/whitepapers/backup/514.php