I have the good fortune this week of being able to attend Mandiant's Incident Response Conference (MIRcon) in Alexandria, Virginia, and so far it's a very good time. For those who couldn't attend, or who may have chosen instead to attend that other conference that's going on right now, I thought I'd blog a few impressions and take aways both to solidify the day in my own mind and provide some food (and flavor) for thought. This won't be a comprehensive, presentation-by-presentation summary, but rather an overview with focus on what I consider to be some of the highlights. And if you weren't at MIRcon today, the single most important highlight you missed was Richard Bejtlich simultaneously coining a new phrase and inventing a new psychological diagnosis: "Incident Intrusion Fatigue Syndrome." So, if you want to find out whether you or your team suffer from this debilitating illness, read on.
The day started strong with a humorous keynote presentation from Kevin Mandia, followed immediately by an excellent panel discussion titled The IR Dream Team, featuring Richard Bejtlich, Ron Davis and Curtis Rose. (You can see the full agenda and presenter details over on the MIRcon Agenda page.) I found this to be the strongest and most insightful part of the day. The discussion kicked off with the panel members being asked about the two biggest challenges Incident Response Teams face. Richard Bejtlich and Ron Davis responded, and between the two, arrived at some of the biggest challenges faced by most responders and response teams. In the order they were presented, these challenges are:
- Visibility (Bejtlich) - the ability to see what is happening across our network and endpoints, in order to detect that a compromise has occurred. For many security/IR teams, it's challenging enough to gain visibility into even the basic building blocks of a network - namely, the network devices, servers and endpoints. But here, Richard noted that devices like Blackberries and iPhones further complicate this issue. How, indeed, do you detect a compromise on the growing number of mobile devices that appear and disappear on our networks?
- Authority (Bejtlich) - Richard identified the challenge of authority as the second biggest challenge for IR teams, then summarized the problem by saying that if you don't have the support of your organization, you can't do anything. As I understand this, it is essentially the challenge of "executive buy-in." As I see it, authority is one aspect of that. Part of supporting an IR team is giving them the authority necessary to execute their job. When this doesn't happen, the IR team can end up dependent on (at the mercy of?) IT operations and other stake holders and find themselves begging and borrowing all of the data they need to do the job.
- Preparation (Davis) - While Ron Davis called this the "preparation" challenge, his real focus was on process, with the challenge being defining and refining response processes for all of the ways we can be attacked. I would suggest the real challenge here is maturation. In other words, creating and following repeatable processes is a mark of a mature IR team and a mature organization. The challenge, then, lies in reaching that level of maturity.
- Coordination (Davis) - The challenge here is in coordinating response activities across distances and time zones. This is particularly challenging for organizations with global presences and operations, as they face increased levels of complexity, but it's important to remember that coordination is not a challenge exclusive to global organizations. Any organization with multiple offices and/or satellite locations experiences some version of this challenge.
The Dream Team panelists were also asked, among other things, to name their best source(s) of threat intelligence. The unanimous consensus here is that the best threat intelligence comes from the incidents you are already working or have worked. In other words, your best threat intelligence comes from responding to and analyzing the attacks you're actually seeing. While I'm sure this is true for organizations who have an adequately mature Incident Response capability, those teams/responders that aren't adequately staffed and funded are likely to miss this kind of intelligence due to a lack of ability to respond to and collect data on all of the incidents that cross their wire. Of course, you can begin to collect threat intelligence from wherever you are on the maturity scale simply by collecting data and beginning to correlate the data you do have. But it would have been nice to hear the panelists more fully address other sources of threat intelligence that might be more useful/actionable for organizations that are still building up their own response capabilities. (Unfortunately, I've only now thought of this, so didn't think to raise the question myself during the panel Q&A.)
Finally, the Dream Team panel was asked how we know we've "won." That is, how do we know we've been successful as incident responders. While the clear consensus is that there is no real "winning" (the fight goes on), Richard Bejtlich did offer some interesting indicators of a successful IR team or organization:
- You've increased the amount of money per megabyte that the intruder must spend to exfiltrate data. That is, you've made the intruder's job more difficult and more expensive.
- You've developed threat intelligence to a level that you are able to predict the intruder's next move.
- You can not only predict the intruder's next move, but you can track the changes they make on compromised systems.
- You've achieved a level of "intrusion suppression," such that you are able to defend your network well enough to keep the number of intrusions to a manageable level, and thus avoid "Intrusion Fatigue Syndrome." It's fairly obvious, after all, that if you're trying to respond to an excessive number of incidents you run the risk of exhausting your team and either compromising their skills or loosing them as employees.
From this high point, the day moved into a couple of Mandiant-centric presentations that, nevertheless conveyed a number of broader ideas worth considering and trying to build on. When I say Mandiant-centric here, it is not derogatory. What I mean is that the presentations focussed on Mandiant's technologies and how they are used. MIR Integration and Automation, for example, introduced the idea of integrating Mandiant Intelligent Response (MIR) with Request Tracker (both offer a Perl REST interface that can be leveraged to make them interoperate) to automate the feeding of incident data and details into a ticketing system, which can then be leveraged to generate metrics and reports that are valuable to both the incident response team and the broader business. Unfortunately, the Mandiant customer that created the presentation had to remain anonymous, so the presentation was delivered by a Mandiant representative. While he presented well, he could not provide the level of detail that the original author might have been able to share. Nevertheless, the presentation transcended product specificity in the sense that this kind of interoperability is the kind of thing we can be looking for and trying to develop in any set of tools we use. In this case, the customer organization automatically creates incident tickets from things like SIEM alerts and MIR scans. This allows them to leverage the collection and reporting capabilities of the ticketing system to automate the production of a wide variety of metrics, including numbers and types of incidents. By doing this, they've eliminated significant effort and hours spent on manual data collection that such reporting often entails.
The weak point of the day, for me, was a panel discussion on intelligence sharing. This was, I think, partly a problem of expectation, as the discussion seemed a relatively minimal part of the presentation. For at least the first 30 minutes, this was more like a series of lightning talks as the panelists each gave brief presentations discussing their involvement in various intelligence sharing organizations and initiatives, such as DSIE (Defense Security Information Exchange) and FS-ISAC (Financial Services Information Sharing and Analysis Center). The one valuable nugget I took away from this came from Kevin Naver of Sandia National Labs, whose information and resource sharing model for the numerous National Laboratories also involves the sharing of personnel resources across the organizations, such that an expert from one site could be leveraged to assist with incident response (or any issue, really) at any of the other sites. This model would lend itself well to small businesses, universities, state and federal agencies at a minimum. I can see where more sensitive private corporations and agencies would have concerns with intelligence sharing and data leakage in a scenario like that, but in a world where true incident response and forensics experts are in short supply, this kind of personnel sharing makes a great deal of sense.
Finally, the presentation day concluded on a heavily technical note with Kelcey Tietjen (Los Alamos National Laboratory) demonstrating the value of Windows Crash Dump Analysis for incident response. The crash dump can provide a lot of useful and detailed information about the crashing process and other items in memory. Where such crashes are related to a compromise, analyzing the crash dump files using WinDbg can provide a wealth of information that can be used to create Indicators of Compromise and other intelligence. Unfortunately, slides of the presentations have not been made available (and I don't know whether they will be), and there was more information here than I was able to capture in my notes. In that regard, the presentation was delivered a bit too quickly, leaving the audience (speaking for myself, here) with some good ideas but unable to capture enough detail to avoid having to go look up the information ourselves.
Overall, however, the weaknesses were exceptionally minimal, and I'm looking forward to Day 2. MIRcon has already provided more value than some conferences and courses I've paid for. So I give props to Mandiant for providing so much to the community in valuable tools and information. And, of course, I would be terribly remiss if I did not also say thanks to Mandiant for the free food and free beer.
Obviously, this is one man's view, and I certainly couldn't capture everything. So if any attendees or presenters are reading this, please share your own impressions and take aways in the comment section.
Gregory Pendergast is the Interim Information Security Officer at Virginia Commonwealth University.
