Jeremiah Grossman, Founder and CTO of WhiteHat Security
April 24th, 2009 By Stephen Northcutt
Update April 23, 2009
Jeremiah and many other software security experts such as Gunnar
Peterson are starting to focus on the need for comprehensive guidance.
There have been some good efforts, but they are not comprehensive.
Jeremiah says, "The problem CIOs and CSOs are facing is that the pseudo
Web security standards available are completely inadequate for
accomplishing the task." If you think about the evolution of security
as a process, you have to point to ISO 27001/2 as a significant
milestone, the point where we matured enough to establish a framework.
Jeremiah is personally committed to the concept and expecting to be
putting some of his own time and energy into it. You can be sure he
will post his thoughts on his blog; by the way, I hope you caught his
April 1, 2009 blogpost. http://www.threatpost.com/blogs/web-site-security-needs-a-strategy
founder and CTO of WhiteHat Security,
agreed to be interviewed for the security lab, and we certainly thank
him for his time.
Thank you, it's my pleasure.
What can you share about the web app security market segment, growing,
shrinking, becoming more sophisticated?
After about a decade, the Web application security market has finally
come into its own as businesses have embraced its importance. Several
distinct solutions have emerged to include vulnerability management
providers (like WhiteHat), developer tools (scanners), Web application
firewalls, and consultants to fill in the professional services gaps.
Each segment solves a particular business problem and successful
vendors are experiencing huge growth. Speaking for WhiteHat Security,
we've been doubling or tripling our business each year for the last
The market gets complicated for customers due to the tremendous volume
of information available and confusing industry terminology, which is
not always in strict agreement. Customers must navigate through
conflicting marketing messages and decide for themselves which
solutions they need when, and evaluate the relative capabilities of
each. It's challenging because there are a lot of lemon
on the market making many misleading claims. That's why for a security
vendor, having a good reputation is everything.
How would you describe the typical customer for WhiteHat
Most of our customers are in the financial services, e-commerce,
healthcare, or high-tech industries. Many of these are the largest and
most visited websites in the world, which have regular code updates,
and require a vulnerability assessment and management solution that can
keep pace with development and production releases. What all of our
customers also have in common is that they conduct a significant
portion of their business - sometimes all of it - through their
website, and view website security as vital. Most customers evaluated
various vulnerability assessment options or had experimented with Web
application vulnerability scanners prior to selecting the WhiteHat
Sentinel Service. For our typical customer, WhiteHat's
Software-as-a-Service (SaaS) model offers the best results and the best
Of course, many people associate you with "vulnerability
clapping" from your time at Yahoo. So one more time for the record, can
you explain what vulnerability clapping is and why it is so important
for large organizations to understand?
That was a bit of a misquotation that I'd like to clarify--it's
vulnerability "flapping" not "clapping". Vulnerability flapping, or a
"flapper" is when a vulnerability opens and closes from one
the next for some unknown reason. Typically, what's happening is that
the website is load balanced and one of the Web servers has a piece of
vulnerable code while the others do not. This causes a lot of
confusion, especially for the customer's security staff, because they
were under the impression the systems are "mirrored". This is a good
lesson for organizations because reviewing development code is not the
same as assessing production websites. Vulnerability flapping is one
example of a security issue that tends not to show up during
development or staging, but can have devastating effects on a
I read http://www.whitehatsec.com/home/resources/blog/whitehatblog.html#blog0625076 Reasons Why Reviewing Development Code is Not the Same as
Production Websites with interest. We are seeing some
held opinions on how to achieve a reasonable degree of assurance
with a reasonable degree of investment. Knowing resources are limited,
what guidance can you give organizations to achieve assurance without
breaking the bank?
Anytime hardcore source code review and black box vulnerability
assessment ideologists converse, spirited conversations take place. The
debates are not necessarily about the value of a particular solution,
but instead when they should be applied. For example, no one says that
source code reviews are unimportant to website security. The process is
helpful for spotting backdoors and various forms of unsafe coding
practices during development, but remain costly in terms of time and
money when websites change frequently. There are also many common and
potentially devastating types of website vulnerabilities that occur
outside the code. Conversely, I think black box vulnerability
assessments provide a better measure of overall website security, but
are unable to occur any earlier than the QA stage of the SDLC.
My advice for organizations is to find all their websites, rate their
value to the business, and assign a responsible party. This process
takes some time and patience, but is not at all expensive. Next,
perform comprehensive and ongoing vulnerability assessments (my job) on
all, if financially feasible, or at a minimum, the most important
websites to the business. All solutions in this segment will cost some
time and money, some having better ROI than others, but it's well worth
the investment. The knowledge provided is essential to quickly
mitigating high-risk issues and developing a long-term game plan for
website security without wasting resources unnecessarily.
are a couple of relevant blog postings that address this very issue.
Congratulations on your book, Cross Site
Scripting Attacks: Xss
Exploits and Defense. I think you have picked the
in information security right now. Can you tell us a bit about your
experience writing the book and since it was a team of authors, what
parts are pure Jeremiah Grossman?
Thank you. I think the timing turned out perfectly and judging from the
reviews, people are really enjoying the read. We're really excited!
Prior to committing to co-authoring XSS Attacks, I held off writing any
book (besides a couple forewards) for years and must have turned down
10 or more offers. Mostly because I knew how much work it was going to
be and I really didn't want to put my name to any book just to say that
I did. When this opportunity came up, there was nothing else like
it. The topic was white hot, but what pushed me over the edge
were the other authors (Seth Fogie, Robert "RSnake" Hansen, Anton
Rager, Petko D. Petkov) already onboard. It was an all-star cast for
the subject matter and I knew I'd regret not taking part. My content is
sprinkled in many areas, but my biggest contributions are in the
sections about the history of cross-site scripting, intranet hacking,
history stealing, defacement, worms & viruses, and the
sections. I'll also say, writing a book or even co-authoring one is a
still a huge amount of work!
Web security is getting to be a crowded market and the big boys are
starting to come in with the acquisition of SPI Dynamics, what sets
WhiteHat apart from your competition?
Actually I think the IBM and HP acquisitions of Watchfire and SPI
Dynamics respectively, largely emptied out the main competitors in the
vulnerability management market, specifically those offering
stand-alone security scanners. The two top products are predicted to
fold into larger development/QA product suites, rather than continuing
on as a "security play". The vulnerability management market
predominantly served by providers such as WhiteHat (SaaS), a myriad of
small to large consulting shops, and late-comer network scan vendors
such Qualys and ScanAlert (SaaS) who've recently started building out
Web application scanning technology. What sets WhiteHat Security apart
is that our customers have an easy time deploying and managing our
service, appreciate the quality of results, and experience a lower TCO.
Since one of the things that sets WhiteHat Security apart from the rest
of the companies is a strong professional services offering, can you
share a bit about the recruiting/hiring/training process? It is getting
better today, but a year ago, finding someone that knew web security
was rare. How do you do this?
It's interesting. Since the company's inception we set out to solve the
problem of high-volume, accurate website vulnerability assessment and
management. In the very early days, while we were developing
Sentinel 1.0, we accepted consulting engagements to pay the bills.
Today we are where we set out to be - a technology-driven SaaS company.
Our website vulnerability assessment and management service, WhiteHat
Sentinel, heavily leverages a proprietary scanning platform, which is
customer-controlled and expert managed. By that I mean WhiteHat's
security engineers perform the necessary configuration, customization
and vulnerability verification to complete the assessment process. It's
our technology that enables us to do what we do, while customers
experience a high quality and consistent vulnerability assessment
service that's easy for them to deploy and manage.
Of course no matter how advanced our technology, we still need good,
smart Web application security engineers on staff, but more importantly
we need to be able to grow our own. WhiteHat has the perfect
environment to do just that. For instance, if you look at the average
consultancy, their webappsec experts might perform assessments on 20-40
websites per year. Our operations team works very closely managing
assessments on more than 500 websites, usually conducted weekly. The
experience they gain is deep and exceptionally fast. Plus, any new
attack techniques and discoveries they develop along the way are placed
back into the scan platform making improvements rapid and easily
disseminated for the benefit of all WhiteHat Sentinel customers.
Thank you for that Jeremiah, the security
market continues to change
and new threats evolve. What are the hottest trends right now in
attacking web applications, and what can we do to prevent them?
In Web application security, we're dealing with many of the same
attacks in greater number that we've been familiar with for a
while: XSS, SQL Injection, CSRF, Authentication/Authorization
Bypass, Predictable Resource Location, Information Leakage, etc. What's
evolving is the impact of a successful attack. For instance, XSS used
to be all about cookie theft, and all but a few figured it was an
insignificant issue. Now we're dealing with malware payloads such as
Web worms, phishing with superbait, intranet hacking, trojans,
keystroke recording and history stealing. The combo attacks are making
things a lot worse.
This is a question I like to ask everyone in this space,
the unique things about web applications is that one programming error
can be referenced in hundreds of instances often all of them Internet
reachable. What do you think the number one error is; the mistake a
programmer can make to guarantee a spot in the hall of shame?
Does anyone give an answer other than "input validation"?
How about a new one: complacency. Most Web developers are well served
by a healthy dose of paranoia. In many ways they are the first and last
line of website defense. Web developers must be aware that not all
their users will be polite, play by the rules, and use the software the
way it was intended. When software is accessible to more than one
billion people, a certain percentage are going to abuse the system any
way they can. Without this touch of paranoia, complacency sets in and
that's when vulnerabilities happen. The job of a security professional
is to help developers understand what they need to be paranoid about
and arm them with the proper set of tools/information.
What advice do you have for someone in the security field to stay
current on web app security? And what is your favorite newsgroup,
mailing list or other information source?
Read and keep reading. Evaluate and reevaluate your assumptions. Here
are my favorite information sources.
You are a senior executive, you have a technical
you had a close friend, who was primarily technical, but was being
offered a senior level position such as a CTO in a mid sized company,
what is the primary piece of advice you would give him or her based on
your own experience?
My advice to a close friend, or anyone else for that matter, would be
exactly the same advice Steve Jobs would give. "You've got to find what
you love." If you love your job, not much else will matter.
You have a boatload of CTO awards, share a bit of your
kung fu with the rest of the industry.
Add value. Seriously that's the key. As an engineer one of the hardest
lessons for me to learn was that hard does not equal valuable, and
something that's valuable isn't necessarily hard. One has little to do
with the other. Whatever you plan to build/research and offer for
free/sale, its purpose must be to make a person's life
better/easier/richer. It doesn't always have to be elaborate or
complicated, just solve a problem that needed to be solved.
In a related question, I have followed Stephanie Fohn, your CEO's
career since she was COO at TripWire and then the amazing job she did
at SecurityFocus, so you have quite an impressive management team. How
do so many smart people with such strongly held opinions manage to come
to consensus when you have different takes on an issue?
Stephanie is great and a lot of WhiteHat's success has to do with her
leadership from the top. What also helps tremendously is that we're not
strangers to each other. Most of the management team has worked
together in the past and were brought in to WhiteHat specifically
because they were great at their job. So teamwork in the group probably
comes a lot easier for us than at other startups. Where I think
WhiteHat really excels organizationally is in fostering a culture such
people are placed in a position where they can succeed and make a real
What haven't I asked? This is your chance to grab the
pulpit, a platform from which to persuasively advocate an agenda,
drive home your Number One point that you are trying to make as a
thought leader in the industry?
1) Asset Tracking - you can't secure what
you can't control
2) Vulnerability Assessment - you can't
secure what you can't measure
3) Development Frameworks - you can't
mandate secure code, only help it
4) Web Application Firewalls - Because
eight in 10 websites are already insecure
5) Business Liability - Because the cost
of insecurity should be borne by those responsible for it
Jeremiah, we really want to thank you for your time and have one last
question, can you tell us just a bit about yourself? What do you like
to do when you are not in front of a computer?
You mean the remaining eight hours of the day? *smile* I
spend "free time"
with my two beautiful children (ages: 2 and 4) and lovely wife Llana
(age: she'd kill me). We go to the park and the pool a lot, learn to
play chess, and do other family type stuff. During the evenings I'm
heavy into Brazilian Jiu Jitsu training and on the weekends I'm usually
playing Aussie Rules Football. I also do my best to get back home to
Maui, Hawaii as often as I can. Yah, life is hard, what can I say.