homepage
Open menu
Go one level top
  • Train and Certify
    Train and Certify

    Immediately apply the skills and techniques learned in SANS courses, ranges, and summits

    • Overview
    • Courses
      • Overview
      • Full Course List
      • By Focus Areas
        • Cloud Security
        • Cyber Defense
        • Cybersecurity and IT Essentials
        • DFIR
        • Industrial Control Systems
        • Offensive Operations
        • Management, Legal, and Audit
      • By Skill Levels
        • New to Cyber
        • Essentials
        • Advanced
        • Expert
      • Training Formats
        • OnDemand
        • In-Person
        • Live Online
      • Course Demos
    • Training Roadmaps
      • Skills Roadmap
      • Focus Area Job Roles
        • Cyber Defence Job Roles
        • Offensive Operations Job Roles
        • DFIR Job Roles
        • Cloud Job Roles
        • ICS Job Roles
        • Leadership Job Roles
      • NICE Framework
        • Security Provisionals
        • Operate and Maintain
        • Oversee and Govern
        • Protect and Defend
        • Analyze
        • Collect and Operate
        • Investigate
        • Industrial Control Systems
    • GIAC Certifications
    • Training Events & Summits
      • Events Overview
      • Event Locations
        • Asia
        • Australia & New Zealand
        • Latin America
        • Mainland Europe
        • Middle East & Africa
        • Scandinavia
        • United Kingdom & Ireland
        • United States & Canada
      • Summits
    • OnDemand
    • Get Started in Cyber
      • Overview
      • Degree and Certificate Programs
      • Scholarships
    • Cyber Ranges
  • Manage Your Team
    Manage Your Team

    Build a world-class cyber team with our workforce development programs

    • Overview
    • Why Work with SANS
    • Group Purchasing
    • Build Your Team
      • Team Development
      • Assessments
      • Private Training
      • Hire Cyber Professionals
      • By Industry
        • Health Care
        • Industrial Control Systems Security
        • Military
    • Leadership Training
  • Security Awareness
    Security Awareness

    Increase your staff’s cyber awareness, help them change their behaviors, and reduce your organizational risk

    • Overview
    • Products & Services
      • Security Awareness Training
        • EndUser Training
        • Phishing Platform
      • Specialized
        • Developer Training
        • ICS Engineer Training
        • NERC CIP Training
        • IT Administrator
      • Risk Assessments
        • Knowledge Assessment
        • Culture Assessment
        • Behavioral Risk Assessment
    • OUCH! Newsletter
    • Career Development
      • Overview
      • Training & Courses
      • Professional Credential
    • Blog
    • Partners
    • Reports & Case Studies
  • Resources
    Resources

    Enhance your skills with access to thousands of free resources, 150+ instructor-developed tools, and the latest cybersecurity news and analysis

    • Overview
    • Webcasts
    • Free Cybersecurity Events
      • Free Events Overview
      • Summits
      • Solutions Forums
      • Community Nights
    • Content
      • Newsletters
        • NewsBites
        • @RISK
        • OUCH! Newsletter
      • Blog
      • Podcasts
      • Summit Presentations
      • Posters & Cheat Sheets
    • Research
      • White Papers
      • Security Policies
    • Tools
    • Focus Areas
      • Cyber Defense
      • Cloud Security
      • Digital Forensics & Incident Response
      • Industrial Control Systems
      • Cyber Security Leadership
      • Offensive Operations
  • Get Involved
    Get Involved

    Help keep the cyber community one step ahead of threats. Join the SANS community or begin your journey of becoming a SANS Certified Instructor today.

    • Overview
    • Join the Community
    • Work Study
    • Teach for SANS
    • CISO Network
    • Partnerships
    • Sponsorship Opportunities
  • About
    About

    Learn more about how SANS empowers and educates current and future cybersecurity practitioners with knowledge and skills

    • SANS
      • Overview
      • Our Founder
      • Awards
    • Instructors
      • Our Instructors
      • Full Instructor List
    • Mission
      • Our Mission
      • Diversity
      • Scholarships
    • Contact
      • Contact Customer Service
      • Contact Sales
      • Press & Media Enquiries
    • Frequent Asked Questions
    • Customer Reviews
    • Press
    • Careers
  • Contact Sales
  • SANS Sites
    • GIAC Security Certifications
    • Internet Storm Center
    • SANS Technology Institute
    • Security Awareness Training
  • Search
  • Log In
  • Join
    • Account Dashboard
    • Log Out
  1. Home >
  2. Blog >
  3. Checklists, Software and Software Security
Jim Bird

Checklists, Software and Software Security

May 2, 2011

It took me too long to finally read Dr. Atul Gawande's Checklist Manifesto, a surprisingly interesting story of how asking surgeons and nurses to follow a simple list of checks, adding just a little bit of red tape, can save lives and save lots of money. It starts with the pioneering work done by Dr. Peter Provonost at Johns Hopkins, where they used checklists to catch stupid mistakes in essential ICU practices, proving that adding some trivial structure could make a real difference in life-critical situations. The book walks through how checklists are relied on by pilots, and how experts at Boeing and other companies continuously perfect these checklists to handle any problem in the air. The rest of the book focuses on the work that Dr. Gawande led at the World Health Organization to develop and test a simple, 3-section, 19-step, 2-minute checklist for safe surgery to be used in hospitals around the world.

A few points especially stood out for me:

First, that so many problems and complications at hospitals are not caused by doctors and nurses not knowing what to do — but because they forget to do what they already know how to do, because they forget trivial basic details, or "lull themselves into skipping steps even when they remember them". And that this applies to complex problems in many other disciplines.

Second, that following a simple set of basic checks, just writing them down and making sure that people double-check that they are doing what they know they are supposed to be doing, can make such a dramatic difference in the quality of complex work, work that is so dependent on expert skill.

It made we think more about how checklists can help in building and maintaining software. If they can find ways to make checklists work in surgery, then we can sure as hell find ways to make them work in software development.

There are already places where checklists are used or can be used in software development. Steve McConnell's Code Complete is full of comprehensive checklists that programmers can follow to help make sure that they are doing a professional job at every step in building software. These checklists act as reminders of best practices, a health check on how a developer should do their job — but there's too much here to use on a day-to-day basis. In his other book Software Estimation: Demystifying the Black Art he also recommends using checklists for creating estimates, and offers a short checklist to help guide developers and managers through choosing an estimation approach.

One place where checklists are used more often is in code reviews, to make sure that reviewers remember to check for what the team (or management, or auditors) have agreed is important. There are a lot of arguments over what should be included in a code review checklist, what's important to check and what's not, and how many checks are too many.

In the "Modern Code Review" chapter of Making Software, Jason Cohen reinforces Dr Gawande's finding that long checklists are a bad idea. If review checklists are too long, programmers will just ignore them. He recommends that code review checklists should be no more than 10 items at the most. What surprised me is that according to Cohen, the most effective checklists are even shorter, with only 2 or 3 items to check: micro-checklists that focus in on the mistakes that the team commonly makes, or the mistakes that have cost them the most. Then, once people on the team stop making these mistakes, or if more important problems are found, you come up with a new checklist.

Cohen says that most code review checklists contain obvious items that are a waste of time like "Does the code accomplish what it is meant to do?" and "Can you understand the code as written?" and so on. Most items on long checklists are unnecessary (of course the reviewer is going to check if the code works and that they can understand it) or fuss about coding style and conventions, which can be handled through static analysis checkers. Checklists should only include common mistakes that cause real problems.

He also recommends that rather than relying on general-purpose checklists, programmers should build their own personal code review checklists, taking an idea from the SEI Personal Software Process (PSP). Since different people make different mistakes, each programmer should come up with their own short checklist of the most serious mistakes that they make on a consistent basis, especially the things that they find that they commonly forget to do, and share this list with the people reviewing their code.

There are still places for longer code review checklists, for areas where reviewers need more hand holding and guidance. Like checking for security vulnerabilities in code. OWASP provides a simple and useful secure coding practices quick reference guide which can be used to build a checklist for secure code reviews. This is work that programmers don't do every day, so you're less concerned about being efficient than you are about making sure that the reviewer covers all of the important bases. You need to make sure that security code reviews are comprehensive and disciplined, and you may need to provide evidence of this, making a tool like Agnitio interesting. But even in a detailed secure code review, you want to make sure that every check is clearly needed, clearly understood and essentially important.

Checklists help you check for what isn't there?

In 11 Best Practices for Peer Code Review, Jason Cohen hilights that checklists are important in code reviews because they help remind reviewers to look beyond what is in the code for what isn't there:

"Checklists are especially important for reviewers, since if the author forgot it, the reviewer is likely to miss it as well".

But can we take this one step farther (or one step back) in software development? Can we go back to fundamental work where programmers commonly make important mistakes, and come up with a simple checklist that will stop people from making these mistakes in the first place? Like the ICU central intravenous line problem that Dr. Provonost started with — a basic but important practice where adding simple checks can save big.

The first problem that comes to my mind is data validation, the cause of roughly half of all software security problems according to Michael Howard at Microsoft. Data validation, like a lot of problems in software security, is one of the worst kinds of problems. It's fundamentally, conceptually simple: everyone understands (or they should) that you need to validate and filter input data, and escape output data. It's one of the basic issues covered in AppSec training from SANS, in books on secure development. But it's like surgery: you have to take all of it seriously, and you need to do it right, every little thing, every time. There are a lot of finicky details to get right, a lot of places to go wrong, and you can't afford to make any mistakes.

This is the kind of work that cries out for a checklist, a clear, concrete set of steps that programmers can follow. Or like pilots, a set of different checklists that programmers can follow in different situations. So that we build software right in the first place. It works in other industries. Why can't it work in ours?

Share:
TwitterLinkedInFacebook
Copy url Url was copied to clipboard
Subscribe to SANS Newsletters
Receive curated news, vulnerabilities, & security awareness tips
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Saudi Arabia
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe

By providing this information, you agree to the processing of your personal data by SANS as described in our Privacy Policy.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Tags:
  • DevSecOps

Related Content

Blog
powershell_option_340x340.jpg
Cyber Defense, DevSecOps, Digital Forensics and Incident Response, Cybersecurity and IT Essentials, Penetration Testing and Red Teaming
July 15, 2022
Month of PowerShell: Working with PowerShell Log Files
In this article we'll look at how we can leverage PowerShell's object-passing pipeline to parse and retrieve data from an IIS web server log file.
370x370_Joshua-Wright.jpg
Joshua Wright
read more
Blog
CloudSecNext Summit Visual Summary
Cloud Security, DevSecOps
May 3, 2022
A Visual Summary of SANS CloudSecNext Summit 2022
On May 3-4, thousands from around the globe tuned in for the SANS CloudSecNext Summit. We invited Ashton Rodenhiser to create graphic recordings of our Summit presentations. If you missed a talk or are looking to view the SANS CloudSecNext Summit through a visual lens, take a look at the recordings...
370x370-person-placeholder.png
Alison Kim
read more
Blog
DevSecOps
May 4, 2015
2015 State of Application Security: Closing the Gap
The 2015 SANS State of Application Security Analyst Paper and webcasts are complete. This year, Jim Bird, the lead author of the SANS Application Security Survey series, Frank Kim, and I all participated in writing the questions, analyzing the results, drafting the paper, and preparing the webcast...
Eric_Johnson_370x370.png
Eric Johnson
read more
  • Register to Learn
  • Courses
  • Certifications
  • Degree Programs
  • Cyber Ranges
  • Job Tools
  • Security Policy Project
  • Posters & Cheat Sheets
  • White Papers
  • Focus Areas
  • Cyber Defense
  • Cloud Security
  • Cybersecurity Leadership
  • Digital Forensics
  • Industrial Control Systems
  • Offensive Operations
Subscribe to SANS Newsletters
Receive curated news, vulnerabilities, & security awareness tips
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Saudi Arabia
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe

By providing this information, you agree to the processing of your personal data by SANS as described in our Privacy Policy.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
  • © 2023 SANS™ Institute
  • Privacy Policy
  • Contact
  • Careers
  • Twitter
  • Facebook
  • Youtube
  • LinkedIn