homepage
Open menu Go one level top
  • Train and Certify
    • Get Started in Cyber
    • Courses & Certifications
    • Training Roadmap
    • Search For Training
    • Online Training
    • OnDemand
    • Live Training
    • Summits
    • Cyber Ranges
    • College Degrees & Certificates
    • NICE Framework
    • DoDD 8140
    • Specials
  • Manage Your Team
    • Overview
    • Security Awareness Training
    • Voucher Program
    • Private Training
    • Workforce Development
    • Skill Assessments
    • Hiring Opportunities
  • Resources
    • Overview
    • Reading Room
    • Webcasts
    • Newsletters
    • Blog
    • Tip of The Day
    • Posters
    • Top 25 Programming Errors
    • The Critical Security Controls
    • Security Policy Project
    • Critical Vulnerability Recaps
    • Affiliate Directory
  • Focus Areas
    • Blue Team Operations
    • Cloud Security
    • Digital Forensics & Incident Response
    • Industrial Control Systems
    • Leadership
    • Offensive Operations
  • Get Involved
    • Overview
    • SANS Community
    • CyberTalent
    • Work Study
    • Instructor Development
    • Sponsorship Opportunities
    • COINS
  • About
    • About SANS
    • Why SANS?
    • Instructors
    • Cybersecurity Innovation Awards
    • Contact
    • Frequently Asked Questions
    • Customer Reviews
    • Press Room
  • Log In
  • Join
  • Contact Us
  • SANS Sites
    • GIAC Security Certifications
    • Internet Storm Center
    • SANS Technology Institute
    • Security Awareness Training
  • Search
  1. Home >
  2. Blog >
  3. AI Wars
Deb Radcliff

AI Wars

Artificial intelligence in cybersecurity can introduce accidental or intentional bias that results in false security data.

December 12, 2019

The benefits of AI in cybersecurity are hard to dispute. When implemented properly, AI enhances detection and response, improves access controls, aids with threat hunting and other capabilities, and uses machine learning to continually improve response.

At the same time, AI is the tool of choice for malware creators who are developing AI-proof hacks against the security systems that deploy AI. Not to mention, the developers of AI-based security systems can accidentally introduce bias into AI-enabled security.

AI Bias

RiskSense CEO Srinivas Mukkamala, Ph.D., earned his doctorate in computational intelligence for large-scale intrusions. He explains that AI bias occurs in at least three places in the AI development and AI data consumption level: humans who are developing it, data that is being used to develop the models, and algorithms behind the AI decision-making program.

"You can actually introduce algorithmic bias that creates sloppy false positive and false negative rates because the program may try to balance the dataset by adding more data when there is not enough data available to the AI system," Mukkamala says.

Take, for example, AI that is based on the national vulnerability database. Of the more than 134,000 vulnerabilities listed as of December 2019, just over 24,000 are critical or high. A much smaller percent of those are capable of a remote code execution or privilege escalation, and only a few hundred of these vulnerabilities are commonly exploited, Mukkamala adds.

In fact, when RiskSense analysts examined a number of AI algorithms against the national vulnerability database, they reported between 25 and 30% false positive and false negative rates.

"Don't just trust your vendors that their AI is good. You have to ask, what is your AI model trained on, who built it, what were that person's biases?" continues Mukkamala, who recently moderated a panel on this topic at SINET. "The AI expertise needs to understand software risk, data modeling and mathematical modeling. And that modeling should be free from bias, which is the only way to remove false positives and false negatives from the equation."

AI Turned Against Us

In his blog about weaponizing AI security against organizations using this technology, Isaac Ben-Israel, director of the ICRC and chairman of Cyberweek, outlines ways AI-savvy malware could inject false data into the AI system and disrupt patterns used for machine-learning decision trees.

Ben-Israel also describes a process called "bobbing and weaving," wherein attackers input innocuous routines that make the AI system deem them normal machine behaviors; then later, attackers come back to exploit those routines without notice. Even changing log files can confuse the AI system, he adds.

Diana Kelley, a Microsoft CTO, said during an CNBC broadcast that AI must be resilient in order for it to remain accurate and stand up against AI-resistant attack methods.

AI Protecting AI

Microsoft's philosophy is that the AI/ML system itself should be able to detect bias in its systems. For instance, the system should discern maliciously introduced data from benign events, as well as have its own built-in self-forensics capabilities for transparency and accountability of the AI.

To do that, Microsoft recommends developing AI in a way that it can detect and overcome accidental or intentionally introduced bias in the datasets it trains on. The AI system itself must ensure the data it trains on is accurate and clean, while also detecting red-flagged words and phrases that would indicate bias.

Organizations purchasing and utilizing AI in their security modules need to monitor the quality of their data and changes to AI data input. And they should remember that AI is no replacement for the human analyst, but rather an augmentation.

"There's a lot of anxiety around AI right now," says Mukkamala. "What we're struggling with is not well-defined, and we're really just in the development phase of the process when it comes to AI in vulnerability and threat management."

Share:
TwitterLinkedInFacebook
Copy url Url was copied to clipboard
Subscribe to SANS Newsletters
Join the SANS Community to receive the latest curated cybersecurity news, vulnerabilities, and mitigations, training opportunities, plus our webcast schedule.
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kingdom of Saudi Arabia
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia (Slovak Republic)
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe

Tags:
  • DevSecOps

Related Content

Blog
Cloud_Ace_Final.png
Cloud Security, DevSecOps
November 20, 2020
Instructor Spotlight: Brandon Evans, SEC510 Lead Author
Get to know SANS Certified Instructor and SANS Cloud Ace, Brandon Evans.
BrandonEvans_Headshot_370x370.png
Brandon Evans
read more
Blog
Cloud Security, DevSecOps
February 24, 2020
The Internet of things: Making Mountains Out of Microprocessors
It is estimated that there are approximately eight billion IoT devices connected to the internet and this number is set to continue to grow.
Simon McNamee
read more
Blog
DevSecOps
October 14, 2018
Exploring the DevSecOps Toolchain
The authors of the SANS Institute's DEV540 Secure DevOps & Cloud Application Security course created the Cloud Security and DevSecOps Best Practices poster to help security teams create a methodology for integrating security into the DevOps workflow. As you can see, the poster breaks DevOps down...
Eric_Johnson_370x370.png
Eric Johnson
read more
  • Register to Learn
  • Courses
  • Certifications
  • Degree Programs
  • Cyber Ranges
  • Job Tools
  • Security Policy Project
  • Posters
  • The Critical Security Controls
  • Focus Areas
  • Blue Team Operations
  • Cloud Security
  • Cybersecurity Leadership
  • Digital Forensics
  • Industrial Control Systems
  • Offensive Operations
Subscribe to SANS Newsletters
Join the SANS Community to receive the latest curated cybersecurity news, vulnerabilities, and mitigations, training opportunities, plus our webcast schedule.
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kingdom of Saudi Arabia
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia (Slovak Republic)
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe
  • © 2021 SANS™ Institute
  • Privacy Policy
  • Contact
  • Twitter
  • Facebook
  • Youtube
  • LinkedIn