homepage
Open menu Go one level top
  • Train and Certify
    • Get Started in Cyber
    • Courses & Certifications
    • Training Roadmap
    • Search For Training
    • Online Training
    • OnDemand
    • Live Training
    • Summits
    • Cyber Ranges
    • College Degrees & Certificates
    • NICE Framework
    • DoDD 8140
    • Specials
  • Manage Your Team
    • Overview
    • Group Purchasing
    • Why Work with SANS
    • Build Your Team
    • Hire Cyber Talent
    • Team Development
    • Private Training
    • Security Awareness Training
    • Leadership Training
    • Industries
  • Resources
    • Overview
    • Reading Room
    • Webcasts
    • Newsletters
    • Blog
    • Tip of The Day
    • Posters
    • Top 25 Programming Errors
    • The Critical Security Controls
    • Security Policy Project
    • Critical Vulnerability Recaps
    • Affiliate Directory
  • Focus Areas
    • Blue Team Operations
    • Cloud Security
    • Digital Forensics & Incident Response
    • Industrial Control Systems
    • Leadership
    • Offensive Operations
  • Get Involved
    • Overview
    • Work Study
    • Teach for SANS
    • Partnerships
    • Sponsorship Opportunities
    • Join the Community
  • About
    • About SANS
    • Instructors
    • Mission
    • Initiatives
    • Diversity
    • Awards
    • Contact
    • Frequently Asked Questions
    • Customer Reviews
    • Press
  • Log In
  • Join
  • Contact Us
  • SANS Sites
    • GIAC Security Certifications
    • Internet Storm Center
    • SANS Technology Institute
    • Security Awareness Training
  • Search
  1. Home >
  2. Blog >
  3. Accessing Web APIs with PowerShell
370x370_Clay-Risenhoover.jpg
Clay Risenhoover

Accessing Web APIs with PowerShell

This is Part 3 of a 3-part Series on Using PowerShell for Continuous Audit & Compliance Automation in Enterprise & Cloud

March 10, 2021

This is the third of a three-part series on using PowerShell for audit and compliance measurements. These blog posts supplement the material presented in the free webcast series "PowerShell for Audit, Compliance, and Security Automation and Visualization".

  • Read Part 1 of the Blog here
  • Read Part 2 of the Blog here

When I work with clients to automate their compliance efforts, I frequently find myself writing a PowerShell script to import or export data using a web API (application programming interface). In this post, I'll share some tips and techniques for working with web APIs using PowerShell.

Web APIs, REST and SOAP

The term "Web API" describes any HTTP-based interface which has been developed to allow users or programs to interact with a software system. APIs usually expose data directly in some structured format like JSON or XML, without wrapping any sort of user interface around it. This straightforward data access makes API access a perfect technique for automated tasks. Let's begin with a brief description of the two types of APIs you are likely to encounter, REST and SOAP.

The original idea behind REST services was first proposed by Roy Fielding in his doctoral dissertation. Fielding's idea has evolved (been appropriated?) so that many REST services map the database CRUD (create, retrieve, update, delete) functions to existing HTTP verbs, like POST, GET, PUT, PATCH and DELETE. This straightforward design has made REST services very popular over the last several years. Many of the security and compliance tools I use in my practice make REST APIs available for controlling settings, launching actions and importing and exporting data. Many of today's REST APIs use JSON (JavaScript Object Notation) as their data interchange format.

SOAP (simple object access protocol) is the XML-based data interchange protocol that drove much of the "service-oriented architecture" revolution in web applications a few years ago. The good news is that everything is well defined in most SOAP APIs. The bad news is that compared to handling JSON data from a REST API, there's a lot of overhead involved in creating a well-structured XML document for the request and then parsing the XML-based response. Fortunately, if you are running Windows PowerShell, I'll be able to show you an often-overlooked shortcut for handling SOAP services.

Accessing REST with Invoke-RestMethod

The first step to working with any API is to review the documentation which describes the API calls, accepted data types, authentication requirements, response formatting, etc. Many REST APIs use the OpenAPI Specification, the renamed and updated version of the older Swagger documentation standard. For this post, we will consume data from GitHub's Issue-tracking API. It is documented here. This API allows us to make both authenticated an unauthenticated requests, so we'll do a little of each.

Imagine that management has asked us to retrieve a list of the 100 most recently closed issues for a particular GitHub repository, and to calculate the mean time to resolution (MTTR) for the 100 issues. Time to resolution (TTR) for a single issue would be the number of days which elapsed between the issue being opened and closed. The MTTR will simply be the average of the TTR for all 100 issues. For this example, we'll use issues from the Microsoft PowerShell repository, since it is pretty active. Upon reviewing the API documentation, we see that we can issue a GET request with query string parameters for the number of issues (up to 100) to return per "page" or request, the state of the issues (open, closed or all) to be returned, and a page number, which tells the API which result page to return.

Armed with that information, we build the PowerShell command to make a test request. We'll start by asking for a single result, using the Invoke-RestMethod cmdlet. It will make the web request and then convert the JSON data returned by the API into a PowerShell object. We'll save the object to a variable to make it easier to analyze. Then we'll use Get-Member to view the properties of the object.

$testResult = (Invoke-RestMethod -Uri 
"https://api.github.com/repos/PowerShell/PowerShell/issues?per_page=1&page=0&state=closed")
$testResult | Get-Member

Blog3-1.png

The created_at and closed_at DateTime objects look like they would be useful for calculating the time to resolve an issue. We'll use a TimeSpan object to calculate the TTR for this issue:

New-TimeSpan -Start $testResult.created_at -End $testResult.closed_at

Blog3-2.png

Now that we know how to calculate the TTR for a single issue, let's gather the 100 most recently closed issues and calculate their MTTR. First, we'll make a GET request and save the results into a variable, then we'll count the results to ensure we got a full 100:

$issues = (Invoke-RestMethod -Uri 
"https://api.github.com/repos/PowerShell/PowerShell/issues?per_page=100&page=0&state=closed")
PS > $issues.count
100

Blog3-3.png

Then, we can calculate the MTTR. We'll use a calculated property for the TTR for each issue, and then use Measure-Object to get the average:

($issues | 
Select-Object @{n='TimeToResolve'; e={(New-TimeSpan -Start $_.created_at -End $_.closed_at).TotalDays} } |
Measure-Object -Property TimeToResolve -Average).Average

Blog3-4.png

The answer to management's question is that the MTTR for the last 100 tickets is 3.14 days. The InvokeRestMethod cmdlet took most of the work out of the problem, and allowed us to focus on gathering the needed data!

Passing Authentication Tokens

Many REST calls will require authentication. GitHub requires you to authenticate if you want to make more than 60 requests during an hour. Let's re-work the previous example with more data. Management has asked for the MTTR for all tickets closed within the last 90 days. This will require requesting multiple pages of data, and it may put us over the API rate limit for unauthenticated requests.

We'll start by logging onto Github and issuing a new API token for accessing public repositories. If you have a GitHub account, you can create a token of your own. Be sure to copy the token from the results page. Once you close that page, you won't be able to retrieve the token in plain text again.

Blog3-5.png

Blog3-6.png

Blog3-7.png

Blog3-8.png

We'll pass our newly created token to Github with each request to authenticate that it came from us, so let's save it in a variable. We'll also calculate the date for 90 days ago and save it in the time format required by the API:

$token = Read-Host -AsSecureString "Paste in GitHub Token"
$since = (Get-Date -Date (Get-Date).addDays(-90) -Format "yyyyMMddTHHmmssZ" -Hour 0 -Minute 0 -Second 0 -Millisecond 0)

Blog3-9.png

We'll need to build a PSCredential object to pass our authentication token using HTTP basic authentication. It will be passed using the Credential parameter of Invoke-RestMethod.

$githubUsername = 'Sec557-Demo'
$cred=New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $githubUsername, $token

Next, we'll build a WHILE loop to keep retrieving pages of data until we have retrieved all the issues closed after the date we specified. We'll add all the issues to a variable so we can do the same math we did above on this larger dataset.

$issues = $null
#Counter variables
$page = -1
$count = 100

#Loop until you receive < 100 results on the page
while ( $count -eq 100)
{

<p>$page++
    "Processing page: $page"

    #Set the URL for the request, plugging in $page as the page number
    $uri = "https://api.github.com/repos/PowerShell/PowerShell/issues?page=$page&per_page=100&state=closed&since=$since"

    #Get the next page and add the contents to the $issues variable
    $nextPage = Invoke-RestMethod -Credential $cred -Uri $uri
    $count = $nextPage.count
    $issues += $nextPage</p>

After retrieving several pages, we received a total of 793 issues.

Blog3-10.png

Finally, we can calculate the MTTR for this larger dataset using the same math as above:

($issues | 
<p>Select-Object @{n='TimeToResolve'; e={(New-TimeSpan -Start $_.created_at -End $_.closed_at).TotalDays} } | 
  Measure-Object -Property TimeToResolve -Average).Average</p>

Blog3-11.png

Again, the authentication and data conversion are handled transparently, allowing us to focus on the data!

Accessing SOAP the Hard Way - PowerShell Core

All the examples we've seen so far will work correctly in PowerShell Core and Windows PowerShell. If you find yourself needing to access a SOAP API, I strongly recommend that you find a Windows computer so you can use Windows PowerShell. The process in PS Core is largely manual. You would need to:

  • Read the WSDL (web service definition language) file to understand the XML schema used for making requests and receiving responses.
  • Build the XML-formatted SOAP request and pass it as the body using Invoke-WebRequest.
  • Save the XML document returned in the response and use a .NET XMLDocument object to load it into a variable.
  • Work with the XMLDocument variable to extract your results.
#Based on the WSDL, create the request to get a list of country names and ISO codes.
$body = '<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:web="http://www.oorsprong.org/websamples.countryinfo">
   <soapenv:Header/>
   <soapenv:Body>
      <web:ListOfCountryNamesByName/>
   </soapenv:Body>
</soapenv:Envelope>
'

#Make the web request and save the result to an XML file
(Invoke-WebRequest -UseBasicParsing -method Post -Body $body `
  -uri 'http://webservices.oorsprong.org/websamples.countryinfo/CountryInfoService.wso' `
  -ContentType "text/xml").content | Out-File -Path countries.xml -Force

#Load the file into an XMLDocument object
$xmlCountries = New-Object System.Xml.XmlDocument
$fn = Resolve-Path(".\countries.xml")
$xmlCountries.Load($fn)

#Drill down in the XML to the actual result and then query it for the 'GB' ISO code
$countryCodes = $xmlCountries.envelope.body.ListOfCountryNamesByNameResponse.ListOfCountryNamesByNameResult.tCountryCodeAndName | Select-Object sISOCode, sName

$countryCodes | Where-Object sISOCode -eq 'GB'

Blog3-12.png

While this is not a complicated problem, it does require a number of steps and a fait bit of research.

Accessing SOAP the Easy Way - Windows PowerShell

The simple way to access SOAP is by using the New-WebServiceProxy cmdlet. It handles all the setup and data conversion for you. Simply point the proxy at the WSDL file and then use the auto-generated object to access the API. To demonstrate this, we'll use a free SOAP API which returns country information like mappings of ISO codes to country names.

$wsp=New-WebServiceProxy -Uri http://webservices.oorsprong.org/websamples.countryinfo/CountryInfoService.wso?wsdl
$wsp | Get-Member -Type Method

Blog3-13.png

One of the methods returned by the API is called ListOfCountryNamesByName. It returns a list of countries and their ISO codes. To call this function, we'll just use the variable created above.

$countryCodes = $wsp.ListOfCountryNamesByName()
$countryCodes | Where-Object sISOCode -eq 'GB'

That's it! Five lines of PowerShell yielded the same results as you achieved in the previous section, with no real analysis required! In the rare event that you have to interact with a SOAP-based API, I recommend that you use Windows PowerShell and the New-WebServiceProxy cmdlet whenever possible.

ABOUT THE AUTHOR

Clay is the president of Risenhoover Consulting, Inc., an IT management consulting firm based in Durant, Oklahoma. Founded in 2003, RCI provides IT audit and IT management consulting services to clients in multiple sectors. Clay is a licensed Certified Public Accountant (CPA) with the Certified Information Technology Professional (CITP) designation. Since 2013, Clay has been involved with the SANS Institute and is the lead author and instructor for AUD507: Auditing & Monitoring Networks, Perimeters, and Systems, the sole author of the brand new SEC557: Continuous Automation for Enterprise and Cloud Monitoring, as well as a Faculty Research Advisor (FRA) for the SANS Technology Institute. Learn More About Clay.

Share:
TwitterLinkedInFacebook
Copy url Url was copied to clipboard
Subscribe to SANS Newsletters
Join the SANS Community to receive the latest curated cybersecurity news, vulnerabilities, and mitigations, training opportunities, plus our webcast schedule.
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kingdom of Saudi Arabia
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia (Slovak Republic)
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe

Tags:
  • Cloud Security
  • Security Management, Legal, and Audit

Related Content

Blog
DFIR_FOR509_Countdown_Social4.jpg
Digital Forensics and Incident Response, Cloud Security
April 9, 2021
NEW FOR509: Enterprise Cloud Forensics & Incident Response - Beta coming June 2021
The new Enterprise Cloud Forensics course brings examiners up to speed with the rapidly changing world of enterprise cloud
SANS DFIR
read more
Blog
SEC540_new_370x370.png
Cloud Security, DevSecOps
April 1, 2021
What's New in SEC540: Cloud Security and DevSecOps Automation
The Cloud Moves Fast. Automate to Keep Up.
Eric_Johnson_370x370.png
Eric Johnson
read more
Blog
Cyber42_Logo_black_370x370.png
Security Management, Legal, and Audit
March 31, 2021
Cyber42 Cybersecurity Leadership Simulation Games
The NetWars of SANS Cybersecurity Leadership Curriculum. Can you win?
370x370_Frank-Kim.jpg
Frank Kim
read more
  • Register to Learn
  • Courses
  • Certifications
  • Degree Programs
  • Cyber Ranges
  • Job Tools
  • Security Policy Project
  • Posters
  • The Critical Security Controls
  • Focus Areas
  • Blue Team Operations
  • Cloud Security
  • Cybersecurity Leadership
  • Digital Forensics
  • Industrial Control Systems
  • Offensive Operations
Subscribe to SANS Newsletters
Join the SANS Community to receive the latest curated cybersecurity news, vulnerabilities, and mitigations, training opportunities, plus our webcast schedule.
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kingdom of Saudi Arabia
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia (Slovak Republic)
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe
  • © 2021 SANSâ„¢ Institute
  • Privacy Policy
  • Contact
  • Twitter
  • Facebook
  • Youtube
  • LinkedIn