homepage
Open menu
Go one level top
  • Train and Certify
    Train and Certify

    Immediately apply the skills and techniques learned in SANS courses, ranges, and summits

    • Overview
    • Courses
      • Overview
      • Full Course List
      • By Focus Areas
        • Cloud Security
        • Cyber Defense
        • Cybersecurity and IT Essentials
        • DFIR
        • Industrial Control Systems
        • Offensive Operations
        • Management, Legal, and Audit
      • By Skill Levels
        • New to Cyber
        • Essentials
        • Advanced
        • Expert
      • Training Formats
        • OnDemand
        • In-Person
        • Live Online
      • Course Demos
    • Training Roadmaps
      • Skills Roadmap
      • Focus Area Job Roles
        • Cyber Defence Job Roles
        • Offensive Operations Job Roles
        • DFIR Job Roles
        • Cloud Job Roles
        • ICS Job Roles
        • Leadership Job Roles
      • NICE Framework
        • Security Provisionals
        • Operate and Maintain
        • Oversee and Govern
        • Protect and Defend
        • Analyze
        • Collect and Operate
        • Investigate
        • Industrial Control Systems
      • European Skills Framework
    • GIAC Certifications
    • Training Events & Summits
      • Events Overview
      • Event Locations
        • Asia
        • Australia & New Zealand
        • Latin America
        • Mainland Europe
        • Middle East & Africa
        • Scandinavia
        • United Kingdom & Ireland
        • United States & Canada
      • Summits
    • OnDemand
    • Get Started in Cyber
      • Overview
      • Degree and Certificate Programs
      • Scholarships
    • Cyber Ranges
  • Manage Your Team
    Manage Your Team

    Build a world-class cyber team with our workforce development programs

    • Overview
    • Why Work with SANS
    • Group Purchasing
    • Build Your Team
      • Team Development
      • Assessments
      • Private Training
      • Hire Cyber Professionals
      • By Industry
        • Health Care
        • Industrial Control Systems Security
        • Military
    • Leadership Training
  • Security Awareness
    Security Awareness

    Increase your staff’s cyber awareness, help them change their behaviors, and reduce your organizational risk

    • Overview
    • Products & Services
      • Security Awareness Training
        • EndUser Training
        • Phishing Platform
      • Specialized
        • Developer Training
        • ICS Engineer Training
        • NERC CIP Training
        • IT Administrator
      • Risk Assessments
        • Knowledge Assessment
        • Culture Assessment
        • Behavioral Risk Assessment
    • OUCH! Newsletter
    • Career Development
      • Overview
      • Training & Courses
      • Professional Credential
    • Blog
    • Partners
    • Reports & Case Studies
  • Resources
    Resources

    Enhance your skills with access to thousands of free resources, 150+ instructor-developed tools, and the latest cybersecurity news and analysis

    • Overview
    • Webcasts
    • Free Cybersecurity Events
      • Free Events Overview
      • Summits
      • Solutions Forums
      • Community Nights
    • Content
      • Newsletters
        • NewsBites
        • @RISK
        • OUCH! Newsletter
      • Blog
      • Podcasts
      • Summit Presentations
      • Posters & Cheat Sheets
    • Research
      • White Papers
      • Security Policies
    • Tools
    • Focus Areas
      • Cyber Defense
      • Cloud Security
      • Digital Forensics & Incident Response
      • Industrial Control Systems
      • Cyber Security Leadership
      • Offensive Operations
  • Get Involved
    Get Involved

    Help keep the cyber community one step ahead of threats. Join the SANS community or begin your journey of becoming a SANS Certified Instructor today.

    • Overview
    • Join the Community
    • Work Study
    • Teach for SANS
    • CISO Network
    • Partnerships
    • Sponsorship Opportunities
  • About
    About

    Learn more about how SANS empowers and educates current and future cybersecurity practitioners with knowledge and skills

    • SANS
      • Overview
      • Our Founder
      • Awards
    • Instructors
      • Our Instructors
      • Full Instructor List
    • Mission
      • Our Mission
      • Diversity
      • Scholarships
    • Contact
      • Contact Customer Service
      • Contact Sales
      • Press & Media Enquiries
    • Frequent Asked Questions
    • Customer Reviews
    • Press
    • Careers
  • Contact Sales
  • SANS Sites
    • GIAC Security Certifications
    • Internet Storm Center
    • SANS Technology Institute
    • Security Awareness Training
  • Search
  • Log In
  • Join
    • Account Dashboard
    • Log Out
  1. Home >
  2. Blog >
  3. Accessing Web APIs with PowerShell
370x370_Clay-Risenhoover.jpg
Clay Risenhoover

Accessing Web APIs with PowerShell

This is Part 3 of a 3-part Series on Using PowerShell for Continuous Audit & Compliance Automation in Enterprise & Cloud

March 10, 2021

This is the third of a three-part series on using PowerShell for audit and compliance measurements. These blog posts supplement the material presented in the free webcast series "PowerShell for Audit, Compliance, and Security Automation and Visualization".

  • Read Part 1 of the Blog here
  • Read Part 2 of the Blog here

When I work with clients to automate their compliance efforts, I frequently find myself writing a PowerShell script to import or export data using a web API (application programming interface). In this post, I'll share some tips and techniques for working with web APIs using PowerShell.

Web APIs, REST and SOAP

The term "Web API" describes any HTTP-based interface which has been developed to allow users or programs to interact with a software system. APIs usually expose data directly in some structured format like JSON or XML, without wrapping any sort of user interface around it. This straightforward data access makes API access a perfect technique for automated tasks. Let's begin with a brief description of the two types of APIs you are likely to encounter, REST and SOAP.

The original idea behind REST services was first proposed by Roy Fielding in his doctoral dissertation. Fielding's idea has evolved (been appropriated?) so that many REST services map the database CRUD (create, retrieve, update, delete) functions to existing HTTP verbs, like POST, GET, PUT, PATCH and DELETE. This straightforward design has made REST services very popular over the last several years. Many of the security and compliance tools I use in my practice make REST APIs available for controlling settings, launching actions and importing and exporting data. Many of today's REST APIs use JSON (JavaScript Object Notation) as their data interchange format.

SOAP (simple object access protocol) is the XML-based data interchange protocol that drove much of the "service-oriented architecture" revolution in web applications a few years ago. The good news is that everything is well defined in most SOAP APIs. The bad news is that compared to handling JSON data from a REST API, there's a lot of overhead involved in creating a well-structured XML document for the request and then parsing the XML-based response. Fortunately, if you are running Windows PowerShell, I'll be able to show you an often-overlooked shortcut for handling SOAP services.

Accessing REST with Invoke-RestMethod

The first step to working with any API is to review the documentation which describes the API calls, accepted data types, authentication requirements, response formatting, etc. Many REST APIs use the OpenAPI Specification, the renamed and updated version of the older Swagger documentation standard. For this post, we will consume data from GitHub's Issue-tracking API. It is documented here. This API allows us to make both authenticated an unauthenticated requests, so we'll do a little of each.

Imagine that management has asked us to retrieve a list of the 100 most recently closed issues for a particular GitHub repository, and to calculate the mean time to resolution (MTTR) for the 100 issues. Time to resolution (TTR) for a single issue would be the number of days which elapsed between the issue being opened and closed. The MTTR will simply be the average of the TTR for all 100 issues. For this example, we'll use issues from the Microsoft PowerShell repository, since it is pretty active. Upon reviewing the API documentation, we see that we can issue a GET request with query string parameters for the number of issues (up to 100) to return per "page" or request, the state of the issues (open, closed or all) to be returned, and a page number, which tells the API which result page to return.

Armed with that information, we build the PowerShell command to make a test request. We'll start by asking for a single result, using the Invoke-RestMethod cmdlet. It will make the web request and then convert the JSON data returned by the API into a PowerShell object. We'll save the object to a variable to make it easier to analyze. Then we'll use Get-Member to view the properties of the object.

$testResult = (Invoke-RestMethod -Uri 
"https://api.github.com/repos/PowerShell/PowerShell/issues?per_page=1&page=0&state=closed")
$testResult | Get-Member

Blog3-1.png

The created_at and closed_at DateTime objects look like they would be useful for calculating the time to resolve an issue. We'll use a TimeSpan object to calculate the TTR for this issue:

New-TimeSpan -Start $testResult.created_at -End $testResult.closed_at

Blog3-2.png

Now that we know how to calculate the TTR for a single issue, let's gather the 100 most recently closed issues and calculate their MTTR. First, we'll make a GET request and save the results into a variable, then we'll count the results to ensure we got a full 100:

$issues = (Invoke-RestMethod -Uri 
"https://api.github.com/repos/PowerShell/PowerShell/issues?per_page=100&page=0&state=closed")
PS > $issues.count
100

Blog3-3.png

Then, we can calculate the MTTR. We'll use a calculated property for the TTR for each issue, and then use Measure-Object to get the average:

($issues | 
Select-Object @{n='TimeToResolve'; e={(New-TimeSpan -Start $_.created_at -End $_.closed_at).TotalDays} } |
Measure-Object -Property TimeToResolve -Average).Average

Blog3-4.png

The answer to management's question is that the MTTR for the last 100 tickets is 3.14 days. The InvokeRestMethod cmdlet took most of the work out of the problem, and allowed us to focus on gathering the needed data!

Passing Authentication Tokens

Many REST calls will require authentication. GitHub requires you to authenticate if you want to make more than 60 requests during an hour. Let's re-work the previous example with more data. Management has asked for the MTTR for all tickets closed within the last 90 days. This will require requesting multiple pages of data, and it may put us over the API rate limit for unauthenticated requests.

We'll start by logging onto Github and issuing a new API token for accessing public repositories. If you have a GitHub account, you can create a token of your own. Be sure to copy the token from the results page. Once you close that page, you won't be able to retrieve the token in plain text again.

Blog3-5.png

Blog3-6.png

Blog3-7.png

Blog3-8.png

We'll pass our newly created token to Github with each request to authenticate that it came from us, so let's save it in a variable. We'll also calculate the date for 90 days ago and save it in the time format required by the API:

$token = Read-Host -AsSecureString "Paste in GitHub Token"
$since = (Get-Date -Date (Get-Date).addDays(-90) -Format "yyyyMMddTHHmmssZ" -Hour 0 -Minute 0 -Second 0 -Millisecond 0)

Blog3-9.png

We'll need to build a PSCredential object to pass our authentication token using HTTP basic authentication. It will be passed using the Credential parameter of Invoke-RestMethod.

$githubUsername = 'Sec557-Demo'
$cred=New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $githubUsername, $token

Next, we'll build a WHILE loop to keep retrieving pages of data until we have retrieved all the issues closed after the date we specified. We'll add all the issues to a variable so we can do the same math we did above on this larger dataset.

$issues = $null
#Counter variables
$page = -1
$count = 100

#Loop until you receive < 100 results on the page
while ( $count -eq 100)
{

<p>$page++
    "Processing page: $page"

    #Set the URL for the request, plugging in $page as the page number
    $uri = "https://api.github.com/repos/PowerShell/PowerShell/issues?page=$page&per_page=100&state=closed&since=$since"

    #Get the next page and add the contents to the $issues variable
    $nextPage = Invoke-RestMethod -Credential $cred -Uri $uri
    $count = $nextPage.count
    $issues += $nextPage</p>

After retrieving several pages, we received a total of 793 issues.

Blog3-10.png

Finally, we can calculate the MTTR for this larger dataset using the same math as above:

($issues | 
<p>Select-Object @{n='TimeToResolve'; e={(New-TimeSpan -Start $_.created_at -End $_.closed_at).TotalDays} } | 
  Measure-Object -Property TimeToResolve -Average).Average</p>

Blog3-11.png

Again, the authentication and data conversion are handled transparently, allowing us to focus on the data!

Accessing SOAP the Hard Way - PowerShell Core

All the examples we've seen so far will work correctly in PowerShell Core and Windows PowerShell. If you find yourself needing to access a SOAP API, I strongly recommend that you find a Windows computer so you can use Windows PowerShell. The process in PS Core is largely manual. You would need to:

  • Read the WSDL (web service definition language) file to understand the XML schema used for making requests and receiving responses.
  • Build the XML-formatted SOAP request and pass it as the body using Invoke-WebRequest.
  • Save the XML document returned in the response and use a .NET XMLDocument object to load it into a variable.
  • Work with the XMLDocument variable to extract your results.
#Based on the WSDL, create the request to get a list of country names and ISO codes.
$body = '<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:web="http://www.oorsprong.org/websamples.countryinfo">
   <soapenv:Header/>
   <soapenv:Body>
      <web:ListOfCountryNamesByName/>
   </soapenv:Body>
</soapenv:Envelope>
'

#Make the web request and save the result to an XML file
(Invoke-WebRequest -UseBasicParsing -method Post -Body $body `
  -uri 'http://webservices.oorsprong.org/websamples.countryinfo/CountryInfoService.wso' `
  -ContentType "text/xml").content | Out-File -Path countries.xml -Force

#Load the file into an XMLDocument object
$xmlCountries = New-Object System.Xml.XmlDocument
$fn = Resolve-Path(".\countries.xml")
$xmlCountries.Load($fn)

#Drill down in the XML to the actual result and then query it for the 'GB' ISO code
$countryCodes = $xmlCountries.envelope.body.ListOfCountryNamesByNameResponse.ListOfCountryNamesByNameResult.tCountryCodeAndName | Select-Object sISOCode, sName

$countryCodes | Where-Object sISOCode -eq 'GB'

Blog3-12.png

While this is not a complicated problem, it does require a number of steps and a fait bit of research.

Accessing SOAP the Easy Way - Windows PowerShell

The simple way to access SOAP is by using the New-WebServiceProxy cmdlet. It handles all the setup and data conversion for you. Simply point the proxy at the WSDL file and then use the auto-generated object to access the API. To demonstrate this, we'll use a free SOAP API which returns country information like mappings of ISO codes to country names.

$wsp=New-WebServiceProxy -Uri http://webservices.oorsprong.org/websamples.countryinfo/CountryInfoService.wso?wsdl
$wsp | Get-Member -Type Method

Blog3-13.png

One of the methods returned by the API is called ListOfCountryNamesByName. It returns a list of countries and their ISO codes. To call this function, we'll just use the variable created above.

$countryCodes = $wsp.ListOfCountryNamesByName()
$countryCodes | Where-Object sISOCode -eq 'GB'

That's it! Five lines of PowerShell yielded the same results as you achieved in the previous section, with no real analysis required! In the rare event that you have to interact with a SOAP-based API, I recommend that you use Windows PowerShell and the New-WebServiceProxy cmdlet whenever possible.

Share:
TwitterLinkedInFacebook
Copy url Url was copied to clipboard
Subscribe to SANS Newsletters
Receive curated news, vulnerabilities, & security awareness tips
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Saudi Arabia
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe

By providing this information, you agree to the processing of your personal data by SANS as described in our Privacy Policy.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Tags:
  • Cloud Security
  • Security Management, Legal, and Audit

Related Content

Blog
SEC541-340x340.png
Cloud Security
March 20, 2023
Cloud Attacks and Threat Detections
SEC541 is a threat-driven class for the cloud. Learn how to build a detection engineering program for your organization.
ShaunMcCullough_370x370.png
Shaun McCullough
read more
Blog
Blog_-_The_New_Financial_Metric_for_Cybersecurity_-_340x340_Thumb.jpg
Security Management, Legal, and Audit, Cybersecurity Insights
March 17, 2023
The New Financial Metric for Cybersecurity
Elevating your workforce with a profit-centric mindset
brieentel_headshot.jpg
Brie Entel
read more
Blog
_Which_human-focused_cybersecurity_course_is_best_for_me_340x340.jpg
Security Management, Legal, and Audit
March 16, 2023
Which Human-Focused Cybersecurity Course is Best for Me?
Examine the differences between the MGT433: Managing Human Risk course and MGT521: Building a Security-Based Culture course.
370x370_Lance-Spitzner.jpg
Lance Spitzner
read more
  • Register to Learn
  • Courses
  • Certifications
  • Degree Programs
  • Cyber Ranges
  • Job Tools
  • Security Policy Project
  • Posters & Cheat Sheets
  • White Papers
  • Focus Areas
  • Cyber Defense
  • Cloud Security
  • Cybersecurity Leadership
  • Digital Forensics
  • Industrial Control Systems
  • Offensive Operations
Subscribe to SANS Newsletters
Receive curated news, vulnerabilities, & security awareness tips
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Saudi Arabia
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe

By providing this information, you agree to the processing of your personal data by SANS as described in our Privacy Policy.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
  • © 2023 SANS™ Institute
  • Privacy Policy
  • Contact
  • Careers
  • Twitter
  • Facebook
  • Youtube
  • LinkedIn