We would be going through the steps to automate the CICD pipeline, reporting and integrating Postman with Azure DevOps for Automation.

Postman Introduction:

Postman has better GUI and features. It’s a powerful tool when developing automated tests, it helps visualize details of each iteration and test results but does not allow the desktop or Mobile testing.

Postman is a tool for API development. Postman’s features simplify each step of building an API and testing it. It offers the possibility to automate manual tests and integrate them into your CI/CD pipeline to ensure that any code changes won’t introduce regressions in Production.

Requests in Postman are organised in collections and folders. Postman contains a collection runner that is useful when you want to automate API testing. When you run a collection, you send all requests in your collection one after another.

Limitation of Postman:

Now postman is not given any option to associate postman test case to DevOps test case. occasional crashes linked to memory issues. To remediate those issues, we are going to use the CLI tool of Postman called Newman and a custom reporter that generates detailed reports. Read more about Newman here (https://www.npmjs.com/package/newman)

Prerequisites:- (If don’t have, So please follow below point to how to install this)

Installed Postman API tool
Node.js with the latest version (v12.16.3) below also working.
NPM it’s also built with node js if not so that you need to install the latest version (6.14.4).
Newman with the latest version (5.0.0).
Azure DevOps Service Account with the required permission.

Below points, we will cover step by step…

How to install and run Newman on command prompt
How to configure postman API tests to Azure DevOps or TFS using Newman?
How to publish HTML Results in Azure DevOps or TFS?

1. NOW START STEP BY STEP TO RUN POSTMAN COLLECTION ON COMMAND PROMPT

Step 1: Install Node (You may skip this step if you already have Node installed)

Follow the steps to download Node which is specific to your CI’s platform.

Step 2: Install Newman (You may skip this step if you already have Newman installed)

Newman is a command-line tool that allows you to run a collection in your local development environment or on your own server. The following command installs Newman.

npm install -g newman

Reporter installing command

npm install -g newman-reporter-htmlextra

Step 3: Export postman Tests Collection In your postman tool Test Collection (looks like a folder) on the Left Side you see 3 dots (Point 1) Click on that, from the menu choose to export (point 2), And saved file with .json on your local machine folder, And for the environment & Global variables also download with .json in the same folder (See snapshot)

Step 4: Run Newman

Run the following Newman command with the appropriate parameters:

{Your folder path}newman run {Collection.json} -e globals.json –reporters cli,junit –reporter-junit-export Results\junitReport.xml

If you need to provide an environment to the collection, change the above command to the following:

{Your folder path}newman run {Collection.json} -e environment.json –reporters cli,junit –reporter-junit-export Results\junitReport.xml

2. HOW TO CONFIGURE POSTMAN API TESTS TO AZURE DEVOPS OR TFS USING NEWMAN?

Before running the pipeline you have to push your collection files on repository

Now in this blog, we are going to push code on azure repository below option repos.

After pushing the code, click on Pipeline >> New Pipeline

Next, Choose to Use the classic editor

Choose your Repository & Continue

From the template Choose an empty job

Name your pipeline and Choose Agent

i.Ensure your Get Sources pointing to correct repository
Click on the Get Sources modify the following
Repository: Should be pointing to correct repository where your postman test collection is located
Default branch: Choose the correct branch (typically this will be master branch)
Clean: Set the clean option to true (default is false, modify from the dropdown)
Clean Options: Set clean option to Source

j. Modify Agent job 1 property
Display Name: Any name which you like (ex: Postman API Automation)
Agent pool: Choose the correct Agent pool from the drop-down

Adding build tasks to Azure DevOps / TFS build pipeline

A. Add Command Line Steps to install Newman in your build agent
i. Click on + button in your agent panel (Refer Image)
ii. Search for task Command Line (Refer Image)
iii. Click on Add

B. Configure Command Line task to install new man
Display name: Mention task name as Install Newman
Script: npm install -g Newman
Click on Advanced section and choose Working directory
Working Directory : $(System.DefaultWorkingDirectory)
Note: $(System.DefaultWorkingDirectory) is inbuilt Azure DevOps variable which will point to your source code (Know more about Azure DevOps Variable)

C. Add task to run Postman tests in Newman
i. Add one more command-line task as discussed above
ii. Modify newly added second command-line task
Display name: Newman-Postman
Script :Newman run {your Collection} -e environment.json –reporters cli,junit –reporter-junit-export Results\junitReport.xml
Working Directory: Choose Correct Directory where your tests are located in git repository and folder (You can optionally use the option “…” (3 dots right to this textbox manually search the folder)
Control Options: Continue on Error (Check the Checkbox)

Note: Few lines about Script, the command we entered in the script is to run the postman tests in Newman tool
If you are not using the environment variable simply use like below
newman run <your collection> –reporters cli,junit –reporter-junit-export Results\junitReport.xml
If you are using environment variable you need to mention the environment variable json file which you exported at the beginning use like below
newman run <yourcollection> -e <your_environmentfile> –reporters cli,junit –reporter-junit-export Results\junitReport.xml

What happens after the execution of this step?
1. Executes All the postman tests
2. Creates a directory called Results in the Working Folder which you mentioned in option
3. Creates junitReport.xml Results File inside the Results Folder
In the above command — reporters cli,junit — reporter-junit-export Results\junitReport.xml This line is responsible for generating results.
Results are folder name and junitReport.xml is the file name (Optionally you can modify it to any name).
— reporters cli,junit With this line we are specifying output as junit and command line interface both. because we want to see the result in both command line and html format.

3. How to publish HTML Results in Azure DevOps or TFS?
Add a task to Publish the Result

Click on The + button on the Agent panel
Search for the Task Publish Test Results and Click on ADD

Modify Publish Result Task
Display name: Publish Test Results
Test Result Format: Junit
Test Result Files : $(System.DefaultWorkingDirectory)\Results\*.xml
(In your case it might be different this should point to the file where your junitReport.xml file is located in the previous step.)
Upload test results files: Check the Checkbox
Please note that you will see that “unable to find the filename after execution if this path is wrong”, so please always do a double check on this.

Finally, click on Save & Queue so the build runs now.

Verifying the results after execution
If you have configured everything properly, as expected all the tasks should pass and will look like below