API Testing Services for Performance Testing with JMeter

An application programming interface (API) is an interface between a client software and a server. APIs are intended to simplify the building of client-side software. If the client-side application makes a request in a specific pre-defined format, it will always get a response in a specific format or initiate a defined action. APIs simplify programming by abstracting the underlying implementation and only exposing objects or actions the developer needs.

API testing involves conducting tests to determine if an API meets the functional, reliability, performance, and security requirements of the application in question. API testing is performed in the message layer as it lacks a GUI interface and is one of the challenging aspects of quality testing in a software development project. Until recently, manually testing all the APIs several times was the common method used by QA teams around the world. It is safe to say that this was a laborious process. However, with automated testing tools becoming popular, API testing services have become a lot easier and faster.

Approach for API integration heavy project

From a quality assurance point of view, it’s best to have a “Shift Left” approach towards software and systems testing. Shift Left emphasizes moving testing towards the left of the project timeline, i.e., test early and often. Identifying and rectifying a bug at the beginning of the project would save a lot of time, cost, and effort compared to identifying a bug in the later stages of development from a project development standpoint.

Testing validates the underlying business logic of the software architecture. Automated testing reduces the time taken for testing by a significant margin. However, it is impossible to automate all testing needs of a project. Determining which tests should be automated and which ones to be tested manually is crucial. These are some of the reasons why API testing services should be automated:

Several versions of APIs developed during the development cycle need to be tested. Manual testing of each version can take, on average, 20% of total development time conservatively. For example, if five versions of APIs are created, the total testing time required would be 100% of the total development time.
APIs tend to have stable input and predetermined outputs and, therefore, are ideal candidates for automated testing.
Once the initial script for automated testing of an API is written, the incremental effort required for additional scripting is minimal (less than 5% of the time required for writing the initial testing script).

API Testing

Experion conducted a 2-week discovery phase to study the existing system and establish the business goals of the client. This phase helped Experion understand the underlying drivers for a successful outcome, in addition to the requirements brief shared by the client.

To fix the API request count, Experion introduced a mechanism to queue and persist the commands on the client browser. This is then processed as a batch with the help of a service worker (a background service) and synchronized with the server at fixed intervals. This implementation reduced the request count substantially, which in turn reduced the cost of the infrastructure. We also designed a resilience mechanism to prevent any further data loss in worst-case scenarios.

Since there were a large number of requests for user authentication, the authentication mechanism was implemented as a separate service.
Additionally, an intuitive user interface was designed and developed as part of the solution re-engineering activity, offering ease of usage to all relevant stakeholders from the client’s side.

Let us look at an example that illustrates the best approach to test a project that used API testing services. Experion worked with a client in the healthcare sector as their technology partner to develop a software solution comprising two independent applications:

A browser-based reporting tool
DICOM* router interfacing different PACS*

*DICOM = Digital Imaging and Communications

*PACS = Picture Archiving and Communication Systems

Radiologists will use the browser-based reporting tool to read and interpret digital images from healthcare devices. This tool also helps radiologists record their dictations using a recorder along with their workflow management.

The second product is an interface to the DICOM router that helps it to import, modify, and export images among different picture archiving and communication systems (PACS). The core of the DICOM router application (implemented in Java) is to be interfaced using RESTful API, which uses JSON format for data exchanges. Executing the APIs sets policy and rules in the DICOM router to enable the exchange of DICOM images between PACS situated in different geographical areas.

The Solution

For such a complicated software system that uses a lot of API calls, it is quite natural to implement test automation to determine the expected outcomes. Experion deployed its test engineers at every stage of the API development process. Involvement of Experion’s Independent Verification and Validation Team members from the initial discussion enabled it to identify and analyze the customer’s business requirements and finalize the best testing approach. The team created test strategies for functional and non-functional tests, such as security testing, compatibility testing, performance testing, API testing services, etc.

Experion’s Test engineers devised a two-stage test strategy to meet the challenges of the project. In stage one of testing, individual APIs were validated for mandatory parameters included in the JSON request and response header, and the JSON response body was also validated. In stage two, APIs were called in sequence to simulate an E2E functionality of the final software solution.

For stage one of testing, the team used Postman as the testing tool. Postman is an API development tool that enables users to build, test, and modify APIs. Postman can select the appropriate API call method from a drop-down provided in the tool, set authorization, header, and body information according to the API call.

For the second stage of testing, where the end-to-end functionality is tested, the Tech QA team automated the script to check and validate APIs as part of the CI-CD (Continuous Integration/Continuous Deployment) pipeline. Now when the developer commits a change, the CI-CD pipeline deploys the latest application in the test environment. The E2E test automation scripts start executing to validate the application. The developer gets immediate feedback on their commits/changes after the scripts are executed.

An automated test script was also designed to run in JMeter. The Apache JMeter™ application, a 100% pure Java application designed originally to load test functional behavior and measure performance, is open-source software. Using JMeter to sequence API calls has the added advantage of load testing API with multiple virtual users.

The JMeter test plan was created with user-defined variables and initialized with values consumed in the script. Simple Controllers were used to section APIs, and under each controller, samplers containing HTTP requests along with response assertions are used to validate the response from the server. JSON Extractors were used to extract information from the response body and supply it to a sampler that consumes the same. During the initial stage of script development, results were viewed in Result Tree for debugging and tabular summary reports. In the deployment pipeline, the script was run in a non-GUI mode and results were viewed in JMeter text logs (.JTL) files.

Results of Testing

Even though automation requires higher initial developer time for a given feature, it increases the efficiency of the overall project, particularly in projects that involve integrating multiple APIs. Some of the advantages of using automated testing are:

1. Immediate feedback to developers: A test can be run immediately after a build is committed, helping the developer to fix errors and bugs without having to wait for the testing team to validate the build. This ensures that software integration is seamless.
2. Better Reliability :Automated tests can be executed by the developer at their end. This increases accountability as any error can be identified and resolved immediately, resulting in an error-free final product.
3. Higher Efficiency: Even though automation requires higher initial developer time for a given feature, the time needed for testing and bug fixing is drastically reduced, resulting in shorter overall development time.
4. Higher ROI: As the overall development and QA requirements are lesser for projects that use an automated testing strategy, the overall cost of the project will also reduce, increasing the total ROI for development and even maintenance.