Case Studies

  • Home
  • Performance Testing for a SaaS based B2C Application

Performance Testing

Experion has an ‘Independent Verification and Validation Team’ which conducts functional as well as nonfunctional testing such as security testing, compatibility testing, performance testing, etc. for web/ mobile applications. This team was assigned to conduct performance testing to ensure that the application runs smoothly with the provisioned infrastructure. Extensive tests were run on the QR Code scan & redeem functionality and the transaction response was investigated for different scenarios. The test was designed to expose any potential performance flaws with respect to the provisioned infrastructure as well as the application architecture. Performance testing was chosen to bring clarity on the following aspects:

  • Ensure non-functional constraints such as reliability and responsiveness match expectations under real-world conditions
  • Ensure error percentage did not exceed the accepted threshold level
  • Validate that the response time for APIs involved in the scan and redeem functionality did not exceed the accepted threshold
  • Benchmark the capacity of provisioned cloud instance during the load, to help infrastructure managers make smart decisions while scaling up
Approach adopted for testing

After a thorough study and research of business workflow model, Experion testing team arrived at a projected number of service providers and simulated a virtual food festival event scenario. The team studied the flow of the application and identified the steps involved in scanning & redemption of the e-tickets. Based on that, the test was modelled. Team opted to engage a reliable, performance testing Platform-as-a-Service (PaaS) solution, BlazeMeter, to work as the load generator on application servers. Request response time along with server KPIs were measured as part of the performance testing process.

Figure 1: Test Configuration

In the above diagram, the vertical flow shows the spawn of virtual users and the horizontal flow shows the sequence in which APIs are invoked in the test configuration. BlazeMeter was configured to spawn virtual users to match the projected number of service providers, in separate threads which run concurrently/ parallelly. In each thread the APIs under test are invoked in sequential order; Login API followed by Scan API and then Redeem API, which represented the Scan and Redeem work flow. A think time or a delay of 5 seconds between each Scan and Redeem routine was applied to model the flow as close to reality as possible. The test was conducted for 20 min and this was considered as a Run. After every Run a considerable delay (pacing) of almost 10mins was given before starting the next run. Results of best of three runs were considered for analysis.

Test Results

Experion testing team analysed the data collected to benchmark threshold values of the server. The results showed that the infrastructure provisioned for the server is adequate. In addition to that, the load was seen evenly distributed among the application servers behind the load balancer, indicating configuration setup for the load balancer is correct for the requirement. Separate test was done to confirm the configuration setup for autoscaling facility available on the cloud. This gave immense confidence to the client that in case of a peak load during a food festival which caters thousands of food lovers and caterers, the infrastructure and the application will deliver seamlessly.
During the time of performance testing, the application servers are already under sufficient stress. At this time, testing team at Experion was conducting exploratory manual testing to verify the application for any performance flaws. List of mobile devices which could be used by the caterers was shortlisted. The manual testing was conducted on these set of devices or like devices to mitigate risk when devices of multiple models/ makes are used during a festival.

The test results of the performance testing were an assurance to the client to face any festival with a sizeable attendance. The request response time was found to be within the acceptable limit. As part of the engagement, application logs were scrutinized, and database was queried to check integrity. The results were positive.


Create Your Breakthrough Moments With Us