How can you get performance testing results of native mobile application simply and fast?
OUR Blog
QA Engineer
Margarita Pasechnik
QA Engineer
APPROACH
Sep 01 2016

How can you get performance testing results of native mobile application simply and fast?

As everything, it is easy, we just use NeoLoad in our daily work. Market provides us with a wide range of commercial and free of charge tools for performance, load and stress testing. We choose Neoload due to several important reasons, they are important exactly for performance testing of downloaded mobile applications and it is difficult to find them in other tools.

Advantages of NeoLoad and why we use it in our testing:

  • wide range of settings for tests conduction: from changing active users quantity during test running to changing type of mobile network that smartphone supports;
  • different test scenarios can be executed for several groups in the same time (a lot of virtual user groups with different parameters and different test scenarios can be created, in the same time can be emulated that all these groups use tested application in the same time);
  • this tool can emulate different types of network in use (3G, 3G+, H+, 4G LTE, etc.);
  • it supports a variety of mobile devices (in the last version NeoLoad 5.2.2 there are 11 Android devices, 8 of iOS present, also Blackberry, Symbian and Windows );
  • ability to compare test results of different test executions;
  • Qualitative and comprehensible visualisation results of testing;
  • Supports protocols HTTPS, HTTP and HTTPS proxy, basic web-authentication and cookies, determines data while recording the script and uses recorded data during execution.

Performance tests recording for web application and for native mobile applications mostly has no significant differences, many actions are identical to those of web application testing. Additional tools make recording of test cases simple and bring them to a similar format. Steps of tests execution are the same: tests recording, setting and tests execution, get results and make analysis. The difference lies only in the availability of two instruments: the first with the installed application under test, and the second one - with the program to execute testing.

Another feature is additional settings for recording actions on mobile phone via proxi:

  • cell phone should be connected to the same wi-fi network as the device with tool that will record test cases;
  • setting mobile network of device via proxy.

After this actions and installation of an application for checking its performance on a cell phone we start to record scenarios. At the main page of NeoLoad find ‘Start Recording button’, press on it, pop-up window ‘ Start Recording’ opens, in our case we should select ‘Identify as’ checkbox and choose one of proposed devices from drop-box. You can choose one of the browsers, if you test web application via cell phone or choose device, if you test native mobile application.

Start recording

List of proposed mobile devices is extended, here the most frequently used devices with Android and iOS operating systems are presented.

Choose device for testing

After choosing a device for test recording, scenarios of native mobile application start to take actions on a tested application. Take actions that have been premeditated before, execute test plan step by step. When you finish last action in application, press ‘Stop’ button on the main page of NeoLoad. Notice that NeoLoad has ‘Pause’ function, pressing on it you can correct your scenario before continue recording. When record is ready, it is saved in the following way: Design button > User Paths tab> User Paths section> Action folder.

Create test scenario

You have an ability to sort your actions by named folders during recording scenarios, just write folder name to ‘Container’ at the NeoLoad recording panel. After you finish recording and saving test plan, you can add parameters of virtual user groups for tests running. All settings are available in the Runtime tab, you can create several groups of users with different parameters, choose set tests running duration: it can depend on quantity of iteration, duration for a certain time or you can set time in hours, minutes or even seconds. For each virtual user group quantity of Simulated users, quantity of iterations can be chosen as well as in other tools for testing performance. NeoLoad proposes to choose behavior for each of groups: whether it will be permanent or increasing load, one or several peaks of iterations, in which time of test execution they would be carried out. We use NeoLoad because it provides a wide range of settings parameters.

Сhoose behavior for each group of users

When scenario is recorded, set all settings of virtual user groups and tests can be run, you just should press ‘Run’ button from menu on the top of the page. Results of testing will be displayed in ‘Result’ page in real time and automatically sorted by such groups as Test Summary, Values, Graphs, Errors, Alerts, etc. Summary tests execution can be saved for future exploitation in . nlp format file, than it can be changed, edited, corrected, you can add additional test cases and scenarios to your saved Test plan.

NeoLoad displays results via graphics in detail under graphic description, in our case average response time for sent requests is 4.78 sec and maximum near 15 sec, this indicators displayed when 50 active users try to login to application simultaneously. All charts can be found at the right side of the Test Summary page in the Main graphs section.

Find summary report of testing

When we analyze results of application’s work, we always make comparative analysis of tests that were run, for example we should compare tests that were run on different platforms, tests before and after defects fixing, after adding solution that make our application faster for requests. In Graphs section you can choose graphs for matching and comparing, add or delete some charts. In most cases we use Response times charts and charts from ‘Top 3 pages folder statistics, this graphs show performance testing more clearly, because we are interested in speed of responses and number of errors, places where they were found.

Chart of average page response from server

Detailed description of errors for each request we see in the Error tab, there you can find request and response from server. All data about time of responses can be download to . scv file format from Value tab and can be used for creation of new charts and to make additional analysis of requests with errors for Summary report.

Choose charts for result displaying

All data of NeoLoad report can be exported in . csv, .doc, .pdf, .html file formats or export as image, if it is graph. This representation gives ability to use results in the Summary Quality Assurance report.

Detailed summary report

NeoLoad is a very useful tool for performance testing, Steelkiwi quality assurance team used it for testing of native mobile application, due to easy connection to cell phones, simple tests scenarios recording, due to the ability to set a lot variants users behavior during test execution, ability to compare results of tests in different conditions. Using this tool we can make performance, load and stress testing, get full and clear report. Recording and test execution does not take more than 2 hours. As a result, using additional utilities you can get information about application stability work, take the necessary measures to identify heck bottles, deficiencies and ensure the desired performance.

If you have missed two articles about Performance Testing for web applications check it here:

  1. Graphical representation for performance testing
  2. How QA experts use Apache JMeter in daily work
SIMILAR POSTS
Nov 23 2016
All necessary features that you should know before developing your own music app. Statistics, estimated cost and our hands-on experience.
Nov 11 2016
Consider the benefits of developing and alternative methods of using driver app. We will count how much it will cost to create an app like Uber or Lyft, analyze some statistics.
Nov 01 2016
Planned well, developed well. Today we are going to talk about the project management triangle and how to plan the project's life cycle effectively. Also you can take a look at how it all works on practice in our team, where our PM and developers collaborate on planning stage using Scrum methodology.