Performance Testing on Smart phone

We often hear performance testing in our software testing field. This was very confusing to me, but now it is a very interesting subject to me. Performance testing does not mean to find any bugs or issue at all instead of it checks software program's

• Speed - Determines whether the application responds quickly
• Scalability - Determines maximum user load the software application can handle.
• Stability - Determines if the application is stable under varying loads

In today’s tech world the technology is improving day by day, as a result, most of us are using high performing smartphone that is capable of doing nearly everything that a computer can do. So here I’m going to discuss how to do performance testing of a mobile based application.

Mostly two types of mobile based application can be seen

• Client-Server App - An application  that communicates with server
• Stand Alone App - An application that doesn’t communicate with server

The performance of client-server app depends on the following parameter:

• CPU usage
• Memory usage
• Power consumption / Battery usage
• Server performance

And for Stand alone based app’s performance also depends on the same parameter except the server performance. Now today’s high-tech world, availability of CPU & Memory of any mobile is huge so we wouldn’t worry about that two parameters. So our main focus will be power consumption & Server performance. Now let’s see how to do that:

Power consumption / Battery usage: First check whether the targeted app is showing under battery usage section or not. If yes, then it is OK otherwise use GSam Battery Monitor like app to monitor your targeted app’s battery consumption. Now reset the battery usage and switch off the phone and charge it fully. Now use it as you normally use your mobile until you need recharge your mobile and check your targeted app’s battery consumption. Repeat this two or three times and you will get your answer.

Server Performance: To check server performance we need to create some virtual users that will simulate the request as if someone using the app and for that we need to know the request format / type. What kind of request is getting generated by app can be known only by recording that request and it can be easily done JMeter recording. Following are the steps to configure it:

• Open a Test plan & Add a Thread group
• Right click on the “Thread Group” and add a recording controller: Add -> Logic Controller -> Recording Controller
• Next, select WorkBench and right click on it and add the recorder: Add -> Non-Test Elements -> HTTP(S) Test Script Recorder
• Configure HTTP(S) Test Script Recorder: Under Test plan content select the Target Controller as Test Plan -> Thread Group -> Recording Controller
• Click on Start Button on HTTP(S) Test Script Recorder page. That will generate a Root CA Certificate in your JMeter’s bin folder.
• Now install this certificate to your targeted devices and configure it’s proxy server to your system’s IP address. Configuration step to mobile:
• Send certificate to mobile and touch it, will be installed automatically
• Go to device’s settings -> go to WIFI section -> modify the WIFI(either by long pressing to that WIFI or from menu option) -> checked the advance settings -> select proxy settings to MANUAL -> set host / IP address to your system’s IP address (for example: 192.168.0.212) -> Set your port 8080 (Can be changed from Global settings -> Port field in JMeter script recorder page) -> Save it.

Note: Your system and you mobile device should be into same LAN otherwise it will not work

• Now open the targeted App in your mobile, JMeter will record your activity. You can check it live.

After recording is over filter it smartly and run it. Now add listener to your script to analyze the server performance in terms of throughput (Request processed by server) Response time.

Any comments, suggestions will be appreciated. Feel free to contact anytime. Thank you for your patience.

Possible way to crash an App

We can say today is mobile or mobile app generation because each and everyone is using the smart phone to their daily routine so as a Tester / QA it is our responsibility to deliver a good quality product. We need to find out as much as possible major issues, and the crashing issue is one of them as it reflects a very bad impact on the end user. So to make this task easier I found some scenario in which major crashing issue may found:

1. Open your App in low memory system which already opened multiple App
2. Change the orientation multiple times, exit the application. Repeat the process again and again to discover any hidden memory leaks.
3. Change the orientation several times when fetching data from the database.
4. Use a memory profiler to found Memory leakage related issue.
5. Click a button that retrieved data from the database at the time of no internet connection.
6. Check whether the application screen is loading continuously without any data and network failure message, when there is network connection problem after the data call.
7. Check whether the blank screen is exposed or not when the user trying to retrieve the data with different filter options continuously.
8. Go into your app with wifi connected and then turn off the wifi in various ways:
1. Normally on / off the wifi
2. Walking out of range
3. Backgrounding the app and manually turn off it, then foregrounding the app.
9. Alternate between Data connection & Wifi connection when the application is running.
10. Run the application when Wi-Fi / Data connection is on but no internet connection.
11. Use Application in 2G like slow internet connection.
12. Turn off internet connection in between API call
13. Check whether continuous tapping on create / save button results in creating duplicate records.
14. Check whether continuously tapping on Load More button leads to application crash.
15. Try to perform multiple operations for different data calls at the same time.
16. Application opens for a long time without any course of action and after a certain, it came into action.
17. Click Back button while touching to other menu / Submenu / Tab / Slider.
18. Scrolling the list items up and down rigorously for a while.
19. Check the application response when tapping on the list items multiple times while the scrolling is in progress.
20. Check the application response, when tapping on two or more action buttons simultaneously at the time internet connection on / off.
21. Proper validation check throughout the application specifically input section. Without any proper validation, the application may throw some exception and hence app may crash.
22. Use the application when battery is pretty low
23. Check the App’s compatibility with OS version i.e check it with different OS version

Hope this post will help you to find out more and more crashing issue in App testing. If you have any query please let me know via Comment, I will try to help you out. You can also send me an email at sandip.08888@gmail.com. Thank you.

Manual Testing Vs Test Automation

Nowadays, in our software testing field we often heard about Manual Testing, Test Automation. Yes, you are right, test that can be done manually is Manual Testing and test that can be done automatically using some script is Test Automation. Now demand for test automation is getting increased day by day, is that means Test Automation is going to take place of manual testing or in other words, era of manual testing is coming to an end?

In a single word the answer is NO. Let’s try to understand it. Manual Testing & Test Automation, these are two different types of testing methodology and have different perspective of use. None of them can take other’s place in terms of usability. They are complemented to each other. Following are the pros for Manual Testing:

• Manual Testing includes everything like Design check, User Experience Check, Functionality check, Regression check, Data Flow Check, Business Logic Check, Performance check and to give a proper bug report to the team. On the other hand. Test Automation includes Regression check, Functionality Check, Business Logic check, Server Load Calculation etc. We can conclude it very easily that what Test Automation is done, can be done by Manually too.
• There is some constraint to develop Test Automation like
• Tester should have the knowledge of programming language
• Tested environment should have stable to some extend
• Automation itself needs time to be developed.  
• And obviously the cost

On the other hand, Manual Testing need no programming skill, no waiting time for stable environment, Manual Test can be done at the time of development simultaneously with less expenses.

• Before develop any Test Automation need proper test cases to be written and for that we need proper knowledge about the system and it can be achieved only by exploring the system manually so Test Automation is dependent to Manual Testing but Manual Testing has no such dependency completely independent.
• Whenever any tester test any system / module repeatedly there is high probability to check the system from different angles with some new test cases / new idea and so more hidden issue can be tracked easily which is not feasible for Test Automation with respect to time and resources.

There are some cons for Manual Testing too which are as follow:

• Test Automation is useful only for long running system. Suppose you have a quite stable or running system and some change has been made then it need to be checked thoroughly otherwise some regression related issue may arise. It is time consuming for manual tester and if you have a test automation script then it saves a lot of time as well as the resources.
• We need to check server performance and its capacity before it goes for live otherwise it may crash during live performance and for that purpose we completely rely on Automation tools like JMeter (Click here to know more), QTP, Load Runner etc. What exactly we do here is that we create numbers of virtual user and hit to the server nearly at the same time to check its performance as well as its capacity. Bit it is nearly impossible to hit the server from such number of user nearly simultaneously.

Manual Testing will be as it is and so the Test Automation. Hope doubt is cleared now. If you have any query please let me know via Comments. Personally you can also reach me via email sandip.08888@gmail.com. Happy to help you. Thank you for your valuable time.

JMeter Overview

JMeter is an open source load / performance testing tool. It generates virtual user in the form of thread and hit the target server and record the response accordingly. Depending upon the recorded response we concluded the server performance in the form of response time, throughput etc. I'm trying to give an overview from the scratch:

1. Download latest version JMeter according to your O/S from JMeter official site. http://jmeter.apache.org/download_jmeter.cgi
2. Unzip it & go to go to bin directory.
3. For windows open jmeter.bat file, for Linux open jmeter.sh file either from CUI or GUI.
4. Now either record the script by using http(s) script recorder & proxy server or write the script manually
• Scrip record by using http(s) script recorder & proxy server
1. Open Test Plan & Add a Thread Group
2. Right click on the Thread Group and add a recording controller: Add > Logic Controller > Recording Controller
3. Next, select WorkBench and right click on it and add the recorder: Add -> Non-Test Elements -> HTTP(S) Test Script Recorder
4. Configure HTTP(S) Test Script Recorder: Under Test plan content select the Target Controller as Test Plan > Thread Group > Recording Controller
5. Click on Start Button on HTTP(S) Test Script Recorder page. That will generate a Root CA Certificate in your JMeter’s bin directory.
6. Import this Root CA Certificate to required browser and also set browser’s proxy server to localhost with port 8080 ( Can be changed from Global settings -> Port field in script recorder page )
7. Now open the browser and navigate it. JMeter will record accordingly. You should filter it smartly to run it further.
• Manually script writing
1. Open Test Plan and Add Thread Group under test plan. Here number of thread represent the total number of virtual user. Here ramps-up time represent the total time in which all the virtual user will hit the server. For example, thread group: 1000 & ramps-up: 100 i.e all 1000 user will hit the server within 100 seconds. Loop count represent how many times this thread group will continue.
2. Add Sampler -> HTTP Request under thread group. If we use any URL repeatedly then we can add Config Element -> HTTP Request Defaults with base URL.
3. We can use HTTP Cookie Manager / HTTP Cache Manager / HTTP Authentication Manager from Config Element option to maintain cookie / cache / authentication respectively.
• Add listener accordingly
1. View Results Tree : Capture all request & Response in details
2. View Results in Table: capture some specific info and shows in tabular form.
3. Summary Report: Shows an overall summary report.
4. Graph Generator : install it using JMeter plugin ( Click Here ) and Configure it to use.
5. Run the script either from graphically or by using command.
• Initially run script graphically for single user to check script and then run script for load test according to required number of user. At the of running the script you may have face memory related issue and to avoid such issue & faster execution, you should set up Distributed Environment ( Link Here ) and also run it using command.
6. Analyze the result from listener & make full report.