Automating Test Case Generation for Android Applications using Model-based Testing
Abstract
Testing of mobile applications (apps) has its quirks as numerous events are required to be tested. Mobile apps testing, being an evolving domain, carries certain challenges that should be accounted for in the overall testing process. Since smartphone apps are moderate in size so we consider that model-based testing (MBT) using state machines and statecharts could be a promising option for ensuring maximum coverage and completeness of test cases. Using model-based testing approach, we can automate the tedious phase of test case generation, which not only saves time of the overall testing process but also minimizes defects and ensures maximum test case coverage and completeness. In this paper, we explore and model the most critical modules of the mobile app for generating test cases to ascertain the efficiency and impact of using model-based testing. Test cases for the targeted model of the application under test were generated on a real device. The experimental results indicate that our framework reduced the time required to execute all the generated test cases by 50%. Experimental setup and results are reported herein.
Downloads
Author Biographies
MUHAMMAD NAEEM AHMED KHAN received the D.Phil. degree in computer system engineering from the University of Sussex, U.K. His research interests include software engineering, cyber administration, digital forensic analysis, and machine learning techniques.
AAMIR MEHMOOD MIRZA received the M.Sc. degree in computer science from SZABIST, Islamabad, Pakistan, in 2006, the M.S. degree in computer science and engineering from Halmstad University, Halmstad, Sweden, in 2009, and the Ph.D. degree in computer science from SZABIST. He is currently associated with the Balochistan University of Information Technology, Engineering and Management Sciences.
MUHAMMAD AKRAM received the B.S. degree in computer engineering from the Balochistan University of Information Technology, Engineering and Management Sciences (BUITEMS), Quetta, Pakistan, the M.S. degree in software engineering from Hamdard University, Pakistan, and the Ph.D. degree in computer engineering from Sungkyunkwan University, South Korea, in 2018. He is currently an Assistant Professor with the Department of Software Engineering, BUITEMS. His research interests include wireless sensor networks, machine learning,
neural networks, and context-aware computing.
Shariqa Fakhar, she is serving as Lecturer at Computer Science Department of SBKWU Pishin, Pakistan. She received her bachelor's degree in Information Technology in 2013 from BUITEMS Quetta, Pakistan. Furthermore, she has completed her Masters of Computer Science from SBKWU Quetta, Pakistan. She is enrolled in PhD in University of Balochistan, Pakistan. Her research areas are information retrieval, image retrieval, emotion recognition using deep learning, internet of things(IoT) and Wireless Sensor Networks.
Shumaila Hussain, she is currently serving as Assistant Professor at department of CS, SBKWU, Pakistan. She received her bachelor’s degree in computer science in 2009 from SBKWU, Pakistan. Furthermore she has completed her Masters Degree in Computer Science from BUITEMS, Pakistan. She was awarded HEC Scholarship for PhD. She is enrolled in PhD in UoB, Pakistan. Her research areas are Human Computer Interaction, Wireless Sensor Networks, Software Security, Continious Integration and Continious Development, and Software self-healing.
Irfan Ahmed Magsi. He is currently serving as a Lecturer at Department of Information Technology, FICT, BUITEMS, Pakistan. He received his bachelor’s degree Bachelor’s of Science in Information Technology (BSIT) in 2009 from Quaid-E-Awam University of Engineering Science and Technology, Pakistan. Furthermore he received in his Masters of Science Computer Science (MSCS) from Mohammad Ali Jinnah University Karachi. His research Interests concentrate on Internet of Things, wireless sensor networks and computer networks.
RAJA ASIF WAGAN received the bachelor's degree in computer science from the University of Sindh, Pakistan, in 2005, the master's degree in information technology from the Universiti Utara Malaysia, and the Ph.D. degree in information and communication engineering from Harbin Engineering University, Harbin, China. He is currently working as an Assistant Professor with the Department of Information Technology, Balochistan University of Information Technology, Engineering and Management Sciences (BUITEMS), Pakistan. His research interests include routing protocols, wireless sensor networks, and computer networks.
References
D. Amalfitano, N. Amatucci, A. M. Memon, P. Tramontana, and A. R. Fasolino, A general framework for comparing automatic testing techniques of Android mobile apps, J. Syst. Softw., vol. 125, pp. 322–343, 2017. DOI: https://doi.org/10.1016/j.jss.2016.12.017
F. Tong and Z. Yan, A hybrid approach of mobile malware detection in Android, J. Parallel Distrib. Comput., vol. 103, pp. 22–31, 2017. DOI: https://doi.org/10.1016/j.jpdc.2016.10.012
Y.-D. Lin, J. F. Rojas, E. T.-H. Chu, and Y.-C. Lai, On the accuracy, efficiency, and reusability of automated test oracles for android devices, IEEE Trans. Softw. Eng., vol. 40, no. 10, pp. 957–970, 2014. DOI: https://doi.org/10.1109/TSE.2014.2331982
R. Abdul, An Improved Model for Model based Software Testing, Int. J. Comput. Sci. Netw. Secur., vol. 17, no. 1, pp. 151–154, 2017.
A. M. Mirza and M. N. A. Khan, An automated functional testing framework for context-aware applications, IEEE Access, vol. 6, pp. 46568–46583, 2018. DOI: https://doi.org/10.1109/ACCESS.2018.2865213
M. Aamir Mehmood, K. Muhammad Naeem Ahmed, and I. Saleem, Identifying Test Complexity Metrics for Multicore, Cloud, Mobile, Ubiquitous and Context Aware Computing, Int. J. Comput. Sci. Inf. Secur., vol. 14, no. 10, 2016.
R. Swain, V. Panthi, P. K. Behera, and D. P. Mohapatra, Automatic test case generation from UML state chart diagram, Int. J. Comput. Appl., vol. 42, no. 7, pp. 26–36, 2012. DOI: https://doi.org/10.5120/5705-7756
J. Lee and H. Kim, QDroid: Mobile Application Quality Analyzer for App Market Curators, Mob. Inf. Syst., vol. 2016, 2016. DOI: https://doi.org/10.1155/2016/1740129
D. Amalfitano, A. R. Fasolino, P. Tramontana, B. D. Ta, and A. M. Memon, MobiGUITAR: Automated model-based testing of mobile apps, IEEE Softw., vol. 32, no. 5, pp. 53–59, 2014. DOI: https://doi.org/10.1109/MS.2014.55
K. Mao, M. Harman, and Y. Jia, Robotic testing of mobile apps for truly black-box automation, IEEE Softw., vol. 34, no. 2, pp. 11–16, 2017. DOI: https://doi.org/10.1109/MS.2017.49
A. R. Espada, M. del M. Gallardo, A. Salmerón, and P. Merino, Using model checking to generate test cases for android applications, ArXiv Prepr. ArXiv150402440, 2015. DOI: https://doi.org/10.4204/EPTCS.180.1
M. A. Mehmood, M. Khan, and W. Afzal, Automating Test Data Generation for Testing Context-Aware Applications, in IEEE 9th International Conference on Software Engineering and Service Science (ICSESS), 2018, pp. 104–108. DOI: https://doi.org/10.1109/ICSESS.2018.8663920
M. Aamir Mehmood, M. N. A. Khan, and W. Afzal, Transforming context-aware application development model into a testing model, in 8th IEEE International Conference on Software Engineering and Service Science (ICSESS), Beijing, 2017, pp. 177–182. DOI: https://doi.org/10.1109/ICSESS.2017.8342891
A. Kaur and V. Vig, Systematic review of automatic test case generation by UML diagrams, Int. J. Eng. Res. Technol., vol. 1, no. 7, 2012.
R. Dev, A. Jääskeläinen, and M. Katara, Model-based GUI testing: Case smartphone camera and messaging development, in Advances in Computers, vol. 85, Elsevier, 2012, pp. 65–122. DOI: https://doi.org/10.1016/B978-0-12-396526-4.00002-3
V. Gudmundsson, M. Lindvall, L. Aceto, J. Bergthorsson, and D. Ganesan, Model-based Testing of Mobile Systems–An Empirical Study on QuizUp Android App, ArXiv Prepr. ArXiv160600503, 2016. DOI: https://doi.org/10.4204/EPTCS.208.2
Y. Jing, G.-J. Ahn, and H. Hu, Model-based conformance testing for android, in International Workshop on Security, 2012, pp. 1–18. DOI: https://doi.org/10.1007/978-3-642-34117-5_1
M. Ahmed, R. Ibrahim, and N. Ibrahim, An Adaptation Model for Android Application Testing with Refactoring, Int. J. Softw. Eng. Its Appl., vol. 9, no. 10, pp. 65–74, 2015. DOI: https://doi.org/10.14257/ijseia.2015.9.10.07
OMG, About the Unified Modeling Language Specification Version 2.0. https://www.omg.org/spec/UML/2.0/About-UML/ (accessed Jun. 16, 2020).
T. Azim and I. Neamtiu, Targeted and depth-first exploration for systematic testing of android apps, in ACM SIGPLAN international conference on Object oriented programming systems languages & applications, 2013, pp. 641–660. DOI: https://doi.org/10.1145/2544173.2509549
G. de Cleva Farto and A. T. Endo, Evaluating the model-based testing approach in the context of mobile applications, Electron. Notes Theor. Comput. Sci., vol. 314, pp. 3–21, 2015. DOI: https://doi.org/10.1016/j.entcs.2015.05.002
J. Gutiérrez, M. Escalona, and M. Mejías, A model-driven approach for functional test case generation, J. Syst. Softw., vol. 109, pp. 214–228, 2015. DOI: https://doi.org/10.1016/j.jss.2015.08.001
B. P. Lamancha, M. Polo, D. Caivano, M. Piattini, and G. Visaggio, Automated generation of test oracles using a model-driven approach, Inf. Softw. Technol., vol. 55, no. 2, pp. 301–319, 2013. DOI: https://doi.org/10.1016/j.infsof.2012.08.009
R. Ramler, G. Buchgeher, and C. Klammer, Adapting automated test generation to GUI testing of industry applications, Inf. Softw. Technol., vol. 93, pp. 248–263, 2018. DOI: https://doi.org/10.1016/j.infsof.2017.07.005
A. R. Espada, M. del M. Gallardo, A. Salmerón, and P. Merino, Performance analysis of Spotify® for Android with model-based testing, Mob. Inf. Syst., vol. 2017, 2017. DOI: https://doi.org/10.1155/2017/2012696
S. Salva and S. R. Zafimiharisoa, APSET, an Android Application Security Testing tool for detecting intent-based vulnerabilities, Int. J. Softw. Tools Technol. Transf., vol. 17, no. 2, pp. 201–221, 2015. DOI: https://doi.org/10.1007/s10009-014-0303-8
C. Tao and J. Gao, On building a cloud-based mobile testing infrastructure service system, J. Syst. Softw., vol. 124, pp. 39–55, 2017. DOI: https://doi.org/10.1016/j.jss.2016.11.016
P. Arcaini and A. Gargantini, Test generation for sequential nets of Abstract State Machines with information passing, Sci. Comput. Program., vol. 94, pp. 93–108, 2014. DOI: https://doi.org/10.1016/j.scico.2014.02.007
W. Yang, M. R. Prasad, and T. Xie, A grey-box approach for automated GUI-model generation of mobile applications, in International Conference on Fundamental Approaches to Software Engineering, 2013, pp. 250–265. DOI: https://doi.org/10.1007/978-3-642-37057-1_19
I. C. Morgado, A. C. Paiva, and J. P. Faria, Automated pattern-based testing of mobile applications, in 9th International Conference on the Quality of Information and Communications Technology, 2014, pp. 294–299. DOI: https://doi.org/10.1109/QUATIC.2014.47
Sami-Ul-Haq, Khan, M. N. A., Mirza, A. M., Saif Ur Rehman, Raja Asif Wagan, & Saleem, I. Addressing Communication, Coordination and Cultural Issues in Global Software Development Projects. EMITTER International Journal of Engineering Technology, 9(1), 13-30, 2021. https://doi.org/10.24003/emitter.v9i1.558 DOI: https://doi.org/10.24003/emitter.v9i1.558
Copyright (c) 2022 EMITTER International Journal of Engineering Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The copyright to this article is transferred to Politeknik Elektronika Negeri Surabaya(PENS) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to PENS. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The copyright transfer form can be downloaded here .
The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.
Retained Rights/Terms and Conditions
- Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
- Authors may reproduce or authorize others to reproduce the work or derivative works for the author’s personal use or company use, provided that the source and the copyright notice of Politeknik Elektronika Negeri Surabaya (PENS) publisher are indicated.
- Authors are allowed to use and reuse their articles under the same CC-BY-NC-SA license as third parties.
- Third-parties are allowed to share and adapt the publication work for all non-commercial purposes and if they remix, transform, or build upon the material, they must distribute under the same license as the original.
Plagiarism Check
To avoid plagiarism activities, the manuscript will be checked twice by the Editorial Board of the EMITTER International Journal of Engineering Technology (EMITTER Journal) using iThenticate Plagiarism Checker and the CrossCheck plagiarism screening service. The similarity score of a manuscript has should be less than 25%. The manuscript that plagiarizes another author’s work or author's own will be rejected by EMITTER Journal.
Authors are expected to comply with EMITTER Journal's plagiarism rules by downloading and signing the plagiarism declaration form here and resubmitting the form, along with the copyright transfer form via online submission.
