Optimizing End-to-End test execution: Unleashing the Resource Dispatcher - WiP

Authors

  • Cristian Augusto Computer Science Department, University of Oviedo
  • Jesús Morán Computer Science Department, University of Oviedo
  • Claudio de la Riva Computer Science Department, University of Oviedo
  • Javier Tuya Computer Science Department, University of Oviedo

Keywords:

End-to-End Testing, E2E Testing, Test Suite Optimization, System Testing, Testing Resources, Test Optimization, Testing

Abstract

Continuous integration practices have transformed software development, but executing test suites of modern software developments addresses new challenges due to its complexity and its huge number of test cases. Certain test levels, like End-to-end testing, are even more challenging due to long execution times and resource-intensive requirements, moreover when we have many End-to-end test suites. Those E2E test suites are executed sequentially and in parallel over the same infrastructure and can be executed several times (e.g., due to some tester consecutive contributions, or version changes performed by automation engines). In previous works, we presented a framework that optimizes E2E test execution by characterizing Resources and grouping/scheduling test cases, based on their compatible usage. However, the approach only optimizes a single test suite execution and neglects other executions or test suites that can share Resources and lead to savings in terms of time and number of Resource redeployments. In this work, we present a new Resource allocation strategy, materialized through a Resource Dispatcher entity. The Resource Dispatcher centralizes the Resource management and allocates the test Resources to the different test suites executed in the continuous integration system, according to their compatible usage. Our approach seeks efficient Resource sharing among test cases, test suites, and suite executions, reducing the need for Resource redeployments and improving the execution time. We have conducted a proof of concept, based on real-world continuous integration data, that shows savings in both Resource redeployments and execution time.

References

M. Meyer, “Continuous integration and its tools,” IEEE Softw., vol. 31, no. 3, pp. 14–16, 2014, doi: 10.1109/MS.2014.58.

H. Esfahani et al., “CloudBuild: Microsoft’s distributed and caching build service,” in Proceedings - International Conference on Software Engineering, in {ICSE} ’16. ACM, 2016, pp. 11–20. doi: 10.1145/2889160.2889222.

A. Memon et al., “Taming google-scale continuous testing,” in Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP 2017, IEEE, May 2017, pp. 233–242. doi: 10.1109/ICSE-SEIP.2017.16.

C. Augusto, J. Morán, A. Bertolino, C. de la Riva, and J. Tuya, “RETORCH: an approach for resource-aware orchestration of end-to-end test cases,” Softw. Qual. J., vol. 28, no. 3, pp. 1147–1171, Sep. 2020, doi: 10.1007/s11219-020-09505-2.

S. Yoo and M. Harman, “Regression testing minimization, selection and prioritization: A survey,” Software Testing Verification and Reliability, vol. 22, no. 2. John Wiley and Sons Ltd., pp. 67–120, Mar. 2012. doi: 10.1002/stv.430.

G. Rothermel, M. J. Harrold, J. Ostrin, and C. Hong, “Empirical study of the effects of minimization on the fault detection capabilities of test suites,” in Conference on Software Maintenance, 1998, pp. 34–43. doi: 10.1109/icsm.1998.738487.

W. E. Wong, J. R. Horgan, A. P. Mathur, and A. Pasquini, “Test set size minimization and fault detection effectiveness: A case study in a space application,” J. Syst. Softw., vol. 48, no. 2, pp. 79–89, 1999, doi: 10.1016/S0164-1212(99)00048-5.

A. Eivy, “Be Wary of the Economics of ‘Serverless’ Cloud Computing,” IEEE Cloud Comput., vol. 4, no. 2, pp. 6–12, 2017, doi: 10.1109/MCC.2017.32.

C. Augusto, J. Morán, C. de la Riva, and J. Tuya, “FullTeaching E2E Test Suite.” 2023. [Online]. Available: https://github.com/giis-uniovi/retorch-st-fullteaching

B. Garcia et al., “A proposal to orchestrate test cases,” in Proceedings - 2018 International Conference on the Quality of Information and Communications Technology, QUATIC 2018, 2018, pp. 38–46. doi: 10.1109/QUATIC.2018.00016.

ElasTest EU Project, “Fullteaching: A web application to make teaching online easy.” Universidad Rey Juan Carlos, 2017. Accessed: Aug. 10, 2023. [Online]. Available: https://github.com/pabloFuente/full-teaching

Downloads

Published

2024-08-20

How to Cite

Augusto, C., Morán, J., de la Riva, C., & Tuya, J. (2024). Optimizing End-to-End test execution: Unleashing the Resource Dispatcher - WiP. WiPiEC Journal - Works in Progress in Embedded Computing Journal, 10(2). Retrieved from https://wipiec.digitalheritage.me/index.php/wipiecjournal/article/view/71

Issue

Vol. 10 No. 2 (2024): WiPiEC Journal

Section

Articles

License

Copyright (c) 2024 Cristian Augusto, Jesús Morán, Claudio de la Riva, Javier Tuya

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

License Terms: 

Except where otherwise noted, content on this website is lincesed under a  Creative Commons Attribution Non-Commercial License (CC BY NC) 

Use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes, is permitted.

Copyright to any article published by WiPiEC retained by the author(s). Authors grant WiPiEC Journal a license to publish the article and identify itself as the original publisher. Authors also grant any third party the right to use the article freely as long as it is not used for commercial purposes and its original authors, citation details, and publisher are identified, in accordance with CC BY NC license. Fore more information on license terms, click here.