Behavior-driven Load Testing Using Contextual Knowledge — Approach and Experiences

TitleBehavior-driven Load Testing Using Contextual Knowledge — Approach and Experiences
Publication TypeConference Paper
Year of Publication2019
AuthorsSchulz, H., D. Okanović, A. van Hoorn, V. Ferme, and C. Pautasso
Conference Name10th ACM/SPEC International Conference on Performance Engineering (ICPE 2019)
Conference LocationMumbai, India
KeywordsBenchFlow, Performance Testing

Load testing is widely considered a meaningful technique for performance quality assurance. However, empirical studies reveal that in practice, load testing is not applied systematically, due to the sound expert knowledge required to specify, implement, and execute load tests. Our Behavior-driven Load Testing (BDLT) approach eases load test specification and execution for users with no or little expert knowledge. It allows a user to describe a load test in a template based on
natural language and to rely on an automated framework to execute the test. Utilizing the system’s contextual knowledge such as workload-influencing events, the framework automatically determines the workload and test configuration. We investigated the applicability of our approach in an industrial case study, where we were able to express four load test concerns using BDLT and received positive feedback from our industrial partner. They understood the BDLT definitions well and proposed further applications, such as the usage for software quality acceptance criteria.

Citation Key2019:icpe:benchflow
Refereed DesignationRefereed