OpenMLPerf - graphical domain-specific modeling language for performance testing in web systems - Carbonell, João Batista Pedroso (2023)
Testing system performance is one of the most crucial tasks in the software development process and its evolution. Most of the current performance tests focus on evaluating an already implemented system, making it difficult to predict issues such as load capacities and bottlenecks. Therefore, it is recommended to employ mechanisms for modeling and specifying system information before implementation. One potential solution is the utilization of models that represent unchangeable aspects of the system domain, such as infrastructure features or user behavior. Using models to abstract, define, and model domain aspects has become the most commonly adopted approach for addressing problems within specific contexts, aligning with the principles of Model-Driven Engineering (MDE), which advocate for using models throughout the software lifecycle. Domain-Specific Languages (DSLs) are one way to specify and model a domain. DSLs are compact languages with limited expressiveness designed for specific domains. Unlike general-purpose languages like Python and Java, DSLs are not intended for implementing systems across all domains; they are restricted to their original purpose. In this study, we propose the reimplementation of a graphical DSL called Canopus for modeling performance tests in web systems. As part of this reimplementation, we have developed a code generator within the DSL. The code generator is specifically designed to generate SCALA code that can be used in conjunction with the JMeter tool. The previous implementation of Canopus utilized commercial licensed technologies, which limited its distribution and evolution. Hence, we present a new version of this DSL, named OpenMLPerf, which will be released under an open-source license, enabling its use, improvement, and evolution. With the addition of the code generator, OpenMLPerf empowers users to automatically generate SCALA code for their performance test scenarios, facilitating the integration with the widely-used JMeter tool. For the selection of a DSL development tool, we conducted a Systematic Literature Mapping, identifying fifty-three (53) available DSL support tools. We chose to use Ecore from the Eclipse Modeling Framework (EMF) and the Sirius framework to implement the language metamodel and the language itself. In addition to these two frameworks, we also utilized the Acceleo framework for template modeling and Scala code generation for performance testing tools. We conducted an empirical evaluation to measure the effort required by users when modeling a performance test scenario using the OpenMLPerf approach, compared to using a UML profile for performance testing. Although the results indicated that both approaches require similar efforts from participants, the OpenMLPerf approach proved to be highly scalable, comprehensive, and intuitive. A crucial factor for adopting the OpenMLPerf approach is its development within an open-source platform, allowing for replication, customization, and community-driven improvements. As a point of consideration, it is worth noting that the adoption of this approach may be influenced by performance and usability issues related to the Eclipse platform.