Study on unsupervised machine learning applied to software testing based on resource usage

• Ano de defesa: 2024
• Autor(a) principal: Ruiz, Kevin Gerardo Polo
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Biblioteca Digitais de Teses e Dissertações da USP
• Palavras-chave em Português: Agglomerative clustering; Agrupamento aglomerativo; Anomaly detection; Aprendizado de máquina; Detecção de anomalias; Detecção de defeitos; Fault detection; Machine learning; Software testing; Teste de software
• Link de acesso: https://www.teses.usp.br/teses/disponiveis/55/55134/tde-11022025-170631/

Systems are becoming increasingly complex in the software industry, which also makes them more prone to failure. Ensuring software quality through rigorous testing is crucial, but this task still needs to be improved and more manageable because it still consumes significant parts of the project budget. Automation is a critical factor in improving testing efficiency, with new methodologies and tools being developed continuously. Machine Learning has attracted considerable interest in several areas over the last ten years, driven by advances in computing power and the ability to manage large volumes of data. Tricorder is a testing methodology designed to detect potential software faults by analyzing changes in the resource consumption behavior of the computing system under test. Unsupervised machine learning provided by DAMICORE characterizes resource consumption behavior. The methodology used by DAMICORE is based on a pipeline of three main steps. The pipeline uses Normalized Compression Distance (NCD) to generate a distance matrix, the Neighbor-Joining algorithm to build a phylogenetic tree, and the Fast Newman method for community detection, which is essential for data clustering. DAMICORE monitors and identifies anomalies in resource usage patterns, such as CPU, memory, and I/O, to indicate the presence of software faults. This project studies the impact generated by DAMICORE in the detection of software faults provided by Tricorder in the context of agglomerative clustering and phylogenetic trees, testing several techniques adapted to DAMICORE. The literature review on Machine Learning applied to Software Testing highlighted that the most widely studied techniques for agglomerative clustering are based on Single, Complete, and Average (UPGMA) Linkages. Neighbor-joining and UPGMA are prominent in the construction of phylogenetic trees. To achieve the project objectives, we consolidated the theoretical foundations, followed by an in-depth review of the state of the art and the specific methodologies of Tricorder and DAMICORE. We then systematically designed and executed a series of experiments, analyzing the generated results. The project demonstrated the potential of using Levenshtein distance, leveraging the network topology for community detection, and incorporating all system metrics into a single analysis. These approaches yielded better results in our context than other techniques found in the literature. NCD and Neighbor-Joining have significant limitations, especially due to their high computational demands, which hinder their practical application in more extensive and complex projects. The improvements introduced in this project should improve DAMICOREs accuracy in detecting software faults. The results of this project contribute to the state-of-the-art application of Machine Learning to Software Testing and reinforce Tricorders position in the Software Testing community.