Improving parallel programming assessment : challenges, methods, and opportunities in coding productivity

• Ano de defesa: 2023
• Autor(a) principal: Andrade, Gabriella Lopes
• Tipo de documento: Tese
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Pontifícia Universidade Católica do Rio Grande do Sul; Escola Politécnica; Brasil; PUCRS; Programa de Pós-Graduação em Ciência da Computação
• Palavras-chave em Português: Coding Metrics; Development Effort; GPU; Multi-Core; Parallel Computing; Esforço de Desenvolvimento; Computação Paralela; Métricas de Codificação; CIENCIA DA COMPUTACAO::TEORIA DA COMPUTACAO
• Link de acesso: https://tede2.pucrs.br/tede2/handle/tede/10886

Developing parallel applications is a challenging task because the developers must be able to deal with several issues, such as implementing data synchronization, dividing the computation problem among threads, and exploiting concurrency. New Parallel Programming Interfaces (PPIs) have emerged to facilitate this task. When evaluating these IPPs, most studies in the parallel programming area focus on assessing the execution time and performance of these IPPs. However, productivity is an important factor that, together with effectiveness and user satisfaction, are usability indicators. From evaluating productivity and usability, it is possible to continue to increase the abstractions of parallelism and create better and simple-to-use PPIs without compromising application performance. Therefore, the main goal of this Ph.D. thesis is to provide methodologies and techniques to improve and support productivity evaluation in parallel programming. To achieve this goal, we initially conducted a literature review to determine how usability and productivity have been evaluated in parallel programming. From this review, we identified that to assess productivity in parallel programming, some researchers are conducting studies with people, usually application developers, which require some time to be planned and executed. On the other hand, some researchers have focused on using Software Engineering metrics (for example, CCN, COCOMO II and Halstead), which were not designed to evaluate parallel application development specifically. Regarding the experimentation process, in this Ph.D. thesis, we presented a methodology to guide other parallel programming researchers during the planning, execution, and analysis of experiment results. We conducted experiments with beginners in parallel programming to validate this methodology by exploring parallelism in stream processing applications in multi-core environments and data parallelism in GPU architectures. Regarding coding metrics, we conducted a study to verify the effectiveness of these metrics when evaluating the productivity of IPPs. From this study, we identified some popular metrics in the Software Engineering area not yet explored in the parallel programming area, including Putnam’s model, Function Points, Use Case Points, and Planning Poker. To identify the limitations and opportunities for improvement of these metrics, we verified their accuracy when estimating the development time of parallel applications. Furthermore, we conducted a survey with parallel application developers to identify the factors that impact development productivity. We aimed to propose improvements to coding metrics based on the identified factors. The results of the experiments with beginners showed that, as expected, PPIs with a higher level of abstraction tend to increase developer productivity in both multi-core and GPU environments. This result was also confirmed by surveying parallel programming developers. Furthermore, the results of this survey confirmed that the developers’ experience is one of the main factors influencing parallel application development. The evaluation of the coding metrics showed that Planning Poker proved to be a promising metric because it considers the opinions of experienced developers when estimating the development effort. In this regard, we proposed modifying the Planning Poker metric by considering the opinion of only one developer instead of a development team. The results showed that Planning Poker is an effective method that requires less effort to use in practice compared to controlled experiments with students that aim to collect development time data. Therefore, this metric could be used as an alternative to measuring development time for parallel applications.