One against many : exploring multi-task learning generalization in source-code tasks

• Ano de defesa: 2023
• Autor(a) principal: Parraga, Otávio
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Pontifícia Universidade Católica do Rio Grande do Sul; Escola Politécnica; Brasil; PUCRS; Programa de Pós-Graduação em Ciência da Computação
• Palavras-chave em Português: multi-task; source-code; transformers; multi-tarefa; código-fonte; CIENCIA DA COMPUTACAO: TEORIA DA COMPUTACAO
• Link de acesso: https://tede2.pucrs.br/tede2/handle/tede/11401

Software engineering is a complex process that involves several steps, often requiring a significant investment of resources. As a result, many tools to support development have emerged, with machine learning models becoming increasingly popular for related tasks. Recently, Transformers, a class of models, has achieved tremendous success in natural language processing and has been adapted to work with source code, with models like CodeBERT trained on both text and code. CodeT5, one such model, employs a prompt multi-task approach during training to ensure better generalization capability for target tasks. First, however, it needs to be clarified what impact this multi-tasking approach has on a Big Code scenario. In this thesis, we studied the various advantages and disadvantages of this learning approach for source-code-related tasks. Using state-of-the-art pre-trained models, we compared task-specific and prompt multi-task methods, analyzing results on specific tasks to understand their influence on performance. We also experimented with different task combinations to determine which are most beneficial and whether they help the model better understand the context in which it is being used. This work sheds light on prompt multi-task learning for source-code tasks, highlighting how it can improve resource efficiency and advance research in multi-task learning for big code