Explorando abordagens para legendagem automática de imagens para a identificação do sujeito principal em fotografias de eventos de formaturas

• Ano de defesa: 2023
• Autor(a) principal: Wamser, Lucas Holtz
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: por
• Instituição de defesa: Universidade Tecnológica Federal do Paraná; Curitiba; Brasil; Programa de Pós-Graduação em Computação Aplicada; UTFPR
• Palavras-chave em Português: Imagens fotográficas - Análise; Aprendizado profundo (Aprendizado do computador); Processamento de linguagem natural (Computação); Visão por computador; Processamento de imagens; Sistemas de reconhecimento de padrões; Legendas (filmes, televisão, etc.); Images, Photografic - Analysis; Deep learning (Machine learning); Natural language processing (Computer science); Computer vision; Image processing; Pattern recognition systems; Subtitles (Motion pictures, television, etc.); CNPQ::CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO; Ciência da Computação
• Link de acesso: http://repositorio.utfpr.edu.br/jspui/handle/1/33178

The selection of photographs from a graduation event is an essential task for companies that organize such events, as the sale of personalized albums is a significant part of their revenue. With that in mind, this work focuses on the application of pre-trained neural networks to assist in organizing graduation albums, exploring techniques for automatic image captioning and main subject identification. The study begins with a comparative analysis of three deep learning- based approaches to automatic image captioning in the context of graduations. The One For All (OFA) model, based on transformers, stands out as a promising choice. OFA is pre-trained on a wide variety of data and has been specialized for the specific task of captioning graduation images. Additionally, the work investigates the use of the implicit knowledge in automatic captioning models to identify the main subject in an image. This is essential for the effective organization of graduation albums, where it is crucial to highlight the main protagonists. The improvements, with average Intersection over Union measures of 0.47, compared to 0.17 without specialization. We also explore the use of the captions generated by the model to generate a word cloud, which can be useful for filtering photographs. The contributions of this work are directly relevant to the organization of graduation albums, including the generation of captions and bounding boxes for the main subject in photographs, as well as the creation of word clouds for efficient album organization. In summary, this study highlights the effectiveness of pre-trained neural networks in automatic image captioning and subject identification, providing significant benefits in automating the organization of graduation albums, a valuable task for companies and graduates.