Descrevendo regiões de imagens através de redes neurais profundas e abstract meaning representation

• Ano de defesa: 2020
• Autor(a) principal: Almeida Neto, Antonio Manoel dos Santos
• Orientador(a): Caseli, Helena de Medeiros
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: por
• Instituição de defesa: Universidade Federal de São Carlos; Câmpus São Carlos
• Programa de Pós-Graduação: Programa de Pós-Graduação em Ciência da Computação - PPGCC
• Palavras-chave em Português: Descrição de regiões de imagem; Representação semântica
• Palavras-chave em Inglês: Abstract meaning representation; Dense captioning
• Área do conhecimento CNPq: CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO
• Link de acesso: https://repositorio.ufscar.br/handle/20.500.14289/13028

The world around us is composed of images that often need to be translated into words. This translation can take place in parts, converting regions of the image into textual descriptions. The description of the region of an image is the transformation of the information contained in this area into words in natural language, to express the way objects relate to each other. Recently, computational models that seek to perform this task in a similar way to human beings are being proposed, mainly using deep neural networks. As a way to improve the quality of the sentences produced by one of these models, this work verified the employability of the Abstract Meaning Representation (AMR) semantic representation in the generation of descriptions for image regions. AMR was investigated as representation formalism, as an alternative to natural language, using it with some variations, so that the machine learning model, using deep neural networks, was able to predict sentences in such representation. The hypothesis of this study, that the use of sentences in the form of AMR would result in better descriptions, was partially confirmed, since the model trained with AMR was superior in almost all evaluations.