Music representation learning based on heterogeneous graph

• Ano de defesa: 2024
• Autor(a) principal: Silva, Angelo Cesar Mendes da
• Tipo de documento: Tese
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Biblioteca Digitais de Teses e Dissertações da USP
• Palavras-chave em Português: Aprendizado de representação; Graph neural networks; Heterogeneous network; Music information retrieval; Recuperação de informação musical; Redes heterogêneas; Redes neurais para grafos; Representation learning
• Link de acesso: https://www.teses.usp.br/teses/disponiveis/55/55134/tde-24012025-150432/

Music has been present in different historical moments since the formation of society, accompanying survival tasks and leisure tasks. Nowadays, the emergence and popularization of media with resources for storing and reproducing music has further introduced the presence of music in peoples daily lives. In addition to artistic content, songs also began to generate volumes of data and new interested markets. In this sense, several data mining methods have been proposed in recent decades to extract information to support decision-making. Defining a representation for such data is essential for its use in data mining algorithms. Musical data is intrinsically multimodal and heterogeneous, so to represent it, a unified structure must be built that supports features with different semantic compositions and organized in different spaces. Some approaches that explore variations of feature fusion processes are proposed with this objective. Due to the multimodal and heterogeneous structure, representations based on fusion methods have restrictions in scenarios where features are absent and can be reduced to unimodal representations, reducing the diversity of musical content in their formation. Musical representations are introduced into algorithms that deal with tasks modeled as machine learning tasks that produce knowledge that supports decision-making. The challenges in defining musical representations are associated with the lack of information in the data due to access restrictions or incomplete modalities and the construction of a method that aggregates heterogeneous information into a unified space. This thesis focuses on developing data structures that support the natural composition of musical data and representation learning methods capable of dealing with tasks related to musical information retrieval. In particular, the contributions of this thesis are related to the tasks of automatic annotation, recognition of emotions in music, prediction of similarity between artists, and a multitask application. We summarize our contributions in: (i) a methodology for modeling musical data in heterogeneous networks; (ii) an algorithm based on information propagation to deal with the challenge of missing features in data; (iii) methods based on graph neural networks to deal with musical information retrieval tasks; (iv) analyzes related to the complementarity of information between multiple musical features and also related tasks.