Efficient adaptive multiresolution representation of music signals

• Ano de defesa: 2020
• Autor(a) principal: Figueiredo, Nicolas Silverio
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Biblioteca Digitais de Teses e Dissertações da USP
• Palavras-chave em Português: Adaptive representation; Automatic music transcription; Computação sonora e musical; Multiresolution representation; Representação adaptativa; Representação multi-resolução; Sound and music computing; Transcrição automática de música
• Link de acesso: https://www.teses.usp.br/teses/disponiveis/45/45134/tde-17022021-201043/

The inherent trade-off between time and frequency resolutions, which exists in conventional transforms (such as the Discrete Fourier Transform) may be a hindrance for the representation of music signals, since these transforms are incapable of simultaneously locating percussive events with precision in time and melodic events with precision in frequency. Adaptive representations intend to address this limitation by varying the analysis window size used in sub-regions of the time-frequency plane (TFP), and can be used as input representations in algorithms for automatic music transcription, source separation and musical expressiveness analysis. The main objective of the presented work is the development of an efficient adaptive transform, that serves as a counterpoint to traditional algorithms based on the combination of precomputed representations with different resolutions. The proposed Iteratively Refined Multi\\-resolution Spectro\\-gram (IRMS) works by performing successive refinements on top of an initial low frequency resolution spectrogram, located in the areas of the TFP that contain musical information such as notes, harmonics and expressive elements. Its development is built on the investigation of musical information estimators and sub-band processing techniques that allow the efficient computation of high resolution representations within isolated subregions of the TFP. As an investigation of sub-band processing algorithms for this task, a GUI application was built for the detailed high-resolution visualization of specific areas of a spectrogram. A comparative experiment between different musical information estimators was conducted, with good results for Shannon and Rényi entropies. This work also presents technical details on the integration between the detection of musically relevant subregions and their refinement via sub-band processing, that defines our final implementation of the IRMS. As an evaluation of the final solution, a comparative experiment based on computing cost between different time-frequency representations was conducted. The IRMS achieved execution times orders of magnitude faster than the other evaluated adaptive representations, and in some configurations presented a competitive computational cost with respect to the STFT and CQT, thus validating our proposal of an efficient alternative for adaptive representations.