Comparative Study of Transformers in Spatiotemporal Precipitation Forecasting

• Ano de defesa: 2024
• Autor(a) principal: Moura, Mauro Sérgio dos Santos
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Laboratório Nacional de Computação Científica; Coordenação de Pós-Graduação e Aperfeiçoamento (COPGA); Brasil; LNCC; Programa de pós-graduação em Modelagem Computacional
• Palavras-chave em Português: Deep Learning.; Transformers; Precipitation; Time Series Forecast; Spatiotemporal data; CNPQ::CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO
• Link de acesso: https://tede.lncc.br/handle/tede/404

Deep Learning has seen significant advancements in recent times, with various architectures excelling in different areas. Among these, the Transformer architecture stands out, initially for its success in natural language processing, thanks to its attention mechanisms based on an encoder-decoder model. Subsequently, studies have emerged exploring the applicability of this architecture in time series forecasting, yielding promising results. However, the high consumption of computational resources, such as time and memory, has led to the proposal of alternative models that seek to achieve comparable or superior performance using simpler architectures, such as linear models. Precipitation forecasting is an important challenge, especially in regions with unstable climatic conditions, such as Rio de Janeiro, where intense rain can occur suddenly. The ability to predict these rainfall patterns, particularly extreme events, is crucial for mitigating the adverse impacts of these phenomena. Although most studies on Transformers have focused on simpler datasets, such as traffic or temperature, precipitation forecasting has proven to be more challenging, often yielding inferior performance in many cases. Moreover, few studies address precipitation forecasting, and they generally focus on long-term predictions, using precipitation aggregated by day, week, or month. The objective of this study is to investigate the applicability of Transformer-based models for short- and medium-term precipitation forecasting. We used spatiotemporal precipitation data from Australia, available on the Kaggle platform, to evaluate the efficiency of these models in predicting daily aggregated precipitation. Additionally, we employed data from the city of Rio de Janeiro, obtained through INMET meteorological stations via the Rionowcast project, with data aggregated hourly. The use of two datasets allows for the assessment of whether the models’ performance is influenced by data quality or the processing used. Initially, two models were tested with different spatiotemporal representations. Since there was no significant difference between the data processing methods, a new test was conducted with all the implemented models. The evaluation was carried out using Mean Squared Error (MSE) and complementary graphs. The results showed that the DLinear model, despite being a simpler architecture, stood out by focusing on the target variable for prediction, achieving the best MSE and the shortest execution time. However, the tests indicated that the type of data used is not sufficient to unlock the full potential of the other architectures, with little difference in performance among the Transformer models tested, suggesting potential issues related to data processing.