Fault tolerance for high-level parallel and distributed stream processing in C++

• Ano de defesa: 2025
• Autor(a) principal: Alf, Lucas Machado
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Pontifícia Universidade Católica do Rio Grande do Sul; Escola Politécnica; Brasil; PUCRS; Programa de Pós-Graduação em Ciência da Computação
• Palavras-chave em Português: Parallel Programming; Domain-Specific Language; Resilience; Message Delivery Guarantees; Linear Pipeline; Stream Parallelism; Programação Paralela; Linguagem de Domínio Específico; Garantias de Entrega de Mensagem; Pipeline Linear; Paralelismo de Stream; CIENCIA DA COMPUTACAO::TEORIA DA COMPUTACAO
• Link de acesso: https://tede2.pucrs.br/tede2/handle/tede/11665

Stream processing is a computing paradigm that addresses the gathering, processing, and analysis of a high-volume heterogeneous continuous data stream, aiming to extract valuable information in real-time. Considering the need for these systems to run for long periods, possibly indefinitely, reprocessing all data in case of failure can be highly costly or even unfeasible. Thus, it is essential for a stream processing system not only to recover after a failure but also to ensure that the generated results are correct. SPar is an annotation-based C++ domain-specific language designed to simplify the development of stream processing applications for multiple parallel architectures. In distributed architectures, SPar generates code using DSParLib, which does not provide fault tolerance mechanisms or strong message delivery guarantees. Other parallel programming alternatives in the literature, such as MPI, are low-level APIs with restrictive support for implementing resilient parallel applications. Most state-of-the-art distributed parallel programming tools for streaming applications with resilience support are available in Java. Given these factors, the main objective of this master’s thesis is to investigate fault tolerance and exactly-once semantics for streaming systems so that it is possible to support it on the SPar software ecosystem without rewriting the user source code. To this end, we created ResiPipe, a C++ library for fault-tolerant distributed stream processing that became a runtime system from the SPar ecosystem to execute parallel code. The results demonstrate that ResiPipe delivers performance comparable to, and in some cases superior o, other analyzed stream processing libraries, in addition to requiring fewer source lines of code (SLOC) per application and presenting a reduced estimated development time, according to Halstead’s method