A high-level DSL in rust for expressing linear pipelines on multi-cores, clusters and gpus - Faé, Leonardo Gibrowski (2025)
Modern data-intensive applications demand efficient processing of continuous data streams, often requiring parallel execution across diverse hardware environments, including shared-memory systems, distributed clusters, and GPUs. Programming these environments effectively presents significant challenges due to their distinct characteristics and programming models. This work presents a novel high-level domain-specific language embedded in Rust for expressing linear pipeline parallelism targeting Clusters, multi-core, and GPU architectures. We named it SPar-Rust, which simplifies the expressiveness of linear pipeline parallelism and provides a unified programming interface targeting different parallel architectures for developing productive parallel stream-processing applications. SPar-Rust performs source-to-source code transformations where low-level details are hidden from developers by leveraging Rust’s powerful macro system. We analyze the Rust tool-chain’s capabilities for code generation and abstraction, focusing on our approach’s performance and programmability implications compared to state-of-theart solutions, showcasing its effectiveness and limitations. To the best of our knowledge, SPar-Rust is the first DSL in Rust to offer integrated support for GPU execution and a unified programming interface targeting different parallel architectures.