Multi-sensor fusion with attention mechanisms for visual perception in autonomous vehicles

• Ano de defesa: 2025
• Autor(a) principal: Honorato, Eduardo Sperle
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Biblioteca Digitais de Teses e Dissertações da USP
• Palavras-chave em Português: 3D object detection; Attention mechanism; Autonomous vehicles; Detecção de objetos 3D; Fusão multi-sensorial; Mecanismo de atenção; Multi-sensor fusion; Segmentação; Segmentation; Veículos autônomos
• Link de acesso: https://www.teses.usp.br/teses/disponiveis/55/55134/tde-22092025-151759/

Autonomous vehicles are increasingly becoming a part of urban life. However, significant challenges still need to be overcome to ensure these vehicles are both safe and efficient. One of the main challenges lies in perception, particularly in 3D object segmentation and detection, which relies on multiple sensors to enhance accuracy and operate under adverse conditions. The efficient fusion of these sensors is a crucial factor, as it directly impacts detection quality and computational cost. Modern multi-sensor fusion methods leverage Deep Learning techniques, and an emerging approach in this field is the use of attention mechanisms. These mechanisms enable more informative representations of feature maps extracted from sensors, highlighting the most relevant information while suppressing less significant data. In this context, this study explores the use of attention mechanisms to optimise the BEVFusion model, which has achieved state-of-the-art performance by employing a unified Camera-LiDAR fusion in a Birds Eye View (BEV) representation. The key advantage of BEVFusion is its highly efficient transformation of camera-perspective views into the BEV representation. However, its sensor fusion approach is limited to a simple concatenation of extracted features, which may not be the most efficient solution. Another critical aspect of BEVFusion is its high computational cost, as it relies on deep learning models that demand powerful hardware, posing a challenge for deployment in autonomous vehicles. This is particularly relevant given the need for embedded hardware in such vehicles to be both cost-effective and energy-efficient. To address these issues, this study investigates and implements attention mechanisms to enhance the sensor fusion process in BEVFusion for 3D object detection and segmentation, while also improving computational efficiency. Modifications were made to reduce VRAM consumption and processing time, ensuring performance comparable to the original model but with lower computational demands. The results are promising, showing a 14.12% increase in IoU for the segmentation task and a 0.732% improvement in mAP for 3D object detection. Additionally, training time was reduced by a factor of 3.3, and VRAM consumption was nearly halved.