Computer vision methods for aerial and ground-based farm monitoring: addressing the labeled data scarcity problem

• Ano de defesa: 2025
• Autor(a) principal: Ferreira, Juliana Quintiliano de Oliveira
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Universidade Federal de Viçosa; Ciência da Computação
• Palavras-chave em Português: Inteligência artificial; Visão por computador; Processamento de imagens; Fazendas; Ciência da Computação
• Link de acesso: https://locus.ufv.br/handle/123456789/35028; https://doi.org/10.47328/ufvbbt.2025.829

The monitoring of agricultural areas is essential to ensure a safe environment, avoid economic losses, and prevent risks to infrastructure and human safety. Effective monitoring can detect lost animals, unauthorized human access, wild animal intrusions, among other issues. Artificial Intelligence (AI) is a powerful tool to automate this process through the processing of images captured from ground or aerial perspectives. However, supervised models require large volumes of labeled data to achieve good performance, and although data exists, most of it is private and inaccessible to the public or available in limited quantity. The available datasets are generally unlabeled or do not cover domain-specific scenarios, such as rural and farm environments. In addition, data collection in agricultural settings presents further challenges, such as the need for drones or other remote sensing technologies, making the process more expensive and complex. Finally, the annotation stage is also a bottleneck, as beyond collecting images, intensive manual work is required to label them, increasing both cost and time. In this context, we propose to address the problem of data scarcity and annotation complexity from two perspectives, aiming to mitigate the bottleneck of training AI models when only a small amount of labeled data is available. The first study focuses on aerial monitoring using images collected by Unmanned Aerial Vehicles (UAVs) on farms for the task of semantic segmentation. Semantic segmentation brings significant benefits to agricultural monitoring by automatically identifying and differentiating important elements of the rural environment, such as vegetation areas, bodies of water, and buildings. By precisely mapping these elements, the technique enables the identification of risk situations for livestock and infrastructure, contributing to safer and more efficient farm management. Thus, we investigated pre-training strategies using synthetic data from the same domain and real data from slightly different domains. We then fine-tuned on the target dataset, and the quantitative and qualitative results demonstrated that pre- training with the synthetic dataset achieved better final performance, leading to an increase of 3.1 p.p. in IoU, 6.4 in F1-Score, and 7.5 in Recall compared to the cross- domain real-image pre-training strategy. In the second study, we focus on object detection using ground-level images similar to security camera footage. This task is important in agricultural monitoring because it allows automatic identification and localization of animals and people, supporting security, livestock management, and the tracking of activities on the farm. To address the data scarcity challenge in this scenario, we proposed a method to effectively use multiple datasets even when they do not share the same classes, ensuring comprehensive coverage of all required categories. The proposed SmartClass methodology achieved more robust and adaptable detection approaches suitable for agricultural environments, with significant increases in Recall, mAP50, and mAP50-95 metrics compared to models trained without the methodology, thus demonstrating improved efficiency and reliability of the model. Keywords: artificial intelligence; computer vision; farm monitoring.