Leakage detection in a real water distribution network through a federated prototype-based model

Detalhes bibliográficos
Ano de defesa: 2024
Autor(a) principal: Sousa, Diego Perdigão
Orientador(a): Cavalcante, Charles Casimiro
Banca de defesa: Não Informado pela instituição
Tipo de documento: Tese
Tipo de acesso: Acesso aberto
Idioma: eng
Instituição de defesa: Não Informado pela instituição
Programa de Pós-Graduação: Não Informado pela instituição
Departamento: Não Informado pela instituição
País: Não Informado pela instituição
Área do conhecimento CNPq:
Link de acesso: http://repositorio.ufc.br/handle/riufc/80329
Resumo: This thesis explores important an intricate trade-off in intelligent systems: the efficiency of leakage detection tasks, the preservation of sensitive information, and the interpretability of the intelligent system. Through a comprehensive analysis of these interconnected domains, we aim to design a low-cost algorithmic solution that can efficiently learn from industrial data collected by distributed sensors while preserving sensitive information. We focus on the following topics: (i) efficient leakage detection on water distribution networks (WDNs); and (ii) formulation of federated prototype-based models. In particular, this thesis mainly focuses on proposing an efficient and low-complexity distributed solution for identifying potential leaks in water distribution networks in municipal areas while ensuring the privacy of the hydraulic data. To this end, we explore and extend existing theories and methods from prototype-based learning and federated learning. We consider a hydraulic dataset, which includes water pressure and flow measurements obtained from pumps within district-metered areas in Stockholm, Sweden. In the first part of this thesis, we have considered a traditional learning paradigm, which contemplates the existence of a server responsible to collect and process all samples from every device and build machine learning models. Our goal is validating our assumption that prototypebased models (PBMs) can efficiently detect leakages in water distribution networks. The feature extraction, PBMs and clustering validation techniques we propose have low complexity, and the numerical experiments show the potential of using machine learning strategies in leakage detection for monitored water distribution networks. Specifically, our experiments show that the proposed learning strategies are able to obtain correct classification rates up to 93.98%. In the second part of this thesis, we have investigated the task of detecting leakages in WDNs by proposing a distributed approach to prototype-based models. In particular, we extended our initial formulation by designing a solution grounded on federated learning. Moreover, we further extended the previous analysis by including water flow measurements. Since we have utilized a federated modeling paradigm, we have considered that every device is responsible for processing their collected samples and generating local models. Meanwhile, the server only processes local models to building an efficient global model. Specifically, our experiments show that the proposed optimized learning method can obtain higher detection rates at each station than the conventional centralized approach, e.g., improvements of purity rates up to 7.6% in one of the pumping stations, which increased the minimum values from 92.13% obtained through centralized learning, to 99.11%, obtained via federated learning.
id UFC-7_bc7c3e6fe58655efa9d0d2eced859ee1
oai_identifier_str oai:repositorio.ufc.br:riufc/80329
network_acronym_str UFC-7
network_name_str Repositório Institucional da Universidade Federal do Ceará (UFC)
repository_id_str
spelling Sousa, Diego PerdigãoFischione, CarloCavalcante, Charles Casimiro2025-04-08T13:35:54Z2025-04-08T13:35:54Z2024SOUSA, Diego Perdigão. Leakage detection in a real water distribution network through a federated prototype-based model. 2024. 154 f. Tese (Doutorado em Engenharia de Teleinformática) – Centro de Tecnologia, Universidade Federal do Ceará, Fortaleza, 2024.http://repositorio.ufc.br/handle/riufc/80329This thesis explores important an intricate trade-off in intelligent systems: the efficiency of leakage detection tasks, the preservation of sensitive information, and the interpretability of the intelligent system. Through a comprehensive analysis of these interconnected domains, we aim to design a low-cost algorithmic solution that can efficiently learn from industrial data collected by distributed sensors while preserving sensitive information. We focus on the following topics: (i) efficient leakage detection on water distribution networks (WDNs); and (ii) formulation of federated prototype-based models. In particular, this thesis mainly focuses on proposing an efficient and low-complexity distributed solution for identifying potential leaks in water distribution networks in municipal areas while ensuring the privacy of the hydraulic data. To this end, we explore and extend existing theories and methods from prototype-based learning and federated learning. We consider a hydraulic dataset, which includes water pressure and flow measurements obtained from pumps within district-metered areas in Stockholm, Sweden. In the first part of this thesis, we have considered a traditional learning paradigm, which contemplates the existence of a server responsible to collect and process all samples from every device and build machine learning models. Our goal is validating our assumption that prototypebased models (PBMs) can efficiently detect leakages in water distribution networks. The feature extraction, PBMs and clustering validation techniques we propose have low complexity, and the numerical experiments show the potential of using machine learning strategies in leakage detection for monitored water distribution networks. Specifically, our experiments show that the proposed learning strategies are able to obtain correct classification rates up to 93.98%. In the second part of this thesis, we have investigated the task of detecting leakages in WDNs by proposing a distributed approach to prototype-based models. In particular, we extended our initial formulation by designing a solution grounded on federated learning. Moreover, we further extended the previous analysis by including water flow measurements. Since we have utilized a federated modeling paradigm, we have considered that every device is responsible for processing their collected samples and generating local models. Meanwhile, the server only processes local models to building an efficient global model. Specifically, our experiments show that the proposed optimized learning method can obtain higher detection rates at each station than the conventional centralized approach, e.g., improvements of purity rates up to 7.6% in one of the pumping stations, which increased the minimum values from 92.13% obtained through centralized learning, to 99.11%, obtained via federated learning.Esta tese explora uma intricada compensação em sistemas inteligentes: a eficiência da tarefa de detecção de anomalias, a preservação de informações sensíveis e a interpretabilidade do sistema inteligente. Através de uma análise abrangente desses domínios interconectados, buscamos desenvolver uma solução algorítmica de baixo custo que possa aprender eficientemente a partir de dados industriais observados por sensores geograficamente distribuídos, onde a preservação de informações sensíveis é uma restrição. Nosso foco está nos seguintes tópicos: (i) detecção eficiente de vazamentos em redes de distribuição de água; e (ii) formulação de modelos federados baseados em protótipos. Especificamente, esta tese concentra-se principalmente em propor uma solução distribuída eficiente e de baixa complexidade para identificar vazamentos potenciais em redes de distribuição de água em áreas municipais, garantindo ao mesmo tempo a privacidade dos dados hidráulicos. Para isso, exploramos e estendemos teorias e métodos existentes sobre aprendizado de máquina baseado em protótipos e aprendizado federado. Além disso, em relação aos dados hidráulicos, analisamos medidas de pressão e fluxo de água obtidas de bombas em distritos de medição e controle em Estocolmo, Suécia. Na primeira parte desta tese, consideramos um paradigma de aprendizado tradicional, que contempla a existência de um servidor responsável por coletar e processar todas as amostras de cada dispositivo e construir modelos de aprendizado de máquina. Nosso objetivo é validar nossa suposição de que modelos baseados em protótipos (MBPs) podem detectar eficientemente vazamentos em redes de distribuição de água. A técnica de extração de características, MBPs e técnicas de validação de agrupamentos que propomos têm baixa complexidade, e os experimentos numéricos mostram o potencial do uso de estratégias de aprendizado de máquina na detecção de vazamentos em redes de distribuição de água monitoradas. Especificamente, nossos experimentos mostram que as estratégias de aprendizado propostas são capazes de obter taxas de classificação correta de até 93,98%. Na segunda parte desta tese, investigamos a tarefa de detectar vazamentos em redes de distribuição de água, propondo uma abordagem distribuída para modelos baseados em protótipos. Em particular, estendemos nossa formulação inicial projetando uma solução baseada em aprendizado federado. Além disso, ampliamos a análise anterior incluindo medidas de fluxo de água. Como utilizamos um paradigma de modelagem federado, consideramos que cada dispositivo é responsável pelo processamento de suas amostras coletadas e pela geração de modelos locais. Enquanto isso, o servidor processa apenas modelos locais para construir um modelo global eficiente. Especificamente, nossos experimentos mostram que o método otimizado de aprendizado proposto pode obter taxas de detecção mais altas em cada estação do que a abordagem centralizada convencional, por exemplo, melhorias nas taxas de pureza de até 7,6% em uma das estações de bombeamento, que aumentou os valores mínimos de 92,13% obtidos através do aprendizado centralizado para 99,11%, obtidos via aprendizado federado.Este documento está disponível online com base na Portaria no 348, de 08 de dezembro de 2022, disponível em: https://biblioteca.ufc.br/wp-content/uploads/2022/12/portaria348-2022.pdf, que autoriza a digitalização e a disponibilização no Repositório Institucional (RI) da coleção retrospectiva de TCC, dissertações e teses da UFC, sem o termo de anuência prévia dos autores. Em caso de trabalhos com pedidos de patente e/ou de embargo, cabe, exclusivamente, ao autor(a) solicitar a restrição de acesso ou retirada de seu trabalho do RI, mediante apresentação de documento comprobatório à Direção do Sistema de Bibliotecas.Leakage detection in a real water distribution network through a federated prototype-based modelLeakage detection in a real water distribution network through a federated prototype-based modelinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/doctoralThesisAprendizado federadoVazamentos de água - DetecçãoModelos baseados em protótiposÁgua - DistribuiçãoFederated learningWater leaks - DetectionPrototype-based modelsWater - DistributionCNPQ::ENGENHARIAS::ENGENHARIA ELETRICAinfo:eu-repo/semantics/openAccessengreponame:Repositório Institucional da Universidade Federal do Ceará (UFC)instname:Universidade Federal do Ceará (UFC)instacron:UFChttp://lattes.cnpq.br/3866699869645553https://orcid.org/0000-0002-4198-4064http://lattes.cnpq.br/4751699166195344https://orcid.org/0000-0001-9810-34782025-02-20ORIGINAL2024_dpsousa.pdf2024_dpsousa.pdfTeseapplication/pdf20551904http://repositorio.ufc.br/bitstream/riufc/80329/1/2024_dpsousa.pdfec0e8d875e55e03500c5e3281308af64MD51LICENSElicense.txtlicense.txttext/plain; charset=utf-81748http://repositorio.ufc.br/bitstream/riufc/80329/2/license.txt8a4605be74aa9ea9d79846c1fba20a33MD52riufc/803292025-04-08 10:58:45.181oai:repositorio.ufc.br:riufc/80329Tk9URTogUExBQ0UgWU9VUiBPV04gTElDRU5TRSBIRVJFClRoaXMgc2FtcGxlIGxpY2Vuc2UgaXMgcHJvdmlkZWQgZm9yIGluZm9ybWF0aW9uYWwgcHVycG9zZXMgb25seS4KCk5PTi1FWENMVVNJVkUgRElTVFJJQlVUSU9OIExJQ0VOU0UKCkJ5IHNpZ25pbmcgYW5kIHN1Ym1pdHRpbmcgdGhpcyBsaWNlbnNlLCB5b3UgKHRoZSBhdXRob3Iocykgb3IgY29weXJpZ2h0Cm93bmVyKSBncmFudHMgdG8gRFNwYWNlIFVuaXZlcnNpdHkgKERTVSkgdGhlIG5vbi1leGNsdXNpdmUgcmlnaHQgdG8gcmVwcm9kdWNlLAp0cmFuc2xhdGUgKGFzIGRlZmluZWQgYmVsb3cpLCBhbmQvb3IgZGlzdHJpYnV0ZSB5b3VyIHN1Ym1pc3Npb24gKGluY2x1ZGluZwp0aGUgYWJzdHJhY3QpIHdvcmxkd2lkZSBpbiBwcmludCBhbmQgZWxlY3Ryb25pYyBmb3JtYXQgYW5kIGluIGFueSBtZWRpdW0sCmluY2x1ZGluZyBidXQgbm90IGxpbWl0ZWQgdG8gYXVkaW8gb3IgdmlkZW8uCgpZb3UgYWdyZWUgdGhhdCBEU1UgbWF5LCB3aXRob3V0IGNoYW5naW5nIHRoZSBjb250ZW50LCB0cmFuc2xhdGUgdGhlCnN1Ym1pc3Npb24gdG8gYW55IG1lZGl1bSBvciBmb3JtYXQgZm9yIHRoZSBwdXJwb3NlIG9mIHByZXNlcnZhdGlvbi4KCllvdSBhbHNvIGFncmVlIHRoYXQgRFNVIG1heSBrZWVwIG1vcmUgdGhhbiBvbmUgY29weSBvZiB0aGlzIHN1Ym1pc3Npb24gZm9yCnB1cnBvc2VzIG9mIHNlY3VyaXR5LCBiYWNrLXVwIGFuZCBwcmVzZXJ2YXRpb24uCgpZb3UgcmVwcmVzZW50IHRoYXQgdGhlIHN1Ym1pc3Npb24gaXMgeW91ciBvcmlnaW5hbCB3b3JrLCBhbmQgdGhhdCB5b3UgaGF2ZQp0aGUgcmlnaHQgdG8gZ3JhbnQgdGhlIHJpZ2h0cyBjb250YWluZWQgaW4gdGhpcyBsaWNlbnNlLiBZb3UgYWxzbyByZXByZXNlbnQKdGhhdCB5b3VyIHN1Ym1pc3Npb24gZG9lcyBub3QsIHRvIHRoZSBiZXN0IG9mIHlvdXIga25vd2xlZGdlLCBpbmZyaW5nZSB1cG9uCmFueW9uZSdzIGNvcHlyaWdodC4KCklmIHRoZSBzdWJtaXNzaW9uIGNvbnRhaW5zIG1hdGVyaWFsIGZvciB3aGljaCB5b3UgZG8gbm90IGhvbGQgY29weXJpZ2h0LAp5b3UgcmVwcmVzZW50IHRoYXQgeW91IGhhdmUgb2J0YWluZWQgdGhlIHVucmVzdHJpY3RlZCBwZXJtaXNzaW9uIG9mIHRoZQpjb3B5cmlnaHQgb3duZXIgdG8gZ3JhbnQgRFNVIHRoZSByaWdodHMgcmVxdWlyZWQgYnkgdGhpcyBsaWNlbnNlLCBhbmQgdGhhdApzdWNoIHRoaXJkLXBhcnR5IG93bmVkIG1hdGVyaWFsIGlzIGNsZWFybHkgaWRlbnRpZmllZCBhbmQgYWNrbm93bGVkZ2VkCndpdGhpbiB0aGUgdGV4dCBvciBjb250ZW50IG9mIHRoZSBzdWJtaXNzaW9uLgoKSUYgVEhFIFNVQk1JU1NJT04gSVMgQkFTRUQgVVBPTiBXT1JLIFRIQVQgSEFTIEJFRU4gU1BPTlNPUkVEIE9SIFNVUFBPUlRFRApCWSBBTiBBR0VOQ1kgT1IgT1JHQU5JWkFUSU9OIE9USEVSIFRIQU4gRFNVLCBZT1UgUkVQUkVTRU5UIFRIQVQgWU9VIEhBVkUKRlVMRklMTEVEIEFOWSBSSUdIVCBPRiBSRVZJRVcgT1IgT1RIRVIgT0JMSUdBVElPTlMgUkVRVUlSRUQgQlkgU1VDSApDT05UUkFDVCBPUiBBR1JFRU1FTlQuCgpEU1Ugd2lsbCBjbGVhcmx5IGlkZW50aWZ5IHlvdXIgbmFtZShzKSBhcyB0aGUgYXV0aG9yKHMpIG9yIG93bmVyKHMpIG9mIHRoZQpzdWJtaXNzaW9uLCBhbmQgd2lsbCBub3QgbWFrZSBhbnkgYWx0ZXJhdGlvbiwgb3RoZXIgdGhhbiBhcyBhbGxvd2VkIGJ5IHRoaXMKbGljZW5zZSwgdG8geW91ciBzdWJtaXNzaW9uLgo=Repositório InstitucionalPUBhttp://www.repositorio.ufc.br/ri-oai/requestbu@ufc.br || repositorio@ufc.bropendoar:2025-04-08T13:58:45Repositório Institucional da Universidade Federal do Ceará (UFC) - Universidade Federal do Ceará (UFC)false
dc.title.pt_BR.fl_str_mv Leakage detection in a real water distribution network through a federated prototype-based model
dc.title.en.pt_BR.fl_str_mv Leakage detection in a real water distribution network through a federated prototype-based model
title Leakage detection in a real water distribution network through a federated prototype-based model
spellingShingle Leakage detection in a real water distribution network through a federated prototype-based model
Sousa, Diego Perdigão
CNPQ::ENGENHARIAS::ENGENHARIA ELETRICA
Aprendizado federado
Vazamentos de água - Detecção
Modelos baseados em protótipos
Água - Distribuição
Federated learning
Water leaks - Detection
Prototype-based models
Water - Distribution
title_short Leakage detection in a real water distribution network through a federated prototype-based model
title_full Leakage detection in a real water distribution network through a federated prototype-based model
title_fullStr Leakage detection in a real water distribution network through a federated prototype-based model
title_full_unstemmed Leakage detection in a real water distribution network through a federated prototype-based model
title_sort Leakage detection in a real water distribution network through a federated prototype-based model
author Sousa, Diego Perdigão
author_facet Sousa, Diego Perdigão
author_role author
dc.contributor.co-advisor.none.fl_str_mv Fischione, Carlo
dc.contributor.author.fl_str_mv Sousa, Diego Perdigão
dc.contributor.advisor1.fl_str_mv Cavalcante, Charles Casimiro
contributor_str_mv Cavalcante, Charles Casimiro
dc.subject.cnpq.fl_str_mv CNPQ::ENGENHARIAS::ENGENHARIA ELETRICA
topic CNPQ::ENGENHARIAS::ENGENHARIA ELETRICA
Aprendizado federado
Vazamentos de água - Detecção
Modelos baseados em protótipos
Água - Distribuição
Federated learning
Water leaks - Detection
Prototype-based models
Water - Distribution
dc.subject.ptbr.pt_BR.fl_str_mv Aprendizado federado
Vazamentos de água - Detecção
Modelos baseados em protótipos
Água - Distribuição
dc.subject.en.pt_BR.fl_str_mv Federated learning
Water leaks - Detection
Prototype-based models
Water - Distribution
description This thesis explores important an intricate trade-off in intelligent systems: the efficiency of leakage detection tasks, the preservation of sensitive information, and the interpretability of the intelligent system. Through a comprehensive analysis of these interconnected domains, we aim to design a low-cost algorithmic solution that can efficiently learn from industrial data collected by distributed sensors while preserving sensitive information. We focus on the following topics: (i) efficient leakage detection on water distribution networks (WDNs); and (ii) formulation of federated prototype-based models. In particular, this thesis mainly focuses on proposing an efficient and low-complexity distributed solution for identifying potential leaks in water distribution networks in municipal areas while ensuring the privacy of the hydraulic data. To this end, we explore and extend existing theories and methods from prototype-based learning and federated learning. We consider a hydraulic dataset, which includes water pressure and flow measurements obtained from pumps within district-metered areas in Stockholm, Sweden. In the first part of this thesis, we have considered a traditional learning paradigm, which contemplates the existence of a server responsible to collect and process all samples from every device and build machine learning models. Our goal is validating our assumption that prototypebased models (PBMs) can efficiently detect leakages in water distribution networks. The feature extraction, PBMs and clustering validation techniques we propose have low complexity, and the numerical experiments show the potential of using machine learning strategies in leakage detection for monitored water distribution networks. Specifically, our experiments show that the proposed learning strategies are able to obtain correct classification rates up to 93.98%. In the second part of this thesis, we have investigated the task of detecting leakages in WDNs by proposing a distributed approach to prototype-based models. In particular, we extended our initial formulation by designing a solution grounded on federated learning. Moreover, we further extended the previous analysis by including water flow measurements. Since we have utilized a federated modeling paradigm, we have considered that every device is responsible for processing their collected samples and generating local models. Meanwhile, the server only processes local models to building an efficient global model. Specifically, our experiments show that the proposed optimized learning method can obtain higher detection rates at each station than the conventional centralized approach, e.g., improvements of purity rates up to 7.6% in one of the pumping stations, which increased the minimum values from 92.13% obtained through centralized learning, to 99.11%, obtained via federated learning.
publishDate 2024
dc.date.issued.fl_str_mv 2024
dc.date.accessioned.fl_str_mv 2025-04-08T13:35:54Z
dc.date.available.fl_str_mv 2025-04-08T13:35:54Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/doctoralThesis
format doctoralThesis
status_str publishedVersion
dc.identifier.citation.fl_str_mv SOUSA, Diego Perdigão. Leakage detection in a real water distribution network through a federated prototype-based model. 2024. 154 f. Tese (Doutorado em Engenharia de Teleinformática) – Centro de Tecnologia, Universidade Federal do Ceará, Fortaleza, 2024.
dc.identifier.uri.fl_str_mv http://repositorio.ufc.br/handle/riufc/80329
identifier_str_mv SOUSA, Diego Perdigão. Leakage detection in a real water distribution network through a federated prototype-based model. 2024. 154 f. Tese (Doutorado em Engenharia de Teleinformática) – Centro de Tecnologia, Universidade Federal do Ceará, Fortaleza, 2024.
url http://repositorio.ufc.br/handle/riufc/80329
dc.language.iso.fl_str_mv eng
language eng
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.source.none.fl_str_mv reponame:Repositório Institucional da Universidade Federal do Ceará (UFC)
instname:Universidade Federal do Ceará (UFC)
instacron:UFC
instname_str Universidade Federal do Ceará (UFC)
instacron_str UFC
institution UFC
reponame_str Repositório Institucional da Universidade Federal do Ceará (UFC)
collection Repositório Institucional da Universidade Federal do Ceará (UFC)
bitstream.url.fl_str_mv http://repositorio.ufc.br/bitstream/riufc/80329/1/2024_dpsousa.pdf
http://repositorio.ufc.br/bitstream/riufc/80329/2/license.txt
bitstream.checksum.fl_str_mv ec0e8d875e55e03500c5e3281308af64
8a4605be74aa9ea9d79846c1fba20a33
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
repository.name.fl_str_mv Repositório Institucional da Universidade Federal do Ceará (UFC) - Universidade Federal do Ceará (UFC)
repository.mail.fl_str_mv bu@ufc.br || repositorio@ufc.br
_version_ 1847793307703312384