Leakage detection in a real water distribution network through a federated prototype-based model
| Ano de defesa: | 2024 |
|---|---|
| Autor(a) principal: | |
| Orientador(a): | |
| Banca de defesa: | |
| Tipo de documento: | Tese |
| Tipo de acesso: | Acesso aberto |
| Idioma: | eng |
| Instituição de defesa: |
Não Informado pela instituição
|
| Programa de Pós-Graduação: |
Não Informado pela instituição
|
| Departamento: |
Não Informado pela instituição
|
| País: |
Não Informado pela instituição
|
| Área do conhecimento CNPq: | |
| Link de acesso: | http://repositorio.ufc.br/handle/riufc/80329 |
Resumo: | This thesis explores important an intricate trade-off in intelligent systems: the efficiency of leakage detection tasks, the preservation of sensitive information, and the interpretability of the intelligent system. Through a comprehensive analysis of these interconnected domains, we aim to design a low-cost algorithmic solution that can efficiently learn from industrial data collected by distributed sensors while preserving sensitive information. We focus on the following topics: (i) efficient leakage detection on water distribution networks (WDNs); and (ii) formulation of federated prototype-based models. In particular, this thesis mainly focuses on proposing an efficient and low-complexity distributed solution for identifying potential leaks in water distribution networks in municipal areas while ensuring the privacy of the hydraulic data. To this end, we explore and extend existing theories and methods from prototype-based learning and federated learning. We consider a hydraulic dataset, which includes water pressure and flow measurements obtained from pumps within district-metered areas in Stockholm, Sweden. In the first part of this thesis, we have considered a traditional learning paradigm, which contemplates the existence of a server responsible to collect and process all samples from every device and build machine learning models. Our goal is validating our assumption that prototypebased models (PBMs) can efficiently detect leakages in water distribution networks. The feature extraction, PBMs and clustering validation techniques we propose have low complexity, and the numerical experiments show the potential of using machine learning strategies in leakage detection for monitored water distribution networks. Specifically, our experiments show that the proposed learning strategies are able to obtain correct classification rates up to 93.98%. In the second part of this thesis, we have investigated the task of detecting leakages in WDNs by proposing a distributed approach to prototype-based models. In particular, we extended our initial formulation by designing a solution grounded on federated learning. Moreover, we further extended the previous analysis by including water flow measurements. Since we have utilized a federated modeling paradigm, we have considered that every device is responsible for processing their collected samples and generating local models. Meanwhile, the server only processes local models to building an efficient global model. Specifically, our experiments show that the proposed optimized learning method can obtain higher detection rates at each station than the conventional centralized approach, e.g., improvements of purity rates up to 7.6% in one of the pumping stations, which increased the minimum values from 92.13% obtained through centralized learning, to 99.11%, obtained via federated learning. |
| id |
UFC-7_bc7c3e6fe58655efa9d0d2eced859ee1 |
|---|---|
| oai_identifier_str |
oai:repositorio.ufc.br:riufc/80329 |
| network_acronym_str |
UFC-7 |
| network_name_str |
Repositório Institucional da Universidade Federal do Ceará (UFC) |
| repository_id_str |
|
| spelling |
Sousa, Diego PerdigãoFischione, CarloCavalcante, Charles Casimiro2025-04-08T13:35:54Z2025-04-08T13:35:54Z2024SOUSA, Diego Perdigão. Leakage detection in a real water distribution network through a federated prototype-based model. 2024. 154 f. Tese (Doutorado em Engenharia de Teleinformática) – Centro de Tecnologia, Universidade Federal do Ceará, Fortaleza, 2024.http://repositorio.ufc.br/handle/riufc/80329This thesis explores important an intricate trade-off in intelligent systems: the efficiency of leakage detection tasks, the preservation of sensitive information, and the interpretability of the intelligent system. Through a comprehensive analysis of these interconnected domains, we aim to design a low-cost algorithmic solution that can efficiently learn from industrial data collected by distributed sensors while preserving sensitive information. We focus on the following topics: (i) efficient leakage detection on water distribution networks (WDNs); and (ii) formulation of federated prototype-based models. In particular, this thesis mainly focuses on proposing an efficient and low-complexity distributed solution for identifying potential leaks in water distribution networks in municipal areas while ensuring the privacy of the hydraulic data. To this end, we explore and extend existing theories and methods from prototype-based learning and federated learning. We consider a hydraulic dataset, which includes water pressure and flow measurements obtained from pumps within district-metered areas in Stockholm, Sweden. In the first part of this thesis, we have considered a traditional learning paradigm, which contemplates the existence of a server responsible to collect and process all samples from every device and build machine learning models. Our goal is validating our assumption that prototypebased models (PBMs) can efficiently detect leakages in water distribution networks. The feature extraction, PBMs and clustering validation techniques we propose have low complexity, and the numerical experiments show the potential of using machine learning strategies in leakage detection for monitored water distribution networks. Specifically, our experiments show that the proposed learning strategies are able to obtain correct classification rates up to 93.98%. In the second part of this thesis, we have investigated the task of detecting leakages in WDNs by proposing a distributed approach to prototype-based models. In particular, we extended our initial formulation by designing a solution grounded on federated learning. Moreover, we further extended the previous analysis by including water flow measurements. Since we have utilized a federated modeling paradigm, we have considered that every device is responsible for processing their collected samples and generating local models. Meanwhile, the server only processes local models to building an efficient global model. Specifically, our experiments show that the proposed optimized learning method can obtain higher detection rates at each station than the conventional centralized approach, e.g., improvements of purity rates up to 7.6% in one of the pumping stations, which increased the minimum values from 92.13% obtained through centralized learning, to 99.11%, obtained via federated learning.Esta tese explora uma intricada compensação em sistemas inteligentes: a eficiência da tarefa de detecção de anomalias, a preservação de informações sensíveis e a interpretabilidade do sistema inteligente. Através de uma análise abrangente desses domínios interconectados, buscamos desenvolver uma solução algorítmica de baixo custo que possa aprender eficientemente a partir de dados industriais observados por sensores geograficamente distribuídos, onde a preservação de informações sensíveis é uma restrição. Nosso foco está nos seguintes tópicos: (i) detecção eficiente de vazamentos em redes de distribuição de água; e (ii) formulação de modelos federados baseados em protótipos. Especificamente, esta tese concentra-se principalmente em propor uma solução distribuída eficiente e de baixa complexidade para identificar vazamentos potenciais em redes de distribuição de água em áreas municipais, garantindo ao mesmo tempo a privacidade dos dados hidráulicos. Para isso, exploramos e estendemos teorias e métodos existentes sobre aprendizado de máquina baseado em protótipos e aprendizado federado. Além disso, em relação aos dados hidráulicos, analisamos medidas de pressão e fluxo de água obtidas de bombas em distritos de medição e controle em Estocolmo, Suécia. Na primeira parte desta tese, consideramos um paradigma de aprendizado tradicional, que contempla a existência de um servidor responsável por coletar e processar todas as amostras de cada dispositivo e construir modelos de aprendizado de máquina. Nosso objetivo é validar nossa suposição de que modelos baseados em protótipos (MBPs) podem detectar eficientemente vazamentos em redes de distribuição de água. A técnica de extração de características, MBPs e técnicas de validação de agrupamentos que propomos têm baixa complexidade, e os experimentos numéricos mostram o potencial do uso de estratégias de aprendizado de máquina na detecção de vazamentos em redes de distribuição de água monitoradas. Especificamente, nossos experimentos mostram que as estratégias de aprendizado propostas são capazes de obter taxas de classificação correta de até 93,98%. Na segunda parte desta tese, investigamos a tarefa de detectar vazamentos em redes de distribuição de água, propondo uma abordagem distribuída para modelos baseados em protótipos. Em particular, estendemos nossa formulação inicial projetando uma solução baseada em aprendizado federado. Além disso, ampliamos a análise anterior incluindo medidas de fluxo de água. Como utilizamos um paradigma de modelagem federado, consideramos que cada dispositivo é responsável pelo processamento de suas amostras coletadas e pela geração de modelos locais. Enquanto isso, o servidor processa apenas modelos locais para construir um modelo global eficiente. Especificamente, nossos experimentos mostram que o método otimizado de aprendizado proposto pode obter taxas de detecção mais altas em cada estação do que a abordagem centralizada convencional, por exemplo, melhorias nas taxas de pureza de até 7,6% em uma das estações de bombeamento, que aumentou os valores mínimos de 92,13% obtidos através do aprendizado centralizado para 99,11%, obtidos via aprendizado federado.Este documento está disponível online com base na Portaria no 348, de 08 de dezembro de 2022, disponível em: https://biblioteca.ufc.br/wp-content/uploads/2022/12/portaria348-2022.pdf, que autoriza a digitalização e a disponibilização no Repositório Institucional (RI) da coleção retrospectiva de TCC, dissertações e teses da UFC, sem o termo de anuência prévia dos autores. Em caso de trabalhos com pedidos de patente e/ou de embargo, cabe, exclusivamente, ao autor(a) solicitar a restrição de acesso ou retirada de seu trabalho do RI, mediante apresentação de documento comprobatório à Direção do Sistema de Bibliotecas.Leakage detection in a real water distribution network through a federated prototype-based modelLeakage detection in a real water distribution network through a federated prototype-based modelinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/doctoralThesisAprendizado federadoVazamentos de água - DetecçãoModelos baseados em protótiposÁgua - DistribuiçãoFederated learningWater leaks - DetectionPrototype-based modelsWater - DistributionCNPQ::ENGENHARIAS::ENGENHARIA ELETRICAinfo:eu-repo/semantics/openAccessengreponame:Repositório Institucional da Universidade Federal do Ceará (UFC)instname:Universidade Federal do Ceará (UFC)instacron:UFChttp://lattes.cnpq.br/3866699869645553https://orcid.org/0000-0002-4198-4064http://lattes.cnpq.br/4751699166195344https://orcid.org/0000-0001-9810-34782025-02-20ORIGINAL2024_dpsousa.pdf2024_dpsousa.pdfTeseapplication/pdf20551904http://repositorio.ufc.br/bitstream/riufc/80329/1/2024_dpsousa.pdfec0e8d875e55e03500c5e3281308af64MD51LICENSElicense.txtlicense.txttext/plain; charset=utf-81748http://repositorio.ufc.br/bitstream/riufc/80329/2/license.txt8a4605be74aa9ea9d79846c1fba20a33MD52riufc/803292025-04-08 10:58:45.181oai:repositorio.ufc.br:riufc/80329Tk9URTogUExBQ0UgWU9VUiBPV04gTElDRU5TRSBIRVJFClRoaXMgc2FtcGxlIGxpY2Vuc2UgaXMgcHJvdmlkZWQgZm9yIGluZm9ybWF0aW9uYWwgcHVycG9zZXMgb25seS4KCk5PTi1FWENMVVNJVkUgRElTVFJJQlVUSU9OIExJQ0VOU0UKCkJ5IHNpZ25pbmcgYW5kIHN1Ym1pdHRpbmcgdGhpcyBsaWNlbnNlLCB5b3UgKHRoZSBhdXRob3Iocykgb3IgY29weXJpZ2h0Cm93bmVyKSBncmFudHMgdG8gRFNwYWNlIFVuaXZlcnNpdHkgKERTVSkgdGhlIG5vbi1leGNsdXNpdmUgcmlnaHQgdG8gcmVwcm9kdWNlLAp0cmFuc2xhdGUgKGFzIGRlZmluZWQgYmVsb3cpLCBhbmQvb3IgZGlzdHJpYnV0ZSB5b3VyIHN1Ym1pc3Npb24gKGluY2x1ZGluZwp0aGUgYWJzdHJhY3QpIHdvcmxkd2lkZSBpbiBwcmludCBhbmQgZWxlY3Ryb25pYyBmb3JtYXQgYW5kIGluIGFueSBtZWRpdW0sCmluY2x1ZGluZyBidXQgbm90IGxpbWl0ZWQgdG8gYXVkaW8gb3IgdmlkZW8uCgpZb3UgYWdyZWUgdGhhdCBEU1UgbWF5LCB3aXRob3V0IGNoYW5naW5nIHRoZSBjb250ZW50LCB0cmFuc2xhdGUgdGhlCnN1Ym1pc3Npb24gdG8gYW55IG1lZGl1bSBvciBmb3JtYXQgZm9yIHRoZSBwdXJwb3NlIG9mIHByZXNlcnZhdGlvbi4KCllvdSBhbHNvIGFncmVlIHRoYXQgRFNVIG1heSBrZWVwIG1vcmUgdGhhbiBvbmUgY29weSBvZiB0aGlzIHN1Ym1pc3Npb24gZm9yCnB1cnBvc2VzIG9mIHNlY3VyaXR5LCBiYWNrLXVwIGFuZCBwcmVzZXJ2YXRpb24uCgpZb3UgcmVwcmVzZW50IHRoYXQgdGhlIHN1Ym1pc3Npb24gaXMgeW91ciBvcmlnaW5hbCB3b3JrLCBhbmQgdGhhdCB5b3UgaGF2ZQp0aGUgcmlnaHQgdG8gZ3JhbnQgdGhlIHJpZ2h0cyBjb250YWluZWQgaW4gdGhpcyBsaWNlbnNlLiBZb3UgYWxzbyByZXByZXNlbnQKdGhhdCB5b3VyIHN1Ym1pc3Npb24gZG9lcyBub3QsIHRvIHRoZSBiZXN0IG9mIHlvdXIga25vd2xlZGdlLCBpbmZyaW5nZSB1cG9uCmFueW9uZSdzIGNvcHlyaWdodC4KCklmIHRoZSBzdWJtaXNzaW9uIGNvbnRhaW5zIG1hdGVyaWFsIGZvciB3aGljaCB5b3UgZG8gbm90IGhvbGQgY29weXJpZ2h0LAp5b3UgcmVwcmVzZW50IHRoYXQgeW91IGhhdmUgb2J0YWluZWQgdGhlIHVucmVzdHJpY3RlZCBwZXJtaXNzaW9uIG9mIHRoZQpjb3B5cmlnaHQgb3duZXIgdG8gZ3JhbnQgRFNVIHRoZSByaWdodHMgcmVxdWlyZWQgYnkgdGhpcyBsaWNlbnNlLCBhbmQgdGhhdApzdWNoIHRoaXJkLXBhcnR5IG93bmVkIG1hdGVyaWFsIGlzIGNsZWFybHkgaWRlbnRpZmllZCBhbmQgYWNrbm93bGVkZ2VkCndpdGhpbiB0aGUgdGV4dCBvciBjb250ZW50IG9mIHRoZSBzdWJtaXNzaW9uLgoKSUYgVEhFIFNVQk1JU1NJT04gSVMgQkFTRUQgVVBPTiBXT1JLIFRIQVQgSEFTIEJFRU4gU1BPTlNPUkVEIE9SIFNVUFBPUlRFRApCWSBBTiBBR0VOQ1kgT1IgT1JHQU5JWkFUSU9OIE9USEVSIFRIQU4gRFNVLCBZT1UgUkVQUkVTRU5UIFRIQVQgWU9VIEhBVkUKRlVMRklMTEVEIEFOWSBSSUdIVCBPRiBSRVZJRVcgT1IgT1RIRVIgT0JMSUdBVElPTlMgUkVRVUlSRUQgQlkgU1VDSApDT05UUkFDVCBPUiBBR1JFRU1FTlQuCgpEU1Ugd2lsbCBjbGVhcmx5IGlkZW50aWZ5IHlvdXIgbmFtZShzKSBhcyB0aGUgYXV0aG9yKHMpIG9yIG93bmVyKHMpIG9mIHRoZQpzdWJtaXNzaW9uLCBhbmQgd2lsbCBub3QgbWFrZSBhbnkgYWx0ZXJhdGlvbiwgb3RoZXIgdGhhbiBhcyBhbGxvd2VkIGJ5IHRoaXMKbGljZW5zZSwgdG8geW91ciBzdWJtaXNzaW9uLgo=Repositório InstitucionalPUBhttp://www.repositorio.ufc.br/ri-oai/requestbu@ufc.br || repositorio@ufc.bropendoar:2025-04-08T13:58:45Repositório Institucional da Universidade Federal do Ceará (UFC) - Universidade Federal do Ceará (UFC)false |
| dc.title.pt_BR.fl_str_mv |
Leakage detection in a real water distribution network through a federated prototype-based model |
| dc.title.en.pt_BR.fl_str_mv |
Leakage detection in a real water distribution network through a federated prototype-based model |
| title |
Leakage detection in a real water distribution network through a federated prototype-based model |
| spellingShingle |
Leakage detection in a real water distribution network through a federated prototype-based model Sousa, Diego Perdigão CNPQ::ENGENHARIAS::ENGENHARIA ELETRICA Aprendizado federado Vazamentos de água - Detecção Modelos baseados em protótipos Água - Distribuição Federated learning Water leaks - Detection Prototype-based models Water - Distribution |
| title_short |
Leakage detection in a real water distribution network through a federated prototype-based model |
| title_full |
Leakage detection in a real water distribution network through a federated prototype-based model |
| title_fullStr |
Leakage detection in a real water distribution network through a federated prototype-based model |
| title_full_unstemmed |
Leakage detection in a real water distribution network through a federated prototype-based model |
| title_sort |
Leakage detection in a real water distribution network through a federated prototype-based model |
| author |
Sousa, Diego Perdigão |
| author_facet |
Sousa, Diego Perdigão |
| author_role |
author |
| dc.contributor.co-advisor.none.fl_str_mv |
Fischione, Carlo |
| dc.contributor.author.fl_str_mv |
Sousa, Diego Perdigão |
| dc.contributor.advisor1.fl_str_mv |
Cavalcante, Charles Casimiro |
| contributor_str_mv |
Cavalcante, Charles Casimiro |
| dc.subject.cnpq.fl_str_mv |
CNPQ::ENGENHARIAS::ENGENHARIA ELETRICA |
| topic |
CNPQ::ENGENHARIAS::ENGENHARIA ELETRICA Aprendizado federado Vazamentos de água - Detecção Modelos baseados em protótipos Água - Distribuição Federated learning Water leaks - Detection Prototype-based models Water - Distribution |
| dc.subject.ptbr.pt_BR.fl_str_mv |
Aprendizado federado Vazamentos de água - Detecção Modelos baseados em protótipos Água - Distribuição |
| dc.subject.en.pt_BR.fl_str_mv |
Federated learning Water leaks - Detection Prototype-based models Water - Distribution |
| description |
This thesis explores important an intricate trade-off in intelligent systems: the efficiency of leakage detection tasks, the preservation of sensitive information, and the interpretability of the intelligent system. Through a comprehensive analysis of these interconnected domains, we aim to design a low-cost algorithmic solution that can efficiently learn from industrial data collected by distributed sensors while preserving sensitive information. We focus on the following topics: (i) efficient leakage detection on water distribution networks (WDNs); and (ii) formulation of federated prototype-based models. In particular, this thesis mainly focuses on proposing an efficient and low-complexity distributed solution for identifying potential leaks in water distribution networks in municipal areas while ensuring the privacy of the hydraulic data. To this end, we explore and extend existing theories and methods from prototype-based learning and federated learning. We consider a hydraulic dataset, which includes water pressure and flow measurements obtained from pumps within district-metered areas in Stockholm, Sweden. In the first part of this thesis, we have considered a traditional learning paradigm, which contemplates the existence of a server responsible to collect and process all samples from every device and build machine learning models. Our goal is validating our assumption that prototypebased models (PBMs) can efficiently detect leakages in water distribution networks. The feature extraction, PBMs and clustering validation techniques we propose have low complexity, and the numerical experiments show the potential of using machine learning strategies in leakage detection for monitored water distribution networks. Specifically, our experiments show that the proposed learning strategies are able to obtain correct classification rates up to 93.98%. In the second part of this thesis, we have investigated the task of detecting leakages in WDNs by proposing a distributed approach to prototype-based models. In particular, we extended our initial formulation by designing a solution grounded on federated learning. Moreover, we further extended the previous analysis by including water flow measurements. Since we have utilized a federated modeling paradigm, we have considered that every device is responsible for processing their collected samples and generating local models. Meanwhile, the server only processes local models to building an efficient global model. Specifically, our experiments show that the proposed optimized learning method can obtain higher detection rates at each station than the conventional centralized approach, e.g., improvements of purity rates up to 7.6% in one of the pumping stations, which increased the minimum values from 92.13% obtained through centralized learning, to 99.11%, obtained via federated learning. |
| publishDate |
2024 |
| dc.date.issued.fl_str_mv |
2024 |
| dc.date.accessioned.fl_str_mv |
2025-04-08T13:35:54Z |
| dc.date.available.fl_str_mv |
2025-04-08T13:35:54Z |
| dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
| dc.type.driver.fl_str_mv |
info:eu-repo/semantics/doctoralThesis |
| format |
doctoralThesis |
| status_str |
publishedVersion |
| dc.identifier.citation.fl_str_mv |
SOUSA, Diego Perdigão. Leakage detection in a real water distribution network through a federated prototype-based model. 2024. 154 f. Tese (Doutorado em Engenharia de Teleinformática) – Centro de Tecnologia, Universidade Federal do Ceará, Fortaleza, 2024. |
| dc.identifier.uri.fl_str_mv |
http://repositorio.ufc.br/handle/riufc/80329 |
| identifier_str_mv |
SOUSA, Diego Perdigão. Leakage detection in a real water distribution network through a federated prototype-based model. 2024. 154 f. Tese (Doutorado em Engenharia de Teleinformática) – Centro de Tecnologia, Universidade Federal do Ceará, Fortaleza, 2024. |
| url |
http://repositorio.ufc.br/handle/riufc/80329 |
| dc.language.iso.fl_str_mv |
eng |
| language |
eng |
| dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
| eu_rights_str_mv |
openAccess |
| dc.source.none.fl_str_mv |
reponame:Repositório Institucional da Universidade Federal do Ceará (UFC) instname:Universidade Federal do Ceará (UFC) instacron:UFC |
| instname_str |
Universidade Federal do Ceará (UFC) |
| instacron_str |
UFC |
| institution |
UFC |
| reponame_str |
Repositório Institucional da Universidade Federal do Ceará (UFC) |
| collection |
Repositório Institucional da Universidade Federal do Ceará (UFC) |
| bitstream.url.fl_str_mv |
http://repositorio.ufc.br/bitstream/riufc/80329/1/2024_dpsousa.pdf http://repositorio.ufc.br/bitstream/riufc/80329/2/license.txt |
| bitstream.checksum.fl_str_mv |
ec0e8d875e55e03500c5e3281308af64 8a4605be74aa9ea9d79846c1fba20a33 |
| bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 |
| repository.name.fl_str_mv |
Repositório Institucional da Universidade Federal do Ceará (UFC) - Universidade Federal do Ceará (UFC) |
| repository.mail.fl_str_mv |
bu@ufc.br || repositorio@ufc.br |
| _version_ |
1847793307703312384 |