Exportação concluída — 

Using deep neural networks for failure prediction in hard disk drives

Detalhes bibliográficos
Ano de defesa: 2018
Autor(a) principal: Lima, Fernando Dione dos Santos
Orientador(a): Gomes, João Paulo Pordeus
Banca de defesa: Não Informado pela instituição
Tipo de documento: Dissertação
Tipo de acesso: Acesso aberto
Idioma: eng
Instituição de defesa: Não Informado pela instituição
Programa de Pós-Graduação: Não Informado pela instituição
Departamento: Não Informado pela instituição
País: Não Informado pela instituição
Palavras-chave em Português:
Link de acesso: http://www.repositorio.ufc.br/handle/riufc/49628
Resumo: Hard disk drives (HDDs) are still the most widely used storage technology employed in large-scale storage systems. This is mainly a result of its excellent cost-benefit relation in terms of cost per gigabyte. Several research efforts have been done to propose early failure detection techniques for these devices in order to improve storage systems availability and avoid data loss. Failure prediction in such circumstances would allow for the reduction of downtime costs through anticipated disk replacements, as well as the migration of data to new devices, avoiding data loss. Many of the techniques proposed so far mainly perform incipient failure detection thus not allowing for proper planning of such maintenance tasks. In this work, we present several remaining useful life (RUL) estimation approaches for hard disk drives based on SMART parameters. These approaches include two different modelings of the problem. The first is a traditional regression-based that allows for fine-grained predictions. The other approach allows for a greater control over the granularity needed by the systems operator, through a previous configuration, and consists in the modeling of the problem as a multiclass, or multinomial, classification task. In the context of the classification problem, we also explore two important aspects of the RUL estimation task: the ordinality between classes, and the predictive bias towards classes that indicate a reduced device lifetime, when an incorrect prediction takes place. All models are based on Deep Neural Networks (DNNs). For evaluating the models, a dataset produced by a cloud storage service provider, and including the complete time-series for 1,697 failing devices, was employed. Experiments showed that the proposed methods produced satisfying results in the regression-based task when assessed with metrics well-suited and designed specifically for prognostics tasks. In the modeling as a traditional classification task, our model produced superior results to the baseline model and, in the asymmetric and ordinal classification task, it outperformed baselines in metrics where both ordinality and asymmetry were taken into account.
id UFC-7_fd2845fc4473895d089fdd247ffc2e09
oai_identifier_str oai:repositorio.ufc.br:riufc/49628
network_acronym_str UFC-7
network_name_str Repositório Institucional da Universidade Federal do Ceará (UFC)
repository_id_str
spelling Lima, Fernando Dione dos SantosMachado, Javam de CastroGomes, João Paulo Pordeus2020-01-24T12:59:21Z2020-01-24T12:59:21Z2018LIMA, Fernando Dione dos Santos. Using deep neural networks for failure prediction in hard disk drives. 2018. 78 f. Dissertação (Mestrado em Ciência da Computação) - Universidade Federal do Ceará, Fortaleza, 2018.http://www.repositorio.ufc.br/handle/riufc/49628Hard disk drives (HDDs) are still the most widely used storage technology employed in large-scale storage systems. This is mainly a result of its excellent cost-benefit relation in terms of cost per gigabyte. Several research efforts have been done to propose early failure detection techniques for these devices in order to improve storage systems availability and avoid data loss. Failure prediction in such circumstances would allow for the reduction of downtime costs through anticipated disk replacements, as well as the migration of data to new devices, avoiding data loss. Many of the techniques proposed so far mainly perform incipient failure detection thus not allowing for proper planning of such maintenance tasks. In this work, we present several remaining useful life (RUL) estimation approaches for hard disk drives based on SMART parameters. These approaches include two different modelings of the problem. The first is a traditional regression-based that allows for fine-grained predictions. The other approach allows for a greater control over the granularity needed by the systems operator, through a previous configuration, and consists in the modeling of the problem as a multiclass, or multinomial, classification task. In the context of the classification problem, we also explore two important aspects of the RUL estimation task: the ordinality between classes, and the predictive bias towards classes that indicate a reduced device lifetime, when an incorrect prediction takes place. All models are based on Deep Neural Networks (DNNs). For evaluating the models, a dataset produced by a cloud storage service provider, and including the complete time-series for 1,697 failing devices, was employed. Experiments showed that the proposed methods produced satisfying results in the regression-based task when assessed with metrics well-suited and designed specifically for prognostics tasks. In the modeling as a traditional classification task, our model produced superior results to the baseline model and, in the asymmetric and ordinal classification task, it outperformed baselines in metrics where both ordinality and asymmetry were taken into account.Discos rígidos (HDDs) ainda são a tecnologia de armazenamento mais amplamente utilizada em sistemas de armazenamento de larga escala. Isso se deve principalmente à sua relação custo-benefício em termos de custo por volume de armazenamento. Várias pesquisas foram feitas para propor técnicas de detecção antecipada de falhas para estes dispositivos, visando aumentar a disponibilidade dos sistemas de armazenamento e evitar a perda de dados. A previsão de falhas em tais circunstâncias permitiria a redução dos custos decorrentes do tempo de indisponibilidade desses sistemas por meio de substituições antecipadas dos dispositivos, além da migração de dados para estes novos dispositivos, evitando perdas de dados. Muitas das técnicas propostas até agora realizam principalmente a detecção incipiente de falhas, não permitindo o planejamento adequado de tais tarefas de manutenção. Neste trabalho, apresentamos várias abordagens de estimativa de vida útil remanescente (RUL) para discos rígidos baseados em parâmetros SMART. Tais abordagens incluem duas diferentes modelagens para o problema. A primeira modelagem é a mais tradicional, baseada em regressão e que propicia uma fina granularidade na predição. A outra abordagem possibilita um maior controle sobre a granularidade necessária pelo operador, através de configuração prévia, e consiste na modelagem do problema como uma tarefa de classificação multiclasse, ou multinomial. No contexto do problema de classificação, exploramos também dois aspectos importantes para o problema de estimativa de RUL: a ordinalidade entre classes, e o viés preditivo para classes que indicam um tempo de vida reduzido, quando da ocorrência de classificações errôneas. Isso se dá através da aplicação de uma codificação ajustável para as classes. Todos os modelos propostos são baseados em redes neurais profundas (DNNs). Na avaliação dos modelos, um conjunto de dados proveniente de uma companhia de armazenamento de dados em nuvem, e contendo amostras de 1,697 dispositivos que falharam, foi utilizado. Experimentos mostraram que as abordagens propostas propiciam resultado satisfatórios na modelagem como problema de regressão, onde foi realizada uma análise aplicando métricas desenhadas para tarefas de prognóstico. Na modelagem como problema de classificação tradicional nosso modelo obteve resultados superiores ao modelo concorrente e, no problema de classificação assimétrica, melhora nas métricas que abordam a ordinalidade juntamente com a assimetria.HDD failure predictionDeep learningRUL estimationUsing deep neural networks for failure prediction in hard disk drivesUsing deep neural networks for failure prediction in hard disk drivesinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesisengreponame:Repositório Institucional da Universidade Federal do Ceará (UFC)instname:Universidade Federal do Ceará (UFC)instacron:UFCinfo:eu-repo/semantics/openAccessLICENSElicense.txtlicense.txttext/plain; charset=utf-81748http://repositorio.ufc.br/bitstream/riufc/49628/4/license.txt8a4605be74aa9ea9d79846c1fba20a33MD54ORIGINAL2018_dis_fdslima.pdf2018_dis_fdslima.pdfapplication/pdf1146644http://repositorio.ufc.br/bitstream/riufc/49628/3/2018_dis_fdslima.pdfaf91f6b47eac83b6edd24ff6c970e2a9MD53riufc/496282020-01-24 09:59:21.839oai:repositorio.ufc.br:riufc/49628Tk9URTogUExBQ0UgWU9VUiBPV04gTElDRU5TRSBIRVJFClRoaXMgc2FtcGxlIGxpY2Vuc2UgaXMgcHJvdmlkZWQgZm9yIGluZm9ybWF0aW9uYWwgcHVycG9zZXMgb25seS4KCk5PTi1FWENMVVNJVkUgRElTVFJJQlVUSU9OIExJQ0VOU0UKCkJ5IHNpZ25pbmcgYW5kIHN1Ym1pdHRpbmcgdGhpcyBsaWNlbnNlLCB5b3UgKHRoZSBhdXRob3Iocykgb3IgY29weXJpZ2h0Cm93bmVyKSBncmFudHMgdG8gRFNwYWNlIFVuaXZlcnNpdHkgKERTVSkgdGhlIG5vbi1leGNsdXNpdmUgcmlnaHQgdG8gcmVwcm9kdWNlLAp0cmFuc2xhdGUgKGFzIGRlZmluZWQgYmVsb3cpLCBhbmQvb3IgZGlzdHJpYnV0ZSB5b3VyIHN1Ym1pc3Npb24gKGluY2x1ZGluZwp0aGUgYWJzdHJhY3QpIHdvcmxkd2lkZSBpbiBwcmludCBhbmQgZWxlY3Ryb25pYyBmb3JtYXQgYW5kIGluIGFueSBtZWRpdW0sCmluY2x1ZGluZyBidXQgbm90IGxpbWl0ZWQgdG8gYXVkaW8gb3IgdmlkZW8uCgpZb3UgYWdyZWUgdGhhdCBEU1UgbWF5LCB3aXRob3V0IGNoYW5naW5nIHRoZSBjb250ZW50LCB0cmFuc2xhdGUgdGhlCnN1Ym1pc3Npb24gdG8gYW55IG1lZGl1bSBvciBmb3JtYXQgZm9yIHRoZSBwdXJwb3NlIG9mIHByZXNlcnZhdGlvbi4KCllvdSBhbHNvIGFncmVlIHRoYXQgRFNVIG1heSBrZWVwIG1vcmUgdGhhbiBvbmUgY29weSBvZiB0aGlzIHN1Ym1pc3Npb24gZm9yCnB1cnBvc2VzIG9mIHNlY3VyaXR5LCBiYWNrLXVwIGFuZCBwcmVzZXJ2YXRpb24uCgpZb3UgcmVwcmVzZW50IHRoYXQgdGhlIHN1Ym1pc3Npb24gaXMgeW91ciBvcmlnaW5hbCB3b3JrLCBhbmQgdGhhdCB5b3UgaGF2ZQp0aGUgcmlnaHQgdG8gZ3JhbnQgdGhlIHJpZ2h0cyBjb250YWluZWQgaW4gdGhpcyBsaWNlbnNlLiBZb3UgYWxzbyByZXByZXNlbnQKdGhhdCB5b3VyIHN1Ym1pc3Npb24gZG9lcyBub3QsIHRvIHRoZSBiZXN0IG9mIHlvdXIga25vd2xlZGdlLCBpbmZyaW5nZSB1cG9uCmFueW9uZSdzIGNvcHlyaWdodC4KCklmIHRoZSBzdWJtaXNzaW9uIGNvbnRhaW5zIG1hdGVyaWFsIGZvciB3aGljaCB5b3UgZG8gbm90IGhvbGQgY29weXJpZ2h0LAp5b3UgcmVwcmVzZW50IHRoYXQgeW91IGhhdmUgb2J0YWluZWQgdGhlIHVucmVzdHJpY3RlZCBwZXJtaXNzaW9uIG9mIHRoZQpjb3B5cmlnaHQgb3duZXIgdG8gZ3JhbnQgRFNVIHRoZSByaWdodHMgcmVxdWlyZWQgYnkgdGhpcyBsaWNlbnNlLCBhbmQgdGhhdApzdWNoIHRoaXJkLXBhcnR5IG93bmVkIG1hdGVyaWFsIGlzIGNsZWFybHkgaWRlbnRpZmllZCBhbmQgYWNrbm93bGVkZ2VkCndpdGhpbiB0aGUgdGV4dCBvciBjb250ZW50IG9mIHRoZSBzdWJtaXNzaW9uLgoKSUYgVEhFIFNVQk1JU1NJT04gSVMgQkFTRUQgVVBPTiBXT1JLIFRIQVQgSEFTIEJFRU4gU1BPTlNPUkVEIE9SIFNVUFBPUlRFRApCWSBBTiBBR0VOQ1kgT1IgT1JHQU5JWkFUSU9OIE9USEVSIFRIQU4gRFNVLCBZT1UgUkVQUkVTRU5UIFRIQVQgWU9VIEhBVkUKRlVMRklMTEVEIEFOWSBSSUdIVCBPRiBSRVZJRVcgT1IgT1RIRVIgT0JMSUdBVElPTlMgUkVRVUlSRUQgQlkgU1VDSApDT05UUkFDVCBPUiBBR1JFRU1FTlQuCgpEU1Ugd2lsbCBjbGVhcmx5IGlkZW50aWZ5IHlvdXIgbmFtZShzKSBhcyB0aGUgYXV0aG9yKHMpIG9yIG93bmVyKHMpIG9mIHRoZQpzdWJtaXNzaW9uLCBhbmQgd2lsbCBub3QgbWFrZSBhbnkgYWx0ZXJhdGlvbiwgb3RoZXIgdGhhbiBhcyBhbGxvd2VkIGJ5IHRoaXMKbGljZW5zZSwgdG8geW91ciBzdWJtaXNzaW9uLgo=Repositório InstitucionalPUBhttp://www.repositorio.ufc.br/ri-oai/requestbu@ufc.br || repositorio@ufc.bropendoar:2020-01-24T12:59:21Repositório Institucional da Universidade Federal do Ceará (UFC) - Universidade Federal do Ceará (UFC)false
dc.title.pt_BR.fl_str_mv Using deep neural networks for failure prediction in hard disk drives
dc.title.en.pt_BR.fl_str_mv Using deep neural networks for failure prediction in hard disk drives
title Using deep neural networks for failure prediction in hard disk drives
spellingShingle Using deep neural networks for failure prediction in hard disk drives
Lima, Fernando Dione dos Santos
HDD failure prediction
Deep learning
RUL estimation
title_short Using deep neural networks for failure prediction in hard disk drives
title_full Using deep neural networks for failure prediction in hard disk drives
title_fullStr Using deep neural networks for failure prediction in hard disk drives
title_full_unstemmed Using deep neural networks for failure prediction in hard disk drives
title_sort Using deep neural networks for failure prediction in hard disk drives
author Lima, Fernando Dione dos Santos
author_facet Lima, Fernando Dione dos Santos
author_role author
dc.contributor.co-advisor.none.fl_str_mv Machado, Javam de Castro
dc.contributor.author.fl_str_mv Lima, Fernando Dione dos Santos
dc.contributor.advisor1.fl_str_mv Gomes, João Paulo Pordeus
contributor_str_mv Gomes, João Paulo Pordeus
dc.subject.por.fl_str_mv HDD failure prediction
Deep learning
RUL estimation
topic HDD failure prediction
Deep learning
RUL estimation
description Hard disk drives (HDDs) are still the most widely used storage technology employed in large-scale storage systems. This is mainly a result of its excellent cost-benefit relation in terms of cost per gigabyte. Several research efforts have been done to propose early failure detection techniques for these devices in order to improve storage systems availability and avoid data loss. Failure prediction in such circumstances would allow for the reduction of downtime costs through anticipated disk replacements, as well as the migration of data to new devices, avoiding data loss. Many of the techniques proposed so far mainly perform incipient failure detection thus not allowing for proper planning of such maintenance tasks. In this work, we present several remaining useful life (RUL) estimation approaches for hard disk drives based on SMART parameters. These approaches include two different modelings of the problem. The first is a traditional regression-based that allows for fine-grained predictions. The other approach allows for a greater control over the granularity needed by the systems operator, through a previous configuration, and consists in the modeling of the problem as a multiclass, or multinomial, classification task. In the context of the classification problem, we also explore two important aspects of the RUL estimation task: the ordinality between classes, and the predictive bias towards classes that indicate a reduced device lifetime, when an incorrect prediction takes place. All models are based on Deep Neural Networks (DNNs). For evaluating the models, a dataset produced by a cloud storage service provider, and including the complete time-series for 1,697 failing devices, was employed. Experiments showed that the proposed methods produced satisfying results in the regression-based task when assessed with metrics well-suited and designed specifically for prognostics tasks. In the modeling as a traditional classification task, our model produced superior results to the baseline model and, in the asymmetric and ordinal classification task, it outperformed baselines in metrics where both ordinality and asymmetry were taken into account.
publishDate 2018
dc.date.issued.fl_str_mv 2018
dc.date.accessioned.fl_str_mv 2020-01-24T12:59:21Z
dc.date.available.fl_str_mv 2020-01-24T12:59:21Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/masterThesis
format masterThesis
status_str publishedVersion
dc.identifier.citation.fl_str_mv LIMA, Fernando Dione dos Santos. Using deep neural networks for failure prediction in hard disk drives. 2018. 78 f. Dissertação (Mestrado em Ciência da Computação) - Universidade Federal do Ceará, Fortaleza, 2018.
dc.identifier.uri.fl_str_mv http://www.repositorio.ufc.br/handle/riufc/49628
identifier_str_mv LIMA, Fernando Dione dos Santos. Using deep neural networks for failure prediction in hard disk drives. 2018. 78 f. Dissertação (Mestrado em Ciência da Computação) - Universidade Federal do Ceará, Fortaleza, 2018.
url http://www.repositorio.ufc.br/handle/riufc/49628
dc.language.iso.fl_str_mv eng
language eng
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.source.none.fl_str_mv reponame:Repositório Institucional da Universidade Federal do Ceará (UFC)
instname:Universidade Federal do Ceará (UFC)
instacron:UFC
instname_str Universidade Federal do Ceará (UFC)
instacron_str UFC
institution UFC
reponame_str Repositório Institucional da Universidade Federal do Ceará (UFC)
collection Repositório Institucional da Universidade Federal do Ceará (UFC)
bitstream.url.fl_str_mv http://repositorio.ufc.br/bitstream/riufc/49628/4/license.txt
http://repositorio.ufc.br/bitstream/riufc/49628/3/2018_dis_fdslima.pdf
bitstream.checksum.fl_str_mv 8a4605be74aa9ea9d79846c1fba20a33
af91f6b47eac83b6edd24ff6c970e2a9
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
repository.name.fl_str_mv Repositório Institucional da Universidade Federal do Ceará (UFC) - Universidade Federal do Ceará (UFC)
repository.mail.fl_str_mv bu@ufc.br || repositorio@ufc.br
_version_ 1847793235053772800