A unified mapreduce programming interface for multi-core and distributed architectures

Detalhes bibliográficos
Ano de defesa: 2015
Autor(a) principal: Adornes, Daniel Couto lattes
Orientador(a): Fernandes, Luiz Gustavo Le?o lattes
Banca de defesa: Não Informado pela instituição
Tipo de documento: Dissertação
Tipo de acesso: Acesso aberto
Idioma: eng
Instituição de defesa: Pontif?cia Universidade Cat?lica do Rio Grande do Sul
Programa de Pós-Graduação: Programa de P?s-Gradua??o em Ci?ncia da Computa??o
Departamento: Faculdade de Inform?tica
País: Brasil
Palavras-chave em Português:
MEM?RIA COMPARTILHADA DISTRIBU?DA
PROCESSAMENTO PARALELO
PROCESSAMENTO DISTRIBU?DO
INFORM?TICA
Área do conhecimento CNPq:
CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO
Link de acesso: http://tede2.pucrs.br/tede2/handle/tede/6782
Resumo: In order to improve performance, simplicity and scalability of large datasets processing, Google proposed the MapReduce parallel pattern. This pattern has been implemented in several ways for different architectural levels, achieving significant results for high performance computing. However, developing optimized code with those solutions requires specialized knowledge in each framework?s interface and programming language. Recently, the DSL-POPP was proposed as a framework with a high-level language for patternsoriented parallel programming, aimed at abstracting complexities of parallel and distributed code. Inspired on DSL-POPP, this work proposes the implementation of a unified MapReduce programming interface with rules for code transformation to optimized solutions for shared-memory multi-core and distributed architectures. The evaluation demonstrates that the proposed interface is able to avoid performance losses, while also achieving a code and a development cost reduction from 41.84% to 96.48%. Moreover, the construction of the code generator, the compatibility with other MapReduce solutions and the extension of DSL-POPP with the MapReduce pattern are proposed as future work.
id P_RS_dab805d72ce5eecc6eb96d8eaeb94339
oai_identifier_str oai:tede2.pucrs.br:tede/6782
network_acronym_str P_RS
network_name_str Biblioteca Digital de Teses e Dissertações da PUC_RS
repository_id_str
spelling Fernandes, Luiz Gustavo Le?o571.500.100-59http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4784653A5008.265.150-70http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4973183U8Adornes, Daniel Couto2016-06-22T19:44:58Z2015-03-31http://tede2.pucrs.br/tede2/handle/tede/6782In order to improve performance, simplicity and scalability of large datasets processing, Google proposed the MapReduce parallel pattern. This pattern has been implemented in several ways for different architectural levels, achieving significant results for high performance computing. However, developing optimized code with those solutions requires specialized knowledge in each framework?s interface and programming language. Recently, the DSL-POPP was proposed as a framework with a high-level language for patternsoriented parallel programming, aimed at abstracting complexities of parallel and distributed code. Inspired on DSL-POPP, this work proposes the implementation of a unified MapReduce programming interface with rules for code transformation to optimized solutions for shared-memory multi-core and distributed architectures. The evaluation demonstrates that the proposed interface is able to avoid performance losses, while also achieving a code and a development cost reduction from 41.84% to 96.48%. Moreover, the construction of the code generator, the compatibility with other MapReduce solutions and the extension of DSL-POPP with the MapReduce pattern are proposed as future work.Visando melhoria de performance, simplicidade e escalabilidade no processamento de dados amplos, o Google prop?s o padr?o paralelo MapReduce. Este padr?o tem sido implementado de variadas formas para diferentes n?veis de arquitetura, alcan?ando resultados significativos com respeito a computa??o de alto desempenho. No entanto, desenvolver c?digo otimizado com tais solu??es requer conhecimento especializado na interface e na linguagem de programa??o de cada solu??o. Recentemente, a DSL-POPP foi proposta como uma solu??o de linguagem de programa??o de alto n?vel para programa??o paralela orientada a padr?es, visando abstrair as complexidades envolvidas em programa??o paralela e distribu?da. Inspirado na DSL-POPP, este trabalho prop?e a implementa??o de uma interface unificada de programa??o MapReduce com regras para transforma??o de c?digo para solu??es otimizadas para arquiteturas multi-core de mem?ria compartilhada e distribu?da. A avalia??o demonstra que a interface proposta ? capaz de evitar perdas de performance, enquanto alcan?a uma redu??o de c?digo e esfor?o de programa??o de 41,84% a 96,48%. Ademais, a constru??o do gerador de c?digo, a compatibilidade com outras solu??es MapReduce e a extens?o da DSL-POPP com o padr?o MapReduce s?o propostas para trabalhos futuros.Submitted by Setor de Tratamento da Informa??o - BC/PUCRS (tede2@pucrs.br) on 2016-06-22T19:44:58Z No. of bitstreams: 1 DIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf: 1894086 bytes, checksum: f87c59fa92f43ed62efaafd9c724ed8d (MD5)Made available in DSpace on 2016-06-22T19:44:58Z (GMT). No. of bitstreams: 1 DIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf: 1894086 bytes, checksum: f87c59fa92f43ed62efaafd9c724ed8d (MD5) Previous issue date: 2015-03-31Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior - CAPESapplication/pdfhttp://tede2.pucrs.br:80/tede2/retrieve/165444/DIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf.jpgengPontif?cia Universidade Cat?lica do Rio Grande do SulPrograma de P?s-Gradua??o em Ci?ncia da Computa??oPUCRSBrasilFaculdade de Inform?ticaMEM?RIA COMPARTILHADA DISTRIBU?DAPROCESSAMENTO PARALELOPROCESSAMENTO DISTRIBU?DOINFORM?TICACIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAOA unified mapreduce programming interface for multi-core and distributed architecturesUma interface de programa??o mapreduce unificada para arquiteturas multi-core e distribu?dainfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesis1974996533081274470600600600600-300854251040114914436717112058112045092075167498588264571info:eu-repo/semantics/openAccessreponame:Biblioteca Digital de Teses e Dissertações da PUC_RSinstname:Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS)instacron:PUC_RSTHUMBNAILDIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf.jpgDIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf.jpgimage/jpeg3935http://tede2.pucrs.br/tede2/bitstream/tede/6782/5/DIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf.jpg7b324fab29797b98abb30bf17f363da5MD55TEXTDIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf.txtDIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf.txttext/plain200155http://tede2.pucrs.br/tede2/bitstream/tede/6782/4/DIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf.txt8c82eeb75e9b11a867ed0cddf1056d32MD54LICENSElicense.txtlicense.txttext/plain; charset=utf-8610http://tede2.pucrs.br/tede2/bitstream/tede/6782/3/license.txt5a9d6006225b368ef605ba16b4f6d1beMD53ORIGINALDIS_DANIEL_COUTO_ADORNES_COMPLETO.pdfDIS_DANIEL_COUTO_ADORNES_COMPLETO.pdfapplication/pdf1894086http://tede2.pucrs.br/tede2/bitstream/tede/6782/2/DIS_DANIEL_COUTO_ADORNES_COMPLETO.pdff87c59fa92f43ed62efaafd9c724ed8dMD52tede/67822016-06-22 20:00:31.553oai:tede2.pucrs.br:tede/6782QXV0b3JpemHDp8OjbyBwYXJhIFB1YmxpY2HDp8OjbyBFbGV0csO0bmljYTogQ29tIGJhc2Ugbm8gZGlzcG9zdG8gbmEgTGVpIEZlZGVyYWwgbsK6OS42MTAsIGRlIDE5IGRlIGZldmVyZWlybyBkZSAxOTk4LCBvIGF1dG9yIEFVVE9SSVpBIGEgcHVibGljYcOnw6NvIGVsZXRyw7RuaWNhIGRhIHByZXNlbnRlIG9icmEgbm8gYWNlcnZvIGRhIEJpYmxpb3RlY2EgRGlnaXRhbCBkYSBQb250aWbDrWNpYSBVbml2ZXJzaWRhZGUgQ2F0w7NsaWNhIGRvIFJpbyBHcmFuZGUgZG8gU3VsLCBzZWRpYWRhIGEgQXYuIElwaXJhbmdhIDY2ODEsIFBvcnRvIEFsZWdyZSwgUmlvIEdyYW5kZSBkbyBTdWwsIGNvbSByZWdpc3RybyBkZSBDTlBKIDg4NjMwNDEzMDAwMi04MSBiZW0gY29tbyBlbSBvdXRyYXMgYmlibGlvdGVjYXMgZGlnaXRhaXMsIG5hY2lvbmFpcyBlIGludGVybmFjaW9uYWlzLCBjb25zw7NyY2lvcyBlIHJlZGVzIMOgcyBxdWFpcyBhIGJpYmxpb3RlY2EgZGEgUFVDUlMgcG9zc2EgYSB2aXIgcGFydGljaXBhciwgc2VtIMO0bnVzIGFsdXNpdm8gYW9zIGRpcmVpdG9zIGF1dG9yYWlzLCBhIHTDrXR1bG8gZGUgZGl2dWxnYcOnw6NvIGRhIHByb2R1w6fDo28gY2llbnTDrWZpY2EuCg==Biblioteca Digital de Teses e Dissertaçõeshttp://tede2.pucrs.br/tede2/PRIhttps://tede2.pucrs.br/oai/requestbiblioteca.central@pucrs.br||opendoar:2016-06-22T23:00:31Biblioteca Digital de Teses e Dissertações da PUC_RS - Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS)false
dc.title.por.fl_str_mv A unified mapreduce programming interface for multi-core and distributed architectures
dc.title.alternative.por.fl_str_mv Uma interface de programa??o mapreduce unificada para arquiteturas multi-core e distribu?da
title A unified mapreduce programming interface for multi-core and distributed architectures
spellingShingle A unified mapreduce programming interface for multi-core and distributed architectures
Adornes, Daniel Couto
MEM?RIA COMPARTILHADA DISTRIBU?DA
PROCESSAMENTO PARALELO
PROCESSAMENTO DISTRIBU?DO
INFORM?TICA
CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO
title_short A unified mapreduce programming interface for multi-core and distributed architectures
title_full A unified mapreduce programming interface for multi-core and distributed architectures
title_fullStr A unified mapreduce programming interface for multi-core and distributed architectures
title_full_unstemmed A unified mapreduce programming interface for multi-core and distributed architectures
title_sort A unified mapreduce programming interface for multi-core and distributed architectures
author Adornes, Daniel Couto
author_facet Adornes, Daniel Couto
author_role author
dc.contributor.advisor1.fl_str_mv Fernandes, Luiz Gustavo Le?o
dc.contributor.advisor1ID.fl_str_mv 571.500.100-59
dc.contributor.advisor1Lattes.fl_str_mv http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4784653A5
dc.contributor.authorID.fl_str_mv 008.265.150-70
dc.contributor.authorLattes.fl_str_mv http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4973183U8
dc.contributor.author.fl_str_mv Adornes, Daniel Couto
contributor_str_mv Fernandes, Luiz Gustavo Le?o
dc.subject.por.fl_str_mv MEM?RIA COMPARTILHADA DISTRIBU?DA
PROCESSAMENTO PARALELO
PROCESSAMENTO DISTRIBU?DO
INFORM?TICA
topic MEM?RIA COMPARTILHADA DISTRIBU?DA
PROCESSAMENTO PARALELO
PROCESSAMENTO DISTRIBU?DO
INFORM?TICA
CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO
dc.subject.cnpq.fl_str_mv CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO
description In order to improve performance, simplicity and scalability of large datasets processing, Google proposed the MapReduce parallel pattern. This pattern has been implemented in several ways for different architectural levels, achieving significant results for high performance computing. However, developing optimized code with those solutions requires specialized knowledge in each framework?s interface and programming language. Recently, the DSL-POPP was proposed as a framework with a high-level language for patternsoriented parallel programming, aimed at abstracting complexities of parallel and distributed code. Inspired on DSL-POPP, this work proposes the implementation of a unified MapReduce programming interface with rules for code transformation to optimized solutions for shared-memory multi-core and distributed architectures. The evaluation demonstrates that the proposed interface is able to avoid performance losses, while also achieving a code and a development cost reduction from 41.84% to 96.48%. Moreover, the construction of the code generator, the compatibility with other MapReduce solutions and the extension of DSL-POPP with the MapReduce pattern are proposed as future work.
publishDate 2015
dc.date.issued.fl_str_mv 2015-03-31
dc.date.accessioned.fl_str_mv 2016-06-22T19:44:58Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/masterThesis
format masterThesis
status_str publishedVersion
dc.identifier.uri.fl_str_mv http://tede2.pucrs.br/tede2/handle/tede/6782
url http://tede2.pucrs.br/tede2/handle/tede/6782
dc.language.iso.fl_str_mv eng
language eng
dc.relation.program.fl_str_mv 1974996533081274470
dc.relation.confidence.fl_str_mv 600
600
600
600
dc.relation.department.fl_str_mv -3008542510401149144
dc.relation.cnpq.fl_str_mv 3671711205811204509
dc.relation.sponsorship.fl_str_mv 2075167498588264571
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Pontif?cia Universidade Cat?lica do Rio Grande do Sul
dc.publisher.program.fl_str_mv Programa de P?s-Gradua??o em Ci?ncia da Computa??o
dc.publisher.initials.fl_str_mv PUCRS
dc.publisher.country.fl_str_mv Brasil
dc.publisher.department.fl_str_mv Faculdade de Inform?tica
publisher.none.fl_str_mv Pontif?cia Universidade Cat?lica do Rio Grande do Sul
dc.source.none.fl_str_mv reponame:Biblioteca Digital de Teses e Dissertações da PUC_RS
instname:Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS)
instacron:PUC_RS
instname_str Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS)
instacron_str PUC_RS
institution PUC_RS
reponame_str Biblioteca Digital de Teses e Dissertações da PUC_RS
collection Biblioteca Digital de Teses e Dissertações da PUC_RS
bitstream.url.fl_str_mv http://tede2.pucrs.br/tede2/bitstream/tede/6782/5/DIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf.jpg
http://tede2.pucrs.br/tede2/bitstream/tede/6782/4/DIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf.txt
http://tede2.pucrs.br/tede2/bitstream/tede/6782/3/license.txt
http://tede2.pucrs.br/tede2/bitstream/tede/6782/2/DIS_DANIEL_COUTO_ADORNES_COMPLETO.pdf
bitstream.checksum.fl_str_mv 7b324fab29797b98abb30bf17f363da5
8c82eeb75e9b11a867ed0cddf1056d32
5a9d6006225b368ef605ba16b4f6d1be
f87c59fa92f43ed62efaafd9c724ed8d
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
MD5
MD5
repository.name.fl_str_mv Biblioteca Digital de Teses e Dissertações da PUC_RS - Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS)
repository.mail.fl_str_mv biblioteca.central@pucrs.br||
_version_ 1796793220712103936

Registros relacionados