你好,游客 登录 注册 发布搜索
背景:
阅读新闻

[PDF]Accelerating Iterative Big Data Computing Through MPI

[日期:2015-03-15] 来源:Computer Science and Technology  作者:Fan Liang Xiaoyi Lu [字体: ]

Accelerating Iterative Big Data Computing Through MPI

Fan Liang  Xiaoyi Lu

In this paper, we first analyze the overhead of shuffle operation in Hadoop and Spark when running PageRank workload, and then propose an event-driven pipeline and in-memory shuffle design with better overlapping of computation and communication as DataMPIIteration, an MPI-based library, for iterative big data computing. Our performance evaluation shows DataMPI-Iteration can achieve 9X~21X speedup over Apache Hadoop, and 2X~3X speedup over Apache Spark for PageRank and K-means.


Accelerating Iterative Big Data Computing Through

 

 

收藏 推荐 打印 | 录入:574107552 | 阅读:
本文评论   查看全部评论 (0)
表情: 表情 姓名: 字数
点评:
       
评论声明
  • 尊重网上道德,遵守中华人民共和国的各项有关法律法规
  • 承担一切因您的行为而直接或间接导致的民事或刑事法律责任
  • 本站管理人员有权保留或删除其管辖留言中的任意内容
  • 本站有权在网站内转载或引用您的评论
  • 参与本评论即表明您已经阅读并接受上述条款