A Dwarf-based Scalable Big Data Benchmarking Methodology release_t6vgjqxomrhbdju225pfc5yreu

by Wanling Gao, Lei Wang, Jianfeng Zhan, Chunjie Luo, Daoyi Zheng, Zhen Jia, Biwei Xie, Chen Zheng, Qiang Yang, Haibin Wang

Released as a article .

2017  

Abstract

Different from the traditional benchmarking methodology that creates a new benchmark or proxy for every possible workload, this paper presents a scalable big data benchmarking methodology. Among a wide variety of big data analytics workloads, we identify eight big data dwarfs, each of which captures the common requirements of each class of unit of computation while being reasonably divorced from individual implementations. We implement the eight dwarfs on different software stacks, e.g., OpenMP, MPI, Hadoop as the dwarf components. For the purpose of architecture simulation, we construct and tune big data proxy benchmarks using the directed acyclic graph (DAG)-like combinations of the dwarf components with different weights to mimic the benchmarks in BigDataBench. Our proxy benchmarks preserve the micro-architecture, memory, and I/O characteristics, and they shorten the simulation time by 100s times while maintain the average micro-architectural data accuracy above 90 percentage on both X86 64 and ARMv8 processors. We will open-source the big data dwarf components and proxy benchmarks soon.
In text/plain format

Archived Files and Locations

application/pdf  1.7 MB
file_yu73c26vw5bg3ic6wiodiy2c6e
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2017-11-09
Version   v1
Language   en ?
arXiv  1711.03229v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 65feb6d8-61b8-4256-b559-e272873f5aca
API URL: JSON