Toward Efficient In-memory Data Analytics on NUMA Systems release_gjxyzgxrsjct3pkyev5njxsr6y

by Puya Memarzia, Suprio Ray, Virendra C Bhavsar

Released as a article .

2019  

Abstract

Data analytics systems commonly utilize in-memory query processing techniques to achieve better throughput and lower latency. Modern computers increasingly rely on Non-Uniform Memory Access (NUMA) architectures in order to achieve scalability. A key drawback of NUMA architectures is that many existing software solutions are not aware of the underlying NUMA topology and thus do not take full advantage of the hardware. Modern operating systems are designed to provide basic support for NUMA systems. However, default system configurations are typically sub-optimal for large data analytics applications. Additionally, achieving NUMA-awareness by rewriting the application from the ground up is not always feasible. In this work, we evaluate a variety of strategies that aim to accelerate memory-intensive data analytics workloads on NUMA systems. We analyze the impact of different memory allocators, memory placement strategies, thread placement, and kernel-level load balancing and memory management mechanisms. With extensive experimental evaluation we demonstrate that methodical application of these techniques can be used to obtain significant speedups in four commonplace in-memory data analytics workloads, on three different hardware architectures. Furthermore, we show that these strategies can speed up two popular database systems running a TPC-H workload.
In text/plain format

Archived Files and Locations

application/pdf  3.0 MB
file_dk5kvo6ranccbpvvnunpc7yzie
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2019-08-07
Version   v2
Language   en ?
arXiv  1908.01860v2
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 9cf3ae81-cceb-4b4c-8bae-ae7e51f6449a
API URL: JSON