Presentation
Enabling Efficient Data Infrastructure and Analytics on HPC Systems
Author
Advisor
Event Type
Doctoral Showcase
W
TUT
TP
EX
EXH
TimeWednesday, November 14th8:30am - 5pm
LocationC2/3/4 Ballroom
DescriptionWe propose to leverage PGAS and one-sided communication for building data infrastructure and analytics frameworks on HPC systems. Specifically, we have developed SHMEMCache, a distributed in-memory key-value store and SHMEMGraph, a balanced graph processing framework. We have tackled unique challenges in data consistency, load balancing, etc. with novel design features. The experimental results show that SHMEMCache achieves significant performance improvements over other KV stores such as Memcached in terms of latency and throughput. In addition, SHMEMGraph achieves an average improvement of 35.5% in terms of job completion time compared to a state-of-the-art graph processing system called Gemini.
Archive