Presentation
Comparison of the HPC and Big Data Java Libraries Spark, PCJ and APGAS
Author/Presenters
Event Type
Workshop
W
Parallel Programming Languages, Libraries, and Models
Productivity
TimeFriday, November 16th9:35am - 9:50am
LocationD168
DescriptionAlthough Java is rarely used in HPC, there are a few notable libraries. Use of Java may help to bridge the gap between HPC and big data processing.
This paper compares the big data library Spark, and the HPC libraries PCJ and APGAS, regarding productivity and performance. We refer to Java versions for all libraries. For APGAS, we include both the original version and an own extension by locality-flexible tasks. We consider three benchmarks: Calculation of pi from HPC, Unbalanced Tree Search (UTS) from HPC, and WordCount from the big data domain.
In performance measurements with up to 144~workers, the extended APGAS library was the clear winner. With 144 workers, APGAS programs were up to a factor of more than two faster than Spark programs, and up to about 30% faster than PCJ programs. Regarding productivity, the extended APGAS programs consistently needed the lowest number of different library constructs. Spark ranged second in productivity, and PCJ third.
This paper compares the big data library Spark, and the HPC libraries PCJ and APGAS, regarding productivity and performance. We refer to Java versions for all libraries. For APGAS, we include both the original version and an own extension by locality-flexible tasks. We consider three benchmarks: Calculation of pi from HPC, Unbalanced Tree Search (UTS) from HPC, and WordCount from the big data domain.
In performance measurements with up to 144~workers, the extended APGAS library was the clear winner. With 144 workers, APGAS programs were up to a factor of more than two faster than Spark programs, and up to about 30% faster than PCJ programs. Regarding productivity, the extended APGAS programs consistently needed the lowest number of different library constructs. Spark ranged second in productivity, and PCJ third.
Archive
