理化学研究所 計算科学研究機構

メニュー
メニュー

第73回

bi

第73回 (AICS Cafe Progress Report)
日時: 2015年6月19日 (金)、 15:00 – 16:00
場所: AICS 6階講堂

講演題目: A Study of Parallel Data Compression on the K Computer
講演者: Chongke BI (可視化技術チーム)
※今回は、採用4年目の特別研究員の研究成果を発表します。

講演要旨: 詳細を見る

One major challenge presented by extreme-scale scientific computing is the huge amount of data that the simulation is capable of generating. Since each run of a state-of-the-art simulation can output data at the petascale, storing all of the data is no longer an option. Aside from simply dropping selected time steps, as has been the practice, additional data reduction methods must be considered in order to meet data movement and storage requirements, while maintaining the accuracy and integrity of the data. This is particularly important as scientific supercomputing moves toward exascale.
Firstly, we proposed a proper orthogonal decomposition (POD) based parallel compression method. A binary load-distributed approach is also proposed for fully utilizing all parallel nodes [1]. However, this method can only deal with the case that both the number of time steps and the number of parallel nodes are power-of-two. In order to resolve this problem, we proposed a 2-3-4 combination method [2]. Furthermore, in order to minimize the error, we proposed an m-swap method [3]. This method allows users to control the compression scale in one node. Recently, we are trying to apply these methods for in-situ visualization.
Besides of the POD-based compression methods mentioned above, a run-length method [4] was also proposed for lossless compression. Now we are proposing a compression method for reducing the size of streamlines and pathlines from large-scale simulations.
These approaches allow us to effectively use all of the processors and to reduce the interprocessor communication cost throughout the parallel compression calculations. The results of tests using the K computer indicate the superior performance of our design and implementation.

参考文献:
[1]. Chongke Bi, Kenji Ono, Kwan-Liu Ma, Haiyuan Wu, and Toshiyuki Imamura, “Proper Orthogonal Decomposition Based Parallel Compression for Visualizing Big Data on the K Computer,” in Proceedings of Eurographics Symposium on Parallel Graphics and Visualization, pp. 1-8, June, 2014.
[2]. Chongke Bi and Kenji Ono, “2-3-4 Combination for Parallel Compression on the K Computer,” in Proceedings of IEEE Pacific Visualization Symposium, pp. 281-285, March, 2014. doi: 10.1109/PacificVis.2014.28
[3]. Chongke Bi, Kenji Ono, and Lu Yang, “Parallel POD Compression of Time-Varying Big Datasets Using m-Swap on the K Computer,” in Proceedings of IEEE International Congress on Big Data, pp. 438-445, June, 2014. doi: 10.1109/BigData.Congress.2014.70
[4]. Shota Ishikawa, Haiyuan Wu, Chongke Bi, Qian Chen, Hirokazu Taki, and Kenji Ono, “Fluid Data Compression and ROI Detection Using Run Length Method,” in Procedia Computer Science, Vol. 35, pp. 1284-1291, 2014. doi: 10.1016/j.procs.2014.08.228