"A Flexible I/O Arbitration Framework for netCDF based Big Data Processing Workflows on High-End Supercomputers"

［ 2017年02月28日 ］

RIKEN International Symposium on Data Assimilation 2017
"A Flexible I/O Arbitration Framework for netCDF based Big Data Processing Workflows on High-End Supercomputers"

We present an initial study of a direct communication framework designed for complex workflows that eliminates unnecessary file I/O among components. Specifically, we propose an I/O arbitrator layer that provides direct parallel data transfer among job components that rely on the netCDF interface for performing I/O operations, with only minimal modifications to application code. We present the design and an early evaluation of the framework on the K Computer using up to 4800 nodes running RIKEN's experimental weather forecasting workflow as a case study.
http://www.data-assimilation.riken.jp/risda2017/

講師プロフィール
名前:Jianwei Liao	所属:Southwest University of China
略歴:外部ページ

eラーニングアーカイブ カテゴリー