Computing Just What You Need: Online Data Analysis and Reduction at Extreme Scales

Ian Foster, Mark Ainsworth, Bryce Allen, Julie Bessac, Franck Cappello, Jong Youl Choi, Emil Constantinescu, Philip E. Davis, Sheng Di, Wendy Di, Hanqi Guo, Scott Klasky, Kerstin Kleese Van Dam, Tahsin Kurc, Qing Liu, Abid Malik, Kshitij Mehta, Klaus Mueller, Todd Munson, George OstouchovManish Parashar, Tom Peterka, Line Pouchard, Dingwen Tao, Ozan Tugluk, Stefan Wild, Matthew Wolf, Justin M. Wozniak, Wei Xu, Shinjae Yoo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

53 Scopus citations

Abstract

A growing disparity between supercomputer computation speeds and I/O rates makes it increasingly infeasible for applications to save all results for offline analysis. Instead, applications must analyze and reduce data online so as to output only those results needed to answer target scientific question(s). This change in focus complicates application and experiment design and introduces algorithmic, implementation, and programming model challenges that are unfamiliar to many scientists and that have major implications for the design of various elements of supercomputer systems. We review these challenges and describe methods and tools that we are developing to enable experimental exploration of algorithmic, software, and system design alternatives.

Original languageEnglish (US)
Title of host publicationEuro-Par 2017
Subtitle of host publicationParallel Processing - 23rd International Conference on Parallel and Distributed Computing, Proceedings
EditorsFrancisco F. Rivera, Tomas F. Pena, Jose C. Cabaleiro
PublisherSpringer Verlag
Pages3-19
Number of pages17
ISBN (Print)9783319642024
DOIs
StatePublished - 2017
Event23rd International Conference on Parallel and Distributed Computing, Euro-Par 2017 - Santiago de Compostela, Spain
Duration: Aug 28 2017Sep 1 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10417 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other23rd International Conference on Parallel and Distributed Computing, Euro-Par 2017
Country/TerritorySpain
CitySantiago de Compostela
Period8/28/179/1/17

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Computing Just What You Need: Online Data Analysis and Reduction at Extreme Scales'. Together they form a unique fingerprint.

Cite this