TY - GEN
T1 - A view from ORNL
T2 - 38th IEEE International Conference on Distributed Computing Systems, ICDCS 2018
AU - Klasky, Scott
AU - Wolf, Matthew
AU - Ainsworth, Mark
AU - Atkins, Chuck
AU - Choi, Jong
AU - Eisenhauer, Greg
AU - Geveci, Berk
AU - Godoy, William
AU - Kim, Mark
AU - Kress, James
AU - Kurc, Tahsin
AU - Liu, Qing
AU - Logan, Jeremy
AU - Maccabe, Arthur B.
AU - Mehta, Kshitij
AU - Ostrouchov, George
AU - Parashar, Manish
AU - Podhorszki, Norbert
AU - Pugmire, David
AU - Suchyta, Eric
AU - Wan, Lipeng
AU - Wang, Ruonan
N1 - Publisher Copyright:
© 2018 IEEE.
PY - 2018/7/19
Y1 - 2018/7/19
N2 - One of the core issues across computer and computational science today is adapting to, managing, and learning from the influx of 'Big Data'. In the commercial space, this problem has led to a huge investment in new technologies and capabilities that are well adapted to dealing with the sorts of human-generated logs, videos, texts, and other large-data artifacts that are processed and resulted in an explosion of useful platforms and languages (Hadoop, Spark, Pandas, etc.). However, translating this work from the enterprise space to the computational science and HPC community has proven somewhat difficult, in part because of some of the fundamental differences in type and scale of data and timescales surrounding its generation and use. We describe a forward-looking research and development plan which centers around the concept of making Input/Output (I/O) intelligent for users in the scientific community, whether they are accessing scalable storage or performing in situ workflow tasks. Much of our work is based on our experience with the Adaptable I/O System (ADIOS 1.X), and our next generation version of the software ADIOS 2.X [1].
AB - One of the core issues across computer and computational science today is adapting to, managing, and learning from the influx of 'Big Data'. In the commercial space, this problem has led to a huge investment in new technologies and capabilities that are well adapted to dealing with the sorts of human-generated logs, videos, texts, and other large-data artifacts that are processed and resulted in an explosion of useful platforms and languages (Hadoop, Spark, Pandas, etc.). However, translating this work from the enterprise space to the computational science and HPC community has proven somewhat difficult, in part because of some of the fundamental differences in type and scale of data and timescales surrounding its generation and use. We describe a forward-looking research and development plan which centers around the concept of making Input/Output (I/O) intelligent for users in the scientific community, whether they are accessing scalable storage or performing in situ workflow tasks. Much of our work is based on our experience with the Adaptable I/O System (ADIOS 1.X), and our next generation version of the software ADIOS 2.X [1].
KW - High Performance Computing
KW - High Performance I/O
KW - In Situ Visualization
KW - Publish/Subscribe
UR - http://www.scopus.com/inward/record.url?scp=85050963067&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85050963067&partnerID=8YFLogxK
U2 - 10.1109/ICDCS.2018.00136
DO - 10.1109/ICDCS.2018.00136
M3 - Conference contribution
AN - SCOPUS:85050963067
T3 - Proceedings - International Conference on Distributed Computing Systems
SP - 1357
EP - 1368
BT - Proceedings - 2018 IEEE 38th International Conference on Distributed Computing Systems, ICDCS 2018
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 2 July 2018 through 5 July 2018
ER -