FlexQuery: An online query system for interactive remote visual data exploration at large scale

Hongbo Zou, Karsten Schwan, Magdalena Slawinska, Matt Wolf, Greg Eisenhauer, Fang Zheng, Jai Dayal, Jeremy Logan, Qing Liu, Scott Klasky, Tanja Bode, Michael Clark, Matt Kinsey

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations

Abstract

The remote visual exploration of live data generated by scientific simulations is useful for scientific discovery, performance monitoring, and online validation for the simulation results. Online visualization methods are challenged, however, by the continued growth in the volume of simulation output data that has to be transferred from its source - the simulation running on the high end machine - to where it is analyzed, visualized, and displayed. A specific challenge in this context is limits in the communication bandwidth between data source(s) and sinks. Previous work places queries 'near' data sources, exploiting their data reduction capabilities, but such work does not address the common scenario in which scientists make multiple different queries on the data being produced. This paper considers the general case in which science users are interested in different (sub)sets of the data produced by a high end simulation. We offer the FlexQuery online data query system that can deploy and execute data queries 'along' the I/O and analytics pipelines. FlexQuery carefully extends such analytics pipelines, using online performance monitoring and data location tracking, to realize data queries in ways that minimize additional data movement and offer low latency in data query execution. Using a real-world scientific application - the Maya astrophysics code and its analytics workflow - we demonstrate FlexQuery's ability to dynamically deploy queries for low-latency remote data visualization.

Original languageEnglish (US)
Title of host publication2013 IEEE International Conference on Cluster Computing, CLUSTER 2013
DOIs
StatePublished - 2013
Externally publishedYes
Event15th IEEE International Conference on Cluster Computing, CLUSTER 2013 - Indianapolis, IN, United States
Duration: Sep 23 2013Sep 27 2013

Publication series

NameProceedings - IEEE International Conference on Cluster Computing, ICCC
ISSN (Print)1552-5244

Other

Other15th IEEE International Conference on Cluster Computing, CLUSTER 2013
CountryUnited States
CityIndianapolis, IN
Period9/23/139/27/13

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Signal Processing

Keywords

  • data reduction
  • online query
  • remote visualization

Fingerprint Dive into the research topics of 'FlexQuery: An online query system for interactive remote visual data exploration at large scale'. Together they form a unique fingerprint.

Cite this