Computational graph analytics for massive streaming data

In this chapter the author presents a new, extensible and flexible data structure for massive graphs called STINGER (Spatio-Temporal Interaction Networks and Graphs (STING) Extensible Representation). Two studies are discussed: the first study, computing a widely used network analysis metric called clustering coefficients, and the second study, the approach for tracking connected components given a stream of edge insertions and removals. The chapter describes the massively multithreaded Cray XMT as well as the more common platform built on Intel’s Nehalem architecture. It presents experimental results on the Cray XMT, with a comparison to the Intel Nehalem platform for the clustering coefficient problem. Regarding clustering coefficients, a similar rate of 200,000 updates per second can be maintained on the Cray XMT.