An effective high-performance multiway spatial join algorithm with spark

Zhenhong Du, Xianwei Zhao, Xinyue Ye, Jingwei Zhou, Zhang Feng, Renyi Liu

Research output: Contribution to journalArticlepeer-review

11 Scopus citations

Abstract

Multiway spatial join plays an important role in GIS (Geographic Information Systems) and their applications. With the increase in spatial data volumes, the performance of multiway spatial join has encountered a computation bottleneck in the context of big data. Parallel or distributed computing platforms, such as MapReduce and Spark, are promising for resolving the intensive computing issue. Previous approaches have focused on developing single-threaded join algorithms as an optimizing and partition strategy for parallel computing. In this paper, we present an effective high-performance multiway spatial join algorithm with Spark (MSJS) to overcome the multiway spatial join bottleneck. MSJS handles the problem through cascaded pairwise join. Using the power of Spark, the formerly inefficient cascaded pairwise spatial join is transformed into a high-performance approach. Experiments using massive real-world data sets prove that MSJS outperforms existing parallel approaches of multiway spatial join that have been described in the literature.

Original languageEnglish (US)
Article number96
JournalISPRS International Journal of Geo-Information
Volume6
Issue number4
DOIs
StatePublished - Apr 2017
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Geography, Planning and Development
  • Computers in Earth Sciences
  • Earth and Planetary Sciences (miscellaneous)

Keywords

  • Geocomputation performance
  • Multiway spatial join
  • Parallel computing
  • Spark

Fingerprint

Dive into the research topics of 'An effective high-performance multiway spatial join algorithm with spark'. Together they form a unique fingerprint.

Cite this