Diverse Power Iteration Embeddings and Its Applications

Hao Huang, Shinjae Yoo, Dantong Yu, Hong Qin

Research output: Contribution to journalConference articlepeer-review

7 Scopus citations

Abstract

Spectral Embedding is one of the most effective dimension reduction algorithms in data mining. However, its computation complexity has to be mitigated in order to apply it for real-world large scale data analysis. Many researches have been focusing on developing approximate spectral embeddings which are more efficient, but meanwhile far less effective. This paper proposes Diverse Power Iteration Embeddings (DPIE), which not only retains the similar efficiency of power iteration methods but also produces a series of diverse and more effective embedding vectors. We test this novel method by applying it to various data mining applications (e.g. Clustering, anomaly detection and feature selection) and evaluating their performance improvements. The experimental results show our proposed DPIE is more effective than popular spectral approximation methods, and obtains the similar quality of classic spectral embedding derived from eigen-decompositions. Moreover it is extremely fast on big data applications. For example in terms of clustering result, DPIE achieves as good as 95% of classic spectral clustering on the complex datasets but 4000+ times faster in limited memory environment.

Original languageEnglish (US)
Article number7023337
Pages (from-to)200-209
Number of pages10
JournalProceedings - IEEE International Conference on Data Mining, ICDM
Volume2015-January
Issue numberJanuary
DOIs
StatePublished - Jan 1 2015
Externally publishedYes
Event14th IEEE International Conference on Data Mining, ICDM 2014 - Shenzhen, China
Duration: Dec 14 2014Dec 17 2014

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Fingerprint Dive into the research topics of 'Diverse Power Iteration Embeddings and Its Applications'. Together they form a unique fingerprint.

Cite this