A Momentum-accelerated Hessian-vector-based Latent Factor Analysis Model

Xin Luo, Weiling Li, Huaqiang Yuan, Mengchu Zhou

Research output: Contribution to journalArticlepeer-review

Abstract

Service-oriented applications commonly involve high-dimensional and sparse (HiDS) interactions among users and service-related entities, e.g., user-item interactions from a personalized recommendation services system. How to perform precise and efficient representation learning on such HiDS interactions data is a hot yet thorny issue. An efficient approach to it is latent factor analysis (LFA), which commonly depends on large-scale non-convex optimization. Hence, it is vital to implement an LFA model able to approximate second-order stationary points efficiently for enhancing its representation learning ability. However, existing ones suffer from high computational cost. To address this issue, this paper presents a Momentum-accelerated Hessian-vector algorithm (MH) for precise and efficient LFA on HiDS data. Its main ideas are two-fold: a) adopting the principle of a Hessian vector-product-based method to utilize the second-order information without manipulating a Hessian matrix directly, and b) incorporating a generalized momentum method into its parameter learning scheme for accelerating its convergence rate to a stationary point. Experimental results on nine industrial datasets demonstrate that compared with state-of-the-art LFA models, an MH-based LFA model achieves gains in both accuracy and convergence rate. These positive outcomes also indicate that a generalized momentum method is compatible with the algorithms, e.g., a second-order algorithms, e.g., a second-order algorithm, which implicitly rely on gradients.

Original languageEnglish (US)
Pages (from-to)1
Number of pages1
JournalIEEE Transactions on Services Computing
DOIs
StateAccepted/In press - 2022

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications
  • Information Systems and Management

Keywords

  • Analytical models
  • Approximation algorithms
  • Computational modeling
  • Convergence
  • Generalized Momentum Method
  • Hessian-vector
  • High-Dimensional and Sparse Data
  • Latent Factor Analysis
  • Optimization
  • Recommendation Service
  • Representation learning
  • Second-order Optimization
  • Service Application
  • Services Computing
  • Symbols

Fingerprint

Dive into the research topics of 'A Momentum-accelerated Hessian-vector-based Latent Factor Analysis Model'. Together they form a unique fingerprint.

Cite this