A Momentum-Accelerated Hessian-Vector-Based Latent Factor Analysis Model

Weiling Li, Xin Luo, Huaqiang Yuan, Meng Chu Zhou

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

Service-oriented applications commonly involve high-dimensional and sparse (HiDS) interactions among users and service-related entities, e.g., user-item interactions from a personalized recommendation services system. How to perform precise and efficient representation learning on such HiDS interactions data is a hot yet thorny issue. An efficient approach to it is latent factor analysis (LFA), which commonly depends on large-scale non-convex optimization. Hence, it is vital to implement an LFA model able to approximate second-order stationary points efficiently for enhancing its representation learning ability. However, existing second-order LFA models suffer from high computational cost, which significantly reduces its practicability. To address this issue, this paper presents a Momentum-accelerated Hessian-vector algorithm (MH) for precise and efficient LFA on HiDS data. Its main ideas are two-fold: a) adopting the principle of a Hessian-vector-product-based method to utilize the second-order information without manipulating a Hessian matrix directly, and b) incorporating a generalized momentum method into its parameter learning scheme for accelerating its convergence rate to a stationary point. Experimental results on nine industrial datasets demonstrate that compared with state-of-the-art LFA models, an MH-based LFA model achieves gains in both accuracy and convergence rate. These positive outcomes also indicate that a generalized momentum method is compatible with the algorithms, e.g., a second-order algorithm, which implicitly rely on gradients.

Original languageEnglish (US)
Pages (from-to)830-844
Number of pages15
JournalIEEE Transactions on Services Computing
Volume16
Issue number2
DOIs
StatePublished - Mar 1 2023

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications
  • Information Systems and Management

Keywords

  • Services computing
  • generalized momentum method
  • hessian-vector
  • high-dimensional and sparse data
  • latent factor analysis
  • machine learning
  • recommendation service
  • representation learning
  • second-order optimization
  • service application

Fingerprint

Dive into the research topics of 'A Momentum-Accelerated Hessian-Vector-Based Latent Factor Analysis Model'. Together they form a unique fingerprint.

Cite this