Graph Enhanced BERT for Query Understanding

Juanhui Li, Wei Zeng, Suqi Cheng, Yao Ma, Jiliang Tang, Shuaiqiang Wang, Dawei Yin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Query understanding plays a key role in exploring users' search intents. However, it is inherently challenging since it needs to capture semantic information from short and ambiguous queries and often requires massive task-specific labeled data. In recent years, pre-trained language models (PLMs) have advanced various natural language processing tasks because they can extract general semantic information from large-scale corpora. However, directly applying them to query understanding is sub-optimal because existing strategies rarely consider to boost the search performance. On the other hand, search logs contain user clicks between queries and urls that provide rich users' search behavioral information on queries beyond their content. Therefore, in this paper, we aim to fill this gap by exploring search logs. In particular, we propose a novel graph-enhanced pre-training framework, GE-BERT, which leverages both query content and the query graph to capture both semantic information and users' search behavioral information of queries. Extensive experiments on offline and online tasks have demonstrated the effectiveness of the proposed framework.

Original languageEnglish (US)
Title of host publicationSIGIR 2023 - Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages3315-3319
Number of pages5
ISBN (Electronic)9781450394086
DOIs
StatePublished - Jul 19 2023
Event46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023 - Taipei, Taiwan, Province of China
Duration: Jul 23 2023Jul 27 2023

Publication series

NameSIGIR 2023 - Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Conference

Conference46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023
Country/TerritoryTaiwan, Province of China
CityTaipei
Period7/23/237/27/23

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design
  • Information Systems
  • Software

Keywords

  • BERT
  • Graph neural networks
  • KL-divergence
  • Query understanding

Fingerprint

Dive into the research topics of 'Graph Enhanced BERT for Query Understanding'. Together they form a unique fingerprint.

Cite this