Keyword extraction and headline generation using novel word features

Songhua Xu, Shaohui Yang, Francis C.M. Lau

Research output: Chapter in Book/Report/Conference proceedingConference contribution

37 Scopus citations

Abstract

We introduce several novel word features for keyword extraction and headline generation. These new word features are derived according to the background knowledge of a document as supplied by Wikipedia. Given a document, to acquire its background knowledge from Wikipedia, we first generat e a query for searching the Wikipedia corpus based on the key facts present in the document. We then use the query to find articles in the Wikipedia corpus that are closely related to the contents of the document. With the Wikipedia search result article set, we extract the inlink, outlink, category and infobox information in each article to derive a set of novel word features which reflect the document's background knowledge. These newly introduced word features of fer valuable indications on individual words' importance in the input document. They serve as nice complements to the traditional word features derivable from explicit information of a document. In addition, we also introduce a word-document fitness feat ure to characterize the influence of a document's genre on the keyword extraction and headline generation process. We study the effectiveness of these novel word features for keyword extraction and headline generation by experiments and have obtained very encouraging results.

Original languageEnglish (US)
Title of host publicationAAAI-10 / IAAI-10 - Proceedings of the 24th AAAI Conference on Artificial Intelligence and the 22nd Innovative Applications of Artificial Intelligence Conference
PublisherAI Access Foundation
Pages1461-1466
Number of pages6
ISBN (Print)9781577354666
StatePublished - 2010
Event24th AAAI Conference on Artificial Intelligence and the 22nd Innovative Applications of Artificial Intelligence Conference, AAAI-10 / IAAI-10 - Atlanta, GA, United States
Duration: Jul 11 2010Jul 15 2010

Publication series

NameProceedings of the National Conference on Artificial Intelligence
Volume3

Other

Other24th AAAI Conference on Artificial Intelligence and the 22nd Innovative Applications of Artificial Intelligence Conference, AAAI-10 / IAAI-10
Country/TerritoryUnited States
CityAtlanta, GA
Period7/11/107/15/10

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Keyword extraction and headline generation using novel word features'. Together they form a unique fingerprint.

Cite this