Gibberish, Assistant, or Master? Using Tweets Linking to News for Extractive Single-Document Summarization Zhongyu Wei1, Wei Gao2 1Computer 2Qatar

Science Department, The University of Texas at Dallas, Texas, USA Computing Research Institute, Qatar Foundation, Doha, Qatar

1. Introduction

2. Dataset - News-tweets-highlights triplings collected from tweets linking to CNN/USAToday news articles (Wei and Gao, 2014) via Topsy search API (alt.qcri.org/~wgao/data/highlights_extraction.zip) - 17 world news events from July 2012 to July 2013 - 121 documents, 455 highlights and 78,419 linking tweets

Using tweets linking to news for extractive single-document summarization: -

Are the linking tweets gibberish or useful? If being useful, do they play assistant or master roles? Is the latency of tweets a major setback for the quality of summaries?

Distribution of documents, highlights and tweets with respect to different events (Wei and Gao, 2014)

3. The Basic Value of Tweets

(b) Tweets hit

(a)Highlights hit

(c) Highlights vs. tweets hit

(d) Max tweet-highlight similarity

Figure 1: (a) Position of highlights hits in the documents; (b) Top-4 tweets hits in the documents; (c) The probability of highlights hit vs. tweets hit in the documents; (d) The maximum similarity between highlights and tweets per document

4. METHODS - Social Vote (SociVote): Directly utilize the votes of each news sentences received from its linking tweets - Heterogeneous Graph Random Walk (HGRW): 1. Create an undirected heterogeneous graph with two types of nodes (i.e., sentences and tweets) 2. Rank both instances simultaneously by random walk. The edge weights are defined as:

𝑠𝑖𝑚 𝑥, 𝑦 =

𝛽 ∗ idf−modified−cosine(𝑥, 𝑦) 1 − 𝛽 ∗ idf−modified−cosine(𝑥, 𝑦)

if 𝑥. 𝑡 ≠ 𝑦. 𝑡 otherwise

5. Experiments and Results Method

ROUGE-1 R(%)

Lead-S

26.3

21.1

37.4

LexRank-S

25.9

20.6

37.6

LexRank-T

25.0

25.4

25.9

*L2R-S

25.6

21.4

34.5

*L2R-T

26.4

28.0

27.4

*CrossL2R-S

29.2

23.9

39.8

*CrossL2R-T

29.5

32.0

29.5

SociVote-S

28.2

23.6

37.6

HGRW-S

29.2

23.7

40.3

HGRW-T

29.3

27.9

32.4

HGRW-ST

29.8

25.8

37.6

Cross-type

Ours

P(%)

Homo-type

Baselines

F (%)

(a) Impact of β

(b) Impact of tweets volume

Table 1: Results on CNN/USAToday corpus Lead: The first n sentences; LexRank: The original LexRank algorithm; L2R: RankBoost takes either sentences or tweets as input; CrossL2R: RankBoost incorporates cross-type features. Bold: best score; Underline: p<0.05 as to the baselines except for Cross L2R based on two-tailed t-test; The suffixes -S, -T and -ST: output contains sentences, tweets and both, respectively; *: supervised models

(d) Time vs. tweets volume (c) Impact of tweets Latency Figure 2: The influence and relation of some major factors