Jure Leskovec, Computer Science Dept., Stanford

Transcription

1 Jure Leskovec, Computer Science Dept., Stanford Includes joint work with Jaewon Yang, Manuel Gomez-Rodriguez, Jon Kleinberg, Lars Backstrom, and Andreas Krause

2 Jure Leskovec 2 Global vs. Local effects: Interaction of global effects from mass media and local effects carried by the social structure (e.g., blogs, Twitter) Internet, blogs, social media: Social media means the dichotomy between global and local influence is evaporating Speed of media reporting and discussion has intensified: very rapid progression of stories How does information transmitted by the media interact with social networks?

3 Jure Leskovec 3 In principle, we can collect nearly all (online) news media content: 10 million articles/day (50GB of data) Collecting data since Aug 08 ~10TB Could study media ecosystem at large Challenges: Humans don t scale Develop automatic computational methods What are basic units of information? Units that propagate between the nodes

4 [w/ Backstrom-Kleinberg, KDD 09] Jure Leskovec 4 Would like units that: Correspond to aggregates of articles, vary over the order of days, and can be handled at terabyte scale Plan: Identify textual fragments, phrases, memes that travel relatively intact through many articles Things that don t work: Cascasding hyper-links to articles: too fine-grained Topics as probabilistic term mixtures: too coarse-grained Named entities: too coarse-grained Common sequence of words: too noisy Idea: Quoted phrases:.* Are integral parts of journalistic practices Tend to follow iterations of a story as it evolves Are attributed to individuals and have time and location

5 Jure Leskovec [w/ Backstrom-Kleinberg, KDD 09] 5 Data from Spinn3r on the 3 months leading up to the 2008 U.S. Presidential Election: 1 million news articles and blog posts per day Essentially a complete online media coverage: 20,000 sites that are part of Google News 1.6 million blogs From August 1 to October million documents from 1.65 million sites, 390GB We extract 112 million quotes (phrases)

6 [w/ Backstrom-Kleinberg, KDD 09] Phrase: Our opponent is someone who sees America, it seems, as being so imperfect, imperfect enough that he s palling around with terrorists who would target their own country. 6

7 is periodic (weekly), no trends The bandwidth of the online media is constant 7

8 August October Volume over time of top 50 largest total volume phrases Jure Leskovec 8

9 9

10 Peak blog intensity comes about 2.5 hours after news peak. Using Google News we label: Mainstream media: 20,000 sites (44% vol.) Blog (everything else): 1.6 million sites (56% vol.) Jure Leskovec 10

11 Can classify individual sources by their typical timing relative to the peak aggregate intensity Professional blogs News media Jure Leskovec 11

12 Jure Leskovec 12 The oscillation of attention between mainstream and social media

13 Jure Leskovec [w/ Yang, ICDM 10] 13 Question: If New York Times mentions a meme at time t How many subsequent mentions of meme does this generate at time t+1, t+2,? Formulation: We want to predict the volume x(t) of phrase x at time t as a function of influences of sites that mentioned the meme before time t

14 Jure Leskovec [w/ Yang, ICDM 10] 14 LIM model: Given a volume over time x(t) of meme x And let: I A (t): influence curve of site A t A : time when A mentioned x Then we model: x(t+1) = W I W (t - t W ) For each site W estimate I W (t) It boils down to a least squares-like problem

15 [w/ Yang, ICDM 10] 15 Task: Predict volume x(t+1) of phrase x based on influences of sites that already mentioned x Setting: Using 1,000 phrases, and only 20 websites Improvement in L1 error over 1-time lag predictor By monitoring only 20 sites, we can reliably predict the overall future volume of a phrase (link, hashtag)

16 Jure Leskovec [w/ Yang, ICDM 10] 16 Business and politics are driven by mainstream media Entertainment (and sports) is driven by blogs and TV Newspapers and news agencies do not influence the volume

17 Jure Leskovec [w/ Gomez-Krause, KDD 10] 17 But how does information really spread? We only see the mentions but not the propagation Can we reconstruct (hidden) diffusion network?

18 [w/ Gomez-Krause, KDD 10] There is a hidden diffusion network: a b We only see times when nodes get infected: c 1 : (a,1), (c,2), (b,3), (e,4) c 2 : (c,1), (a,4), (b,5), (d,6) Want to infer who-infects-whom network The problem is NP-hard c e Our algorithm can do it near-optimally in O(N 2 ) Jure Leskovec (jure@cs.stanford.edu) d 18

19 [w/ Gomez-Krause, KDD 10] 5,000 news sites: Blogs Mainstream media Jure Leskovec 19

20 [w/ Gomez-Krause, 10] Blogs Mainstream media Jure Leskovec 20

21 Want to read things before others do. Detect blue & yellow soon but miss red. Jure Leskovec Detect all stories but late. 21

22 Jure Leskovec 22 Given a budget (e.g., of 3 blogs) Select sites to cover the most of the Web Bad news: Solving this exactly is NP-hard Good news: Theorem: Our algorithm can do it in linear time near-optimally Blogosphere

23 Question: Which websites should one read to catch big stories? Idea: Each blog covers part of the Web Each dot is a blog Proximity is based on the number of common cascades Jure Leskovec (jure@cs.stanford.edu) 23

24 Which blogs to read to be most up to date? Our solution % of stories detected (higher is better) In-links Out-links # posts (used by Technorati) Random Number of selected blogs Jure Leskovec (jure@cs.stanford.edu) 24

25 Jure Leskovec 25

26 Jure Leskovec 26 Meme-tracking and the Dynamics of the News Cycle, by J. Leskovec, L. Backstrom, J. Kleinberg. KDD, Modeling Information Diffusion in Implicit Networks, by J. Yang, J. Leskovec, ICDM, Inferring networks of diffusion and influence, by M. Gomez-Rodriguez, J. Leskovec, A. Krause. KDD Covering the great recession by Pew research center's project for excellence in journalism, Cost-effective Outbreak Detection in Networks by J. Leskovec, A. Krause, C. Guestrin, C. Faloutsos, J. VanBriesen, N. Glance. KDD Cascading Behavior in Large Blog Graphs by J. Leskovec, M. McGlohon, C. Faloutsos, N. Glance, M. Hurst. SDM,