Practical Sentiment Analysis

  • Published on
    11-Aug-2014

  • View
    1.964

  • Download
    1

DESCRIPTION

An informative tutorial on practical sentiment analysis, natural language processing, and semi-supervised learning. Learn how your company can leverage the crowd for sentiment analysis of structured and un-structured content. Dr. Jason Baldridge is co-founder and Chief Scientist at People Pattern, and Associate Professor, Linguistics, at the University of Texas

Transcript

  • Practical Sentiment Analysis Tutorial Jason Baldridge @jasonbaldridge Sentiment Analysis Symposium 2014 Associate Professor Co-founder & Chief Scientist Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 About the presenter Associate Professor, Linguistics Department, The University of Texas at Austin (2005-present) Ph.D., Informatics, The University of Edinburgh, 2002 MA (Linguistics), MSE (Computer Science), The University of Pennsylvania, 1998 Co-founder & Chief Scientist, People Pattern (2013-present) Built Converseons Convey text analytics engine, with Philip Resnik and Converseon programmers. 2 Wednesday, March 5, 14
  • Why NLP is hard Sentiment analysis overview Document classication break Aspect-based sentiment analysis Visualization Semi-supervised learning break Stylistics & author modeling Beyond text Wrap up Wednesday, March 5, 14
  • Why NLP is hard Sentiment analysis overview Document classication Aspect-based sentiment analysis Visualization Semi-supervised learning Stylistics & author modeling Beyond text Wrap up Wednesday, March 5, 14
  • Text is pervasive = big opportunities Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Texts as bags of words (with apologies) (http://www.wordle.net/) 6 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Texts as bags of words (with apologies) (http://www.wordle.net/) http://www.wired.com/magazine/2010/12/ff_ai_essay_airevolution/ 6 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 That is of course not the full story... Texts are not just bags-of-words. Order and syntax affect interpretation of utterances. 7 leg on manthe dog bit Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 That is of course not the full story... Texts are not just bags-of-words. Order and syntax affect interpretation of utterances. 7 legonmanthe dogbit thethe Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 That is of course not the full story... Texts are not just bags-of-words. Order and syntax affect interpretation of utterances. 7 legonmanthe dogbit thethe mandog Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 That is of course not the full story... Texts are not just bags-of-words. Order and syntax affect interpretation of utterances. 7 legonmanthe dogbit thethe mandog Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 That is of course not the full story... Texts are not just bags-of-words. Order and syntax affect interpretation of utterances. 7 legonmanthe dogbit thethe mandog Subject Object Modifier Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 That is of course not the full story... Texts are not just bags-of-words. Order and syntax affect interpretation of utterances. 7 legonmanthe dogbit thethe mandog Subject Object Location Wednesday, March 5, 14
  • What does this sentence mean? I saw her duck with a telescope. Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l I saw her duck with a telescope. Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l I saw her duck with a telescope. verb Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l I saw her duck with a telescope. verb Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l I saw her duck with a telescope. verb Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l I saw her duck with a telescope. verb Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l I saw her duck with a telescope. verb Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l [http://www.supercoloring.com/pages/duck-outline/] I saw her duck with a telescope. verb Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l [http://www.supercoloring.com/pages/duck-outline/] I saw her duck with a telescope. verb noun Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l [http://www.supercoloring.com/pages/duck-outline/] I saw her duck with a telescope. verb noun Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l [http://www.supercoloring.com/pages/duck-outline/] I saw her duck with a telescope. verb noun Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l [http://www.supercoloring.com/pages/duck-outline/] I saw her duck with a telescope. verb noun Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l [http://www.supercoloring.com/pages/duck-outline/] I saw her duck with a telescope. verb noun Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l [http://www.supercoloring.com/pages/duck-outline/] I saw her duck with a telescope. verb noun [http://www.clker.com/clipart-green-eyes-3.html] Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l [http://www.supercoloring.com/pages/duck-outline/] I saw her duck with a telescope. verb noun [http://www.clker.com/clipart-3163.html] Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l [http://www.supercoloring.com/pages/duck-outline/] I saw her duck with a telescope. verb noun [http://www.simonpalfrader.com/category/tournament-poker] Slide by Lillian Lee Wednesday, March 5, 14
  • What does this sentence mean? [http://casablancapa.blogspot.com/2010/05/fore.htm]l [http://www.supercoloring.com/pages/duck-outline/] I saw her duck with a telescope. verb noun [http://casablancapa.blogspot.com/2010/05/fore.htm]l Slide by Lillian Lee Wednesday, March 5, 14
  • Ambiguity is pervasive the a are of I [Steve Abney] Slide by Lillian Lee Wednesday, March 5, 14
  • Ambiguity is pervasive the a are of I [Steve Abney] an are (100 m2) another are Slide by Lillian Lee Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 And it goes further... Rhetorical structure affects the interpretation of the text as a whole. 10 Max fell. John pushed him. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 And it goes further... Rhetorical structure affects the interpretation of the text as a whole. 10 Max fell. John pushed him. Max fell. John pushed him. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 And it goes further... Rhetorical structure affects the interpretation of the text as a whole. 10 Max fell. John pushed him.(Because) Explanation Max fell. John pushed him. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 And it goes further... Rhetorical structure affects the interpretation of the text as a whole. 10 Max fell. John pushed him.(Because) Explanation Max fell. John pushed him.(Then) Continuation Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 And it goes further... Rhetorical structure affects the interpretation of the text as a whole. 10 Max fell. John pushed him.(Because) Explanation Max fell. John pushed him.(Then) Continuation pushing precedes falling Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 And it goes further... Rhetorical structure affects the interpretation of the text as a whole. 10 Max fell. John pushed him.(Because) Explanation Max fell. John pushed him.(Then) Continuation pushing precedes falling falling precedes pushing Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] To get a spare tire (donut) for his car? John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] store where donuts shop? or is run by donuts? or looks like a big donut? or made of donut? or has an emptiness at its core? John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] I stopped smoking freshman year, but John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] Describes where the store is? Or when he stopped? John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] Well, actually, he stopped there from hunger and exhaustion, not just from work. John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] At that moment, or habitually? John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] Thats how often he thought it? John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] But actually, a coffee only stays good for about 10 minutes before it gets cold. John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] Similarly: In America a woman has a baby every 15 minutes. Our job is to nd that woman and stop her. John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] the particular coffee that was good every few hours? the donut store? the situation? John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] too expensive for what? what are we supposed to conclude about what John did? John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 201411 Whats hard about this story? [Slide from Jason Eisner] how do we connect it to expensive? John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. Wednesday, March 5, 14
  • NLP has come a long way Wednesday, March 5, 14
  • Sentiment analysis overview Why NLP is hard Document classication Aspect-based sentiment analysis Visualization Semi-supervised learning Stylistics & author modeling Beyond text Wrap up Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Sentiment analysis: background [slide from Lillian Lee] People search for and are affected by online opinions. TripAdvisor, Rotten Tomatoes, Yelp, Amazon, eBay, YouTube, blogs, Q&A and discussion sites According to a Comscore 07 report and an 08 Pew survey: 60% of US residents have done online product research, and 15% do so on a typical day. 73%-87% of US readers of online reviews of services say the reviews were signicant inuences. (more on economics later) But, 58% of US internet users report that online information was missing, impossible to nd, confusing, and/or overwhelming. Creating technologies that nd and analyze reviews would answer a tremendous information need. 14 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Broader implications: economics [slide from Lillian Lee] Consumers report being willing to pay from 20% to 99% more for a 5-star-rated item than a 4-star-rated item. [comScore] But, does the polarity and/or volume of reviews have measurable, signicant inuence on actual consumer purchasing? Implications for bang-for-the-buck, manipulation, etc. 15 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Social media analytics: acting on sentiment 16 Richard Lawrence, Prem Melville, Claudia Perlich, Vikas Sindhwani, Estepan Meliksetian et al. In ORMS Today, Volume 37, Number 1, February, 2010. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Polarity classication [slide from Lillian Lee] Consider just classifying an avowedly subjective text unit as either positive or negative (thumbs up or thumbs down). One application: review summarization. Elvis Mitchell, May 12, 2000: It may be a bit early to make such judgments, but Battleeld Earth may well turn out to be the worst movie of this century. Cant we just look for words like great, terrible, worst? Yes, but ... learning a sufcient set of such words or phrases is an active challenge. 17 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Using a lexicon [slide from Lillian Lee] From a small scale human study: 18 Proposed word lists Accuracy Subject 1 Positive: dazzling, brilliant, phenomenal, excellent, fantastic Negative: suck, terrible, awful, unwatchable, hideous 58% Subject 2 Positive: gripping, mesmerizing, riveting, spectacular, cool, awesome, thrilling, badass, excellent, moving, exciting Negative: bad, cliched, sucks, boring, stupid, slow 64% Automatically determined (from data) Positive: love, wonderful, best, great, superb, beautiful, still Negative: bad, worst, stupid, waste, boring, ?, ! 69% Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Polarity words are not enough [slide from Lillian Lee] Cant we just look for words like great or terrible? Yes, but ... This laptop is a great deal. A great deal of media attention surrounded the release of the new laptop. This laptop is a great deal ... and Ive got a nice bridge you might be interested in. 19 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Polarity words are not enough Polarity ippers: some words change positive expressions into negative ones and vice versa. Negation: America still needs to be focused on job creation. Not among Obama's great accomplishments since coming to ofce !! [From a tweet in 2010] Contrastive discourse connectives: I used to HATE it. But this stuff is yummmmmy :) [From a tweet in 2011 -- the tweeter had already bolded HATE and But!] Multiword expressions: other words in context can make a negative word positive: That movie was shit. [negative] That movie was the shit. [positive] (American slang from the 1990s) 20 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 More subtle sentiment (from Pang and Lee) With many texts, no ostensibly negative words occur, yet they indicate strong negative polarity. If you are reading this because it is your darling fragrance, please wear it at home exclusively, and tape the windows shut. (review by Luca Turin and Tania Sanchez of the Givenchy perfume Amarige, in Perfumes: The Guide, Viking 2008.) She runs the gamut of emotions from A to B. (Dorothy Parker, speaking about Katharine Hepburn.) Jane Austens books madden me so that I cant conceal my frenzy from the reader. Every time I read Pride and Prejudice I want to dig her up and beat her over the skull with her own shin-bone. (Mark Twain.) 21 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Thwarted expectations (from Pang and Lee) 22 This lm should be brilliant. It sounds like a great plot, the actors are first grade, and the supporting cast is good as well, and Stallone is attempting to deliver a good performance. However, it cant hold up. There are also highly negative texts that use lots of positive words, but ultimately are reversed by the nal sentence. For example This is referred to as a thwarted expectations narrative because in the nal sentence the author sets up a deliberate contrast to the preceding discourse, giving it more impact. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Polarity classication: its more than positive and negative Positive: As a used vehicle, the Ford Focus represents a solid pick. Negative: Still, the Focus' interior doesn't quite measure up to those offered by some of its competitors, both in terms of materials quality and design aesthetic. Neutral: The Ford Focus has been Ford's entry-level car since the start of the new millennium. Mixed: The current Focus has much to offer in the area of value, if not renement. 23 http://www.edmunds.com/ford/focus/review.html Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Other dimensions of sentiment analysis Subjectivity: is an opinion even being expressed? Many statements are simply factual. Target: what exactly is an opinion being expressed about? Important for aggregating interesting and meaningful statistics about sentiment. Also, it affects how the language use indicates polarity: e.g, unpredictable is usually positive for movie reviews, but is very negative for a cars steering Ratings: rather than a binary decision, it is often of interest to provide or interpret predictions about sentiment on a scale, such as a 5-star system. 24 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Other dimensions of sentiment analysis Perspective: an opinion can be positive or negative depending on who is saying it entry-level could be good or bad for different people it also affects how an author describes a topic: e.g. pro-choice vs pro-life, affordable health care vs obamacare. Authority: was the text written by someone whose opinion matters more than others? it is more important to identify and address negative sentiment expressed by a popular blogger than a one-off commenter or supplier of a product reviewer on a sales site follower graphs (where applicable) are very useful in this regard Spam: is the text even valid or at least something of interest? many tweets and blog post comments are just spammers trying to drive trafc to their sites 25 Wednesday, March 5, 14
  • Document Classication Why NLP is hard Sentiment analysis overview Aspect-based sentiment analysis Visualization Semi-supervised learning Stylistics & author modeling Beyond text Wrap up Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Text analysis, in brief 27 f( , ,... ) = [ , ,... ] Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Text analysis, in brief 27 f( , ,... ) = [ , ,... ] Sentiment labels Parts-of-speech Named Entities Topic assignments Geo-coordinates Syntactic structures Translations Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Text analysis, in brief 27 f( , ,... ) = [ , ,... ] Sentiment labels Parts-of-speech Named Entities Topic assignments Geo-coordinates Syntactic structures Translations Rules Annotation & Learning - annotated examples - annotated knowledge - interactive annotation and learning Scalable human annotation Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Document classication: automatically label some text Language identication: determine the language that a text is written in Spam ltering: label emails, tweets, blog comments as spam (undesired) or ham (desired) Routing: label emails to an organization based on which department should respond to them (e.g. complaints, tech support, order status) Sentiment analysis: label some text as being positive or negative (polarity classication) Georeferencing: identify the location (latitude and longitude) associated with a text 28 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Desiderata for text analysis function f task is well-dened outputs are meaningful precision, recall, etc. are measurable and sufcient for desired use 29 Performant Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Desiderata for text analysis function f task is well-dened outputs are meaningful precision, recall, etc. are measurable and sufcient for desired use 29 Performant affordable access to annotated examples and/or knowledge sources able to exploit indirect or noisy annotations access to unlabeled examples and ability to exploit them tools to learn f are available or can be built within budget Reasonable cost (time & money) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Four sentiment datasets 30 Dataset Topic Year # Train # Dev #Test Reference Debate08 Obama vs McCain debate 2008 795 795 795 Shamma, et al. (2009) "Tweet the Debates: Understanding Community Annotation of Uncollected Sources." HCR Health care reform 2010 839 838 839 Speriosu et al. (2011) "Twitter Polarity Classication with Label Propagation over Lexical Links and the Follower Graph." STS (Stanford) Twitter Sentiment 2009 - 216 - Go et al. (2009) "Twitter sentiment classication using distant supervision" IMDB IMDB movie reviews 2011 25,000 25,000 - Mas et al. (2011) "Learning WordVectors for Sentiment Analysis" Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Rule-based classication Identify words and patterns that are indicative of positive or negative sentiment: polarity words: e.g. good, great, love; bad, terrible, hate polarity ngrams: the shit (+), must buy (+), could care less (-) casing: uppercase often indicates subjectivity punctuation: lots of ! and ? indicates subjectivity (often negative) emoticons: smiles like :) are generally positive, while frowns like :( are generally negative Use each pattern as a rule; if present in the text, the rule indicates whether the text is positive or negative. How to deal with conicts? (E.g. multiple rules apply, but indicate both positive and negative?) Simple: count number of matching rules and take the max. 31 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Simplest polarity classier ever? 32 def polarity(document) = if (document contains good) positive else if (document contains bad) negative else neutral Debate08 HCR STS IMDB 20.5 21.6 19.4 27.4 No better than ipping a (three-way) coin? Code and data here: https://github.com/utcompling/sastut Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The confusion matrix We need to look at the confusion matrix and breakdowns for each label. For example, here it is for Debate08: 33 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 + is positive, - is negative, ~ is neutral Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The confusion matrix We need to look at the confusion matrix and breakdowns for each label. For example, here it is for Debate08: 33 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 Corpus labels + is positive, - is negative, ~ is neutral Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The confusion matrix We need to look at the confusion matrix and breakdowns for each label. For example, here it is for Debate08: 33 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 Corpus labels Machine predictions + is positive, - is negative, ~ is neutral Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The confusion matrix We need to look at the confusion matrix and breakdowns for each label. For example, here it is for Debate08: 33 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 Total count of documents in the corpus Corpus labels Machine predictions + is positive, - is negative, ~ is neutral Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The confusion matrix We need to look at the confusion matrix and breakdowns for each label. For example, here it is for Debate08: 33 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 Total count of documents in the corpus Corpus labels Machine predictions Correct predictions + is positive, - is negative, ~ is neutral Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The confusion matrix We need to look at the confusion matrix and breakdowns for each label. For example, here it is for Debate08: 33 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 Total count of documents in the corpus Corpus labels Machine predictions Correct predictions + is positive, - is negative, ~ is neutral (5+140+18)/795 = 0.205 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The confusion matrix We need to look at the confusion matrix and breakdowns for each label. For example, here it is for Debate08: 33 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 Total count of documents in the corpus Corpus labels Machine predictions Correct predictions Incorrect predictions + is positive, - is negative, ~ is neutral (5+140+18)/795 = 0.205 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The confusion matrix We need to look at the confusion matrix and breakdowns for each label. For example, here it is for Debate08: 33 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 Total count of documents in the corpus Corpus labels Machine predictions Column showing outcomes of documents labeled negative by the machine Correct predictions Incorrect predictions + is positive, - is negative, ~ is neutral (5+140+18)/795 = 0.205 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The confusion matrix We need to look at the confusion matrix and breakdowns for each label. For example, here it is for Debate08: 33 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 Total count of documents in the corpus Corpus labels Machine predictions Row showing outcomes of documents labeled negative in the corpus Column showing outcomes of documents labeled negative by the machine Correct predictions Incorrect predictions + is positive, - is negative, ~ is neutral (5+140+18)/795 = 0.205 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Precision, Recall, and F-score: per category scores Precision: the number of correct guesses (true positives) for the category divided by all guesses for it (true positives and false positives) Recall: the number of correct guesses (true positives) for the category divided by all the true documents in that category (true positives plus false negatives) F-score: derived measure combining precision and recall. 34 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 P R F - ~ + Avg 83.3 1.1 2.2 18.3 99.3 30.1 72.0 9.0 16.0 57.9 36.5 16.4 P = TP/(TP+FP) R = TP/(TP+FN) F = 2PR/(P+R) P~ = 140+442+182 140 = .183 R- = 5+442+7 5 = .011 F+ = .72+.09 2 .72 .09 = .16 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 What does it tell us? Overall accuracy is low, because the model overpredicts neutral. Precision is pretty good for negative, and okay for positive. This means the simple rules has the word good and has the word bad are good predictors. 35 - ~ + - ~ + 5 442 7 454 1 140 0 141 0 182 18 200 6 764 25 795 P R F - ~ + Avg 83.3 1.1 2.2 18.3 99.3 30.1 72.0 9.0 16.0 57.9 36.5 16.4 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Where do the rules go wrong? Confusion matrix for STS: 36 The one negative-labeled tweet that is actually positive, using the very positive expression bad ass (thus matching bad). Booz Allen Hamilton has a bad ass homegrown social collaboration platform.Way cool! #ttiv - ~ + - ~ + 0 73 2 75 0 31 2 33 1 96 11 108 1 200 15 216 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 A bigger lexicon (rule set) and a better rule Good improvements for ve minutes of effort! Why such a large improvement for IMDB? 37 pos_words = {"good","awesome","great","fantastic","wonderful"} neg_words = {"bad","terrible","worst","sucks","awful","dumb"} def polarity(document) = num_pos = count of words in document also in pos_words num_neg = count of words in document also in neg_words if (num_pos == 0 and num_neg == 0) neutral else if (num_pos > num_neg) positive else negative Debate08 HCR STS IMDB Super simple Small lexicon 20.5 21.6 19.4 27.4 21.5 22.1 25.5 51.4 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 IMDB: no neutrals! Data is from 10 star movie ratings (>=7 are pos, 70.2 for Debate08, and 51.3 -> 60.5 for HRC). More labeled examples almost always help, especially if you have no in-domain training data (e.g. 56.5/54.2 -> 59.7 for STS). Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Accuracy isnt enough, part 1 The class balance can shift considerably without affecting the accuracy! 58 58+24+47 216 = 59.7 D08+HRC on STS - ~ + - ~ + 58 12 5 75 7 24 2 33 34 27 47 108 99 63 54 216 8+15+106 216 = 59.7 (Made up) Positive-heavy classier - ~ + - ~ + 8 12 55 75 7 24 11 33 1 1 106 108 16 28 172 216 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Accuracy isnt enough, part 1 Need to also consider the per-category precision, recall, and f-score. 59 - ~ + - ~ + 58 12 5 75 7 24 2 33 34 27 47 108 99 63 54 216 P R F - ~ + Avg 58.6 77.3 66.7 38.1 72.7 50.0 87.0 43.5 58.0 61.2 64.5 58.2 Acc: 59.7 Big differences in precision for the three categories! Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Accuracy isnt enough, part 2 Errors on neutrals are typically less grievous than positive/ negative errors, yet raw accuracy makes one pay the same penalty. 60 D08+HRC on STS One solution: allow varying penalties such that no points are awarded for positive/negative errors, but some partial credit is given for positive/neutral and negative/neutral ones. - ~ + - ~ + 58 12 5 75 7 24 2 33 34 27 47 108 99 63 54 216 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Accuracy isnt enough, part 3 Who says the gold standard is correct? There is often signicant variation among human annotators, especially for positive vs neutral and negative vs neutral. Solution one: work on your annotations (including creating conventions) until you get very high inter-annotator agreement. This arguably reduces the linguistic variability/subtlety characterized in the annotations. Also, humans often fail to get the intended sentiment, e.g. sarcasm. Solution two: measure performance differently. For example, given a set of examples annotated by three or more human annotators and the machine, is the machine distinguishable from the humans in terms of the amount it disagrees with their annotations? 61 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Accuracy isnt enough, part 4 Often, what is of interest is an aggregate sentiment for some topic or target. E.g. given a corpus of tweets about cars, 80% of the mentions of the Ford Focus are positive while 70% of the mentions of the Chevy Malibu are positive. Note: you can get the sentiment value wrong for some of the documents while still getting the overall, aggregate sentiment correct (as errors can cancel each other). Note also: generally, this requires aspect-based analysis (more later). 62 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Caveat emptor, part 1 In measuring accuracy, the methodology can vary dramatically from vendor to vendor, at times in unclear ways. For example, some seem to measure accuracy by presenting a human judge with examples annotated by a machine. The human then marks which examples they believe were incorrect. Accuracy is then num_correct/num_examples. Problem: people get lazy and often end up giving the machine the benet of the doubt. I have even heard that some vendors take their high- condence examples and do the above exercise. This is basically cheating: high-condence machine label assignments are on average more correct than low- condence ones. 63 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Caveat emptor, part 2 Performance on in-domain data is nearly always better than out-of-domain (see the previous experiments). The nature of the world is that the language of today is a step away from the language of yesterday (when you developed your algorithm or trained your model). Also, because there are so many things to talk about (and because people talk about everything), a given model is usually going to end up employed in domains it never saw in its training data. 64 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Caveat emptor, part 3 With nice, controlled datasets like those given previously, the experimenter has total control over which documents her algorithm is applied too. However, a deployed system will likely confront many irrelevant documents, e.g. documents written in other languages Sprint the company wants tweets by their customers, but also get many tweets of people talking about the activity of sprinting. documents that match, but which are not about the target of interest documents that should have matched, but were missed in retrieval Thus, identication of relevant documents and even sub- documents with relevant targets, is an important component of end-to-end sentiment solutions. 65 Wednesday, March 5, 14
  • Aspect-based sentiment analysis Why NLP is hard Sentiment analysis overview Document classication Visualization Semi-supervised learning Stylistics & author modeling Beyond text Wrap up Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Is it coherent to ask what the sentiment of a document is? Documents tend to discuss many entities and ideas, and they can express varying opinions, even toward the same entity. This is true even in tweets, e.g. positive towards the HCR bill negative towards Mitch McConnell 67 Here's a #hcr proposal short enough for Mitch McConnell to read: pass the damn bill now Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Fine-grained sentiment Two products, iPhone and Blackberry Overall positive to iPhone, negative to Blackberry Postive aspect/features of iPhone: touch screen, voice quality. Negative (for the mother): expensive. 68 Slide adapted from Bing Liu Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Components of ne-grained analysis Opinion targets: entities and their features/aspects Sentiment orientations: positive, negative, or neutral Opinion holders: persons holding the opinions Time: when the opinions are expressed 69 Slide adapted from Bing Liu Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 An entity e is a product, person, event, organization, or topic. e is represented as a hierarchy of components, sub-components, and so on. Each node represents a component and is associated with a set of attributes of the component. An opinion can be expressed on any node or attribute of the node. For simplicity, we use the term aspects (features) to represent both components and attributes. Entity and aspect (Hu and Liu, 2004; Liu, 2006) 70 iPhone screen battery {cost,size,appearance,...} {battery_life,size,...}{...} ... Slide adapted from Bing Liu Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Opinion denition (Liu, Ch. in NLP handbook, 2010) An opinion is a quintuple (e,a,so,h,t) where: e is a target entity. a is an aspect/feature of the entity e. so is the sentiment value of the opinion from the opinion holder h on feature a of entity e at time t. so is positive, negative or neutral (or more granular ratings). h is an opinion holder. t is the time when the opinion is expressed. Examples from the previous passage: 71 (iPhone, GENERAL, +,Abc123, 5-1-2008) (iPhone, touch_screen, +,Abc123, 5-1-2008) (iPhone, cost, -, mother_of(Abc123), 5-1-2008) Slide adapted from Bing Liu Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The goal: turn unstructured text into structured opinions Given an opinionated document (or set of documents) discover all quintuples (e,a, so, h, t) or solve a simpler form of it, such as the document level task considered earlier Having extracted the quintuples, we can feed them into traditional visualization and analysis tools. 72 Slide adapted from Bing Liu Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Several sub-problems e is a target entity: Named Entity Extraction (more) a is an aspect of e: Information Extraction so is sentiment: Sentiment Identication h is an opinion holder: Information/Data Extraction t is the time: Information/Data Extraction 73 Slide adapted from Bing Liu All of these tasks can make use of deep language processing methods, including parsing, coreference, word sense disambiguation, etc. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Named entity recognition Given a document, identify all text spans that mention an entity (person, place, organization, or other named thing). Requires having performed tokenization, and possibly part-of- speech tagging. Though it is a bracketing task, it can be transformed into a sequence task using BIO labels (Begin, Inside, Outside) Usually, discriminative sequence models like Maxent Markov Models and Conditional Random Fields are trained on such sequences, and used for prediction. 74 Mr. [John Smith]Person traveled to [NewYork City]Location to visit [ABC Corporation]Organization. Mr. John Smith traveled to New York City to visit ABC Corporation O B-PER I-PER O O B-LOC I-LOC I-LOC O O B-ORG I-ORG Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 OpenNLP Pipeline demo Sentence detection Tokenization Part-of-speech tagging Chunking NER: persons and organizations 75 PierreVinken, 61 years old, will join the board as a nonexecutive director Nov. 29. Mr.Vinken is chairman of Elsevier N.V., the Dutch publishing group. Rudolph Agnew, 55 years old and former chairman of Consolidated Gold Fields PLC, was named a director of this British industrial conglomerate. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Things are tricky in Twitterland - need domain adaptation 76 .@Peters4Michigan's camp tells me he'll appear w Obama in MI tomorrow. Not as scared as other Sen Dems of the prez: .[@Peters4Michigan]PER 's camp tells me he'll appear w [Obama]PER in [MI]LOC tomorrow. Not as scared as other Sen [Dems]ORG of the prez: Named entities referred to with @-mentions (makes things easier, but also harder for model solely trained on newswire text) Tokenization: many new innovations, including .@account at begining of tweet (which blocks it being an @-reply to that account) Abbreviations mess with features learned on standard text, e.g. w for with (as above), or even for George W. Bush: And who changed that? Remember Dems, many on foreign soil, criticizing W vehemently? Speaking of rooting against a Prez .... Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Identifying targets and aspects We can specify targets, their sub-components, and their attributes: But language is varied and evolving, so we are likely to miss many ways to refer to targets and their aspects. E.g. A person declaring knowledge about phones might forget (or not even know) that juice is a way of referring to power consumption. Also: there are many ways of referring to product lines (and their various releases, e.g. iPhone 4s) and their competitors, and we often want to identify these semi- automatically. Much research has worked on bootstrapping these. See Bing Lius tutorial for an excellent overview: http://www.cs.uic.edu/~liub/FBS/Sentiment-Analysis-tutorial-AAAI-2011.pdf 77 iPhone screen battery {cost,size,appearance,...} {battery_life,size,...}{...} ... Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Target-based feature engineering Given a sentence like We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. NER to identify the Porsche Panamera as the target Aspect identication to see that opinions are being expressed about the cars driving and styling. Sentiment analysis to identify positive sentiment toward the driving and negative toward the styling. Targeted sentiment analysis require positional features use string relationship to the target or aspect or use features from a parse of the sentence (if you can get it) 78 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 In addition to the standard document-level features used previously, we build features particularized for each target. These are just a subset of the many possible features. Positional features 79 We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 In addition to the standard document-level features used previously, we build features particularized for each target. These are just a subset of the many possible features. Positional features 79 We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 In addition to the standard document-level features used previously, we build features particularized for each target. These are just a subset of the many possible features. Positional features 79 We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 In addition to the standard document-level features used previously, we build features particularized for each target. These are just a subset of the many possible features. Positional features 79 We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. We love how the Porsche Panamera drives, but its bulbous exterior is unfortunately ugly. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Challenges Positional features greatly expands the space of possible features. We need more training data to estimate parameters for such features. Highly specic features increase the risk of overtting to whatever training data you have. Deep learning has a lot of potential to help with learning feature representations that are effective for the task by reducing the need for careful feature engineering. But obviously: we need to be able to use this sort of evidence in order to do the job well via automated means. 80 Wednesday, March 5, 14
  • Visualization Why NLP is hard Sentiment analysis overview Document classication Aspect-based sentiment analysis Semi-supervised learning Stylistics & author modeling Beyond text Wrap up Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Visualize, but be careful when doing so It's often the case that a visualization can capture nuances in the data that numerical or linguistic summaries cannot easily capture. Visualization is an art and a science in its own right. The following advice from Tufte (2001, 2006) is easy to keep in mind (if only so that your violations of it are conscious and motivated): Draw attention to the data, not the visualization. Use a minimum of ink. Avoid creating graphical puzzles. Use tables where possible. 82 Slide by Chris Potts Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Sentiment lexicons: SentiWordNet 83 Slide by Chris Potts Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter Sentiment results for Netix. 84 Slide by Chris Potts Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitrratr blends the data and summarization together 85 Slide by Chris Potts Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Relationships between modiers in WordNet similar-to graph 86 Slide by Chris Potts Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Relationships between modiers in WordNet similar-to graph 87 Slide by Chris Potts Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Visualizing discussions: Wikipedia deletions [http://notabilia.net/] 88 Could be used as a visualization for evolving sentiment over time in a discussion among many individuals. Wednesday, March 5, 14
  • Semi-supervised Learning Why NLP is hard Sentiment analysis overview Document classication Aspect-based sentiment analysis Visualization Stylistics & author modeling Beyond text Wrap up Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Scaling 90 Scaling for text analysis tasks typically requires more than big computation or big data. Most interesting tasks involve representations below the text itself. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Scaling 90 Scaling for text analysis tasks typically requires more than big computation or big data. Most interesting tasks involve representations below the text itself. Being big helps when you know what you are computing and how you can compute it. GIGO, and big garbage is still garbage. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Scaling 90 Scaling for text analysis tasks typically requires more than big computation or big data. Most interesting tasks involve representations below the text itself. Being big helps when you know what you are computing and how you can compute it. GIGO, and big garbage is still garbage. Scaling often requires being creative about how to learn f from relatively little explicit information about the task. Semi-supervised methods and indirect supervision to the rescue. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Scaling annotations 91 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Scaling annotations 91 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Scaling annotations 91 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Scaling annotations 91 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Scaling annotations 91 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Scaling annotations 91 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Scaling annotations 91 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Accurate tool Extremely low annotation 92 Annotation is relatively expensive Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Accurate tool Extremely low annotation 92 Annotation is relatively expensive Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Accurate tool Extremely low annotation ? 92 Annotation is relatively expensive Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Accurate tool Extremely low annotation ? 92 Annotation is relatively expensive We lack sufcient resources for most languages, most domains and most problems. Semi-supervised learning approaches become essential. See Philip Resniks SAS 2011 keynote: http://vimeo.com/32506363 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Example: Learning part-of-speech taggers 93 They often book ights . The red book fell . Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Example: Learning part-of-speech taggers 93 They often book ights . The red book fell . N Adv V N PUNC D Adj N V PUNC Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Example: Learning part-of-speech taggers 93 They often book ights . The red book fell . N Adv V N PUNC D Adj N V PUNC Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Example: Learning part-of-speech taggers 93 They often book ights . The red book fell . POS Taggers are usually trained on hundreds of thousands of annotated word tokens.What if we have almost nothing? N Adv V N PUNC D Adj N V PUNC Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 annotation HMM 94 The overall strategy: grow, shrink, learn Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 annotation HMMEM 94 The overall strategy: grow, shrink, learn Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 annotation HMMEM 94 The overall strategy: grow, shrink, learn Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 annotation HMM Tag Dict Generalization EM 94 The overall strategy: grow, shrink, learn Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 annotation HMM Tag Dict Generalization EM cover the vocabulary 94 The overall strategy: grow, shrink, learn Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 annotation HMM Model Minimization Tag Dict Generalization EM cover the vocabulary remove noise 94 The overall strategy: grow, shrink, learn Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 annotation HMM Model Minimization Tag Dict Generalization EM cover the vocabulary remove noise train 94 The overall strategy: grow, shrink, learn Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Extremely low annotation scenario [Garrette & Baldridge 2013] Obtain word types or tokens annotated with their parts-of-speech by a linguist in under two hours 95 the D book N, V often Adv red Adj, N Types: construct a tag dic.onary from scratch (not simulated) Tokens: standard word-by-word annota.on They often book ights . N Adv V N PUNC The red book fell . D Adj N V PUNC Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Strategy: connect annotations to raw corpus and propagate them 96 Raw Corpus Tokens TypesWednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Strategy: connect annotations to raw corpus and propagate them 96 Raw Corpus Tokens TypesWednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Strategy: connect annotations to raw corpus and propagate them 96 Raw Corpus Tokens TypesWednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Strategy: connect annotations to raw corpus and propagate them 96 Raw Corpus Tokens TypesWednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Strategy: connect annotations to raw corpus and propagate them 96 Raw Corpus Tokens TypesWednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Label propagation for video recommendation, in brief 97 Alice Bob Eve Basil Marceaux for Tennessee Governor Jimmy Fallon: Whip My Hair Radiohead: Paranoid Android Pink Floyd: The Wall (Full Movie) Local updates, so scales easily! Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 NEXT_walksPREV_ PREV_the PRE1_tPRE2_th SUF1_g TYPE_the TYPE_thug TYPE_dog Tag dictionary generalization 98 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 NEXT_walksPREV_ PREV_the PRE1_tPRE2_th SUF1_g TYPE_the TYPE_thug TYPE_dog Type Annotations ____________ ____________ the dog DT NN Tag dictionary generalization 98 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 NEXT_walksPREV_ PREV_the PRE1_tPRE2_th SUF1_g TYPE_the TYPE_thug TYPE_dog Type Annotations ____________ ____________ the dog DT NN Tag dictionary generalization 98 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 NEXT_walksPREV_ PREV_the PRE1_tPRE2_th SUF1_g TYPE_the TYPE_thug TYPE_dog Token Annotations ____________ ____________ Type Annotations ____________ ____________ the dog the dog walks DT NN VBZ DT NN Tag dictionary generalization 98 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 NEXT_walksPREV_ PREV_the PRE1_tPRE2_th SUF1_g TYPE_the TYPE_thug TYPE_dog Token Annotations ____________ ____________ Type Annotations ____________ ____________ the dog the dog walks DT NN DT NN Tag dictionary generalization 98 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 NEXT_walksPREV_ PREV_the PRE1_tPRE2_th SUF1_g TYPE_the TYPE_thug TYPE_dog DT NN DT NN Tag dictionary generalization 98 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN Tag dictionary generalization 99 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Tag dictionary generalization 99 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Tag dictionary generalization 99 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN Tag dictionary generalization 100 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN Tag dictionary generalization 100 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN Tag dictionary generalization 101 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN Tag dictionary generalization 101 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN Tag dictionary generalization 102 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN Tag dictionary generalization 102 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN Tag dictionary generalization 103 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN Tag dictionary generalization 103 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN Tag dictionary generalization 104 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 DT NN DT NN TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 Tag dictionary generalization 104 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 Tag dictionary generalization 104 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 Tag dictionary generalization 104 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 Tag dictionary generalization 105 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 Tag dictionary generalization 105 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Result: TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 Tag dictionary generalization 105 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Result: a tag distribution on every token TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 Tag dictionary generalization 105 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Result: a tag distribution on every token an expanded tag dictionary (non-zero tags) TOK_the_1 TOK_dog_2TOK_the_4 TOK_thug_5 Tag dictionary generalization 105 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 0 25 50 75 100 English Kinyarwanda Malagasy EM only EM only + Our approach + Our approach Tokens Types EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach Total Accuracy 106 Results (two hours of annotation) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 0 25 50 75 100 English Kinyarwanda Malagasy EM only EM only + Our approach + Our approach Tokens Types EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach Total Accuracy 106 Results (two hours of annotation) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 0 25 50 75 100 English Kinyarwanda Malagasy EM only EM only + Our approach + Our approach Tokens Types EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach Total Accuracy 106 Results (two hours of annotation) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 0 25 50 75 100 English Kinyarwanda Malagasy EM only EM only + Our approach + Our approach Tokens Types EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach Total Accuracy 106 Results (two hours of annotation) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 0 25 50 75 100 English Kinyarwanda Malagasy EM only EM only + Our approach + Our approach Tokens Types EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach Total Accuracy 106 Results (two hours of annotation) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 0 25 50 75 100 English Kinyarwanda Malagasy EM only EM only + Our approach + Our approach Tokens Types EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach EM only EM only + Our approach + Our approach Total Accuracy 106 Results (two hours of annotation) With 4 hours + a bit more 90% [Garrette, Mielens, & Baldridge 2013] Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Polarity classication for Twitter 107 Obama looks good. #tweetdebate #current+ - McCain is not answering the questions #tweetdebate Sen McCain would be a very popular President - $5000 tax refund per family! #tweetdebate+ - "it's like you can see Obama trying to remember all the "talking points" and get his slogans out there #tweetdebate" Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Polarity classication for Twitter 107 Obama looks good. #tweetdebate #current+ - McCain is not answering the questions #tweetdebate Sen McCain would be a very popular President - $5000 tax refund per family! #tweetdebate+ - "it's like you can see Obama trying to remember all the "talking points" and get his slogans out there #tweetdebate" Logistic regression... and... done! Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Polarity classication for Twitter 107 Obama looks good. #tweetdebate #current+ - McCain is not answering the questions #tweetdebate Sen McCain would be a very popular President - $5000 tax refund per family! #tweetdebate+ - "it's like you can see Obama trying to remember all the "talking points" and get his slogans out there #tweetdebate" Logistic regression... and... done! What if instance labels arent there? Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 No explicitly labeled examples? 108 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 No explicitly labeled examples? 108 Positive/negative ratio using polarity lexicon. Easy & works okay for many cases, but fails spectactularly elsewhere. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 No explicitly labeled examples? 108 Positive/negative ratio using polarity lexicon. Easy & works okay for many cases, but fails spectactularly elsewhere. Emoticons as labels + logistic regression. Easy, but emoticon to polarity mapping is actually vexed. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 No explicitly labeled examples? 108 Positive/negative ratio using polarity lexicon. Easy & works okay for many cases, but fails spectactularly elsewhere. Emoticons as labels + logistic regression. Easy, but emoticon to polarity mapping is actually vexed. Label propagation using the above as seeds. Noisy labels provide soft indicators, the graph smooths things out. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 No explicitly labeled examples? 108 Positive/negative ratio using polarity lexicon. Easy & works okay for many cases, but fails spectactularly elsewhere. Emoticons as labels + logistic regression. Easy, but emoticon to polarity mapping is actually vexed. Label propagation using the above as seeds. Noisy labels provide soft indicators, the graph smooths things out. If you have annotations, you can use those too. Including ordered labels like star ratings: see Talukdar & Crammer 2009 Wednesday, March 5, 14
  • Using social interaction: Twitter sentiment Obama is making the repubs look silly and petty bird images from http://www.mytwitterlayout.com/ http://starwars.wikia.com/wiki/R2-D2 Papers: Speriosu et al. 2011;Tan et al. KDD 2011 Wednesday, March 5, 14
  • Using social interaction: Twitter sentiment Obama is making the repubs look silly and petty bird images from http://www.mytwitterlayout.com/ http://starwars.wikia.com/wiki/R2-D2 Obama,silly,petty Papers: Speriosu et al. 2011;Tan et al. KDD 2011 Wednesday, March 5, 14
  • Using social interaction: Twitter sentiment Obama is making the repubs look silly and petty bird images from http://www.mytwitterlayout.com/ http://starwars.wikia.com/wiki/R2-D2 Obama,silly,petty = Papers: Speriosu et al. 2011;Tan et al. KDD 2011 Wednesday, March 5, 14
  • Using social interaction: Twitter sentiment Obama is making the repubs look silly and petty bird images from http://www.mytwitterlayout.com/ http://starwars.wikia.com/wiki/R2-D2 Papers: Speriosu et al. 2011;Tan et al. KDD 2011 Wednesday, March 5, 14
  • Using social interaction: Twitter sentiment Obama is making the repubs look silly and petty bird images from http://www.mytwitterlayout.com/ http://starwars.wikia.com/wiki/R2-D2 Papers: Speriosu et al. 2011;Tan et al. KDD 2011 Wednesday, March 5, 14
  • Using social interaction: Twitter sentiment Obama is making the repubs look silly and petty bird images from http://www.mytwitterlayout.com/ http://starwars.wikia.com/wiki/R2-D2 is happy Obama is president Obamas doing great! Papers: Speriosu et al. 2011;Tan et al. KDD 2011 Wednesday, March 5, 14
  • Using social interaction: Twitter sentiment Obama is making the repubs look silly and petty bird images from http://www.mytwitterlayout.com/ http://starwars.wikia.com/wiki/R2-D2 is happy Obama is president Obamas doing great! Obama,silly,petty Papers: Speriosu et al. 2011;Tan et al. KDD 2011 Wednesday, March 5, 14
  • Using social interaction: Twitter sentiment Obama is making the repubs look silly and petty bird images from http://www.mytwitterlayout.com/ http://starwars.wikia.com/wiki/R2-D2 is happy Obama is president Obamas doing great! = (hopefully) Obama,silly,petty Papers: Speriosu et al. 2011;Tan et al. KDD 2011 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 Alice I love #NY! :) Ahhh #Obamacare Bob We cant pass this :( #killthebill I hate #Obamacare! #killthebill Eve We need health care! Lets get it passed! :) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 Alice I love #NY! :) Ahhh #Obamacare Bob We cant pass this :( #killthebill I hate #Obamacare! #killthebill Eve We need health care! Lets get it passed! :) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 Alice I love #NY! :) Ahhh #Obamacare Bob We cant pass this :( #killthebill I hate #Obamacare! #killthebill Eve We need health care! Lets get it passed! :) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 Alice I love #NY! :) Ahhh #Obamacare Bob We cant pass this :( #killthebill I hate #Obamacare! #killthebill Eve We need health care! Lets get it passed! :) Hashtags killthebillobamacareny Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 Wordn-grams we cant love ny i love Alice I love #NY! :) Ahhh #Obamacare Bob We cant pass this :( #killthebill I hate #Obamacare! #killthebill Eve We need health care! Lets get it passed! :) Hashtags killthebillobamacareny Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 OpinionFinder care hate love Wordn-grams we cant love ny i love Alice I love #NY! :) Ahhh #Obamacare Bob We cant pass this :( #killthebill I hate #Obamacare! #killthebill Eve We need health care! Lets get it passed! :) Hashtags killthebillobamacareny Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 OpinionFinder care hate love Wordn-grams we cant love ny i love Emoticons ;-):(:) Alice I love #NY! :) Ahhh #Obamacare Bob We cant pass this :( #killthebill I hate #Obamacare! #killthebill Eve We need health care! Lets get it passed! :) Hashtags killthebillobamacareny Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 OpinionFinder care hate love Wordn-grams we cant love ny i love Emoticons ;-):(:) Alice I love #NY! :) Ahhh #Obamacare Bob We cant pass this :( #killthebill I hate #Obamacare! #killthebill Eve We need health care! Lets get it passed! :) Hashtags killthebillobamacareny - + + Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 OpinionFinder care hate love Wordn-grams we cant love ny i love Emoticons ;-):(:) Alice I love #NY! :) Ahhh #Obamacare Bob We cant pass this :( #killthebill I hate #Obamacare! #killthebill Eve We need health care! Lets get it passed! :) Hashtags killthebillobamacareny + +- - + + Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Twitter polarity graph with knowledge and noisy seeds 110 OpinionFinder care hate love Wordn-grams we cant love ny i love Emoticons ;-):(:) Alice I love #NY! :) Ahhh #Obamacare Bob We cant pass this :( #killthebill I hate #Obamacare! #killthebill Eve We need health care! Lets get it passed! :) Hashtags killthebillobamacareny + +- - + + + - + - + - + - + - + - Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Results: polarity assignment (positive/negative, no neutral) 111 Stanford Twitter Sentiment Obama-McCain Debate Health Care Reform Random Lexicon Ratio Emoticon-trained (Logistic regression) Label propagation 50.0 50.0 50.0 72.1 59.1 58.1 83.1 61.3 62.9 84.7 66.7 71.2 Take-home message: label propagation can make effective use of labeled features (from external knowledge sources) and noisy annotations. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Lets not forget scalable human annotation Mechanical Turk: can work well, but also problematic (e.g. lack of workers for many languages). Also real-time polling and reactions, e.g., ReactLabs. 112 Wednesday, March 5, 14
  • Stylistics & author modeling Why NLP is hard Sentiment analysis overview Document classication Aspect-based sentiment analysis Visualization Semi-supervised learning Beyond text Wrap up Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Content-based analysis vs stylistics For general categorization tasks, the content words are the most helpful. E.g. to know whether a document is about sports or nance, it helps to know it contains baseball, umpire and game versus money, stocks, and bonds. Often we lter so-called stop-words when creating features for such tasks. Stylistics: tasks of interest include authorship attribution, status, depression, deceit, demographics, and more. Stylistics is different from content categorizaiton: the subtle differences in use of function words is very important. E.g. in authorship attribution studies, content words are often ltered out. The stop words instead become the keep words! 114 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Author comparison: match texts on left to those by same author on the right 115 His manner was not effusive. It seldom was; but he was glad, I think, to see me.With hardly a word spoken, but with a kindly eye, he waved me to an armchair, threw across his case of cigars, and indicated a spirit case and a gasogene in the corner. Then he stood before the re and looked me over in his singular introspective fashion. For all the preposterous hat and the vacuous face, there was something noble in the simple faith of our visitor which compelled our respect. She laid her little bundle of papers upon the table and went her way, with a promise to come again whenever she might be summoned. He was invited to Kellynch Hall; he was talked of and expected all the rest of the year; but he never came.The following spring he was seen again in town, found equally agreeable, again encouraged, invited, and expected, and again he did not come; and the next tidings were that he was married. There are many theories about what happened, but two general narratives seem to be gaining prominence, which we will call the greed narrative and the stupidity narrative.The two overlap, but they lead to different ways of thinking about where we go from here. He was not an ill-disposed young man, unless to be rather cold hearted and rather selsh is to be ill-disposed: but he was, in general, well respected; for he conducted himself with propriety in the discharge of his ordinary duties. Had he married a more amiable woman, he might have been made still more respectable than he was. Our moral and economic system is based on individual responsibility. Its based on the idea that people have to live with the consequences of their decisions.This makes them more careful deciders.This means that society tends toward justice people get what they deserve as much as possible. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Author comparison: match texts on left to those by same author on the right 115 His manner was not effusive. It seldom was; but he was glad, I think, to see me.With hardly a word spoken, but with a kindly eye, he waved me to an armchair, threw across his case of cigars, and indicated a spirit case and a gasogene in the corner. Then he stood before the re and looked me over in his singular introspective fashion. For all the preposterous hat and the vacuous face, there was something noble in the simple faith of our visitor which compelled our respect. She laid her little bundle of papers upon the table and went her way, with a promise to come again whenever she might be summoned. He was invited to Kellynch Hall; he was talked of and expected all the rest of the year; but he never came.The following spring he was seen again in town, found equally agreeable, again encouraged, invited, and expected, and again he did not come; and the next tidings were that he was married. There are many theories about what happened, but two general narratives seem to be gaining prominence, which we will call the greed narrative and the stupidity narrative.The two overlap, but they lead to different ways of thinking about where we go from here. He was not an ill-disposed young man, unless to be rather cold hearted and rather selsh is to be ill-disposed: but he was, in general, well respected; for he conducted himself with propriety in the discharge of his ordinary duties. Had he married a more amiable woman, he might have been made still more respectable than he was. Our moral and economic system is based on individual responsibility. Its based on the idea that people have to live with the consequences of their decisions.This makes them more careful deciders.This means that society tends toward justice people get what they deserve as much as possible. Brooks, NewYork Times, Apr 2, 2009 Brooks, NewYork Times, Feb 19, 2009 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Author comparison: match texts on left to those by same author on the right 115 His manner was not effusive. It seldom was; but he was glad, I think, to see me.With hardly a word spoken, but with a kindly eye, he waved me to an armchair, threw across his case of cigars, and indicated a spirit case and a gasogene in the corner. Then he stood before the re and looked me over in his singular introspective fashion. For all the preposterous hat and the vacuous face, there was something noble in the simple faith of our visitor which compelled our respect. She laid her little bundle of papers upon the table and went her way, with a promise to come again whenever she might be summoned. He was invited to Kellynch Hall; he was talked of and expected all the rest of the year; but he never came.The following spring he was seen again in town, found equally agreeable, again encouraged, invited, and expected, and again he did not come; and the next tidings were that he was married. There are many theories about what happened, but two general narratives seem to be gaining prominence, which we will call the greed narrative and the stupidity narrative.The two overlap, but they lead to different ways of thinking about where we go from here. He was not an ill-disposed young man, unless to be rather cold hearted and rather selsh is to be ill-disposed: but he was, in general, well respected; for he conducted himself with propriety in the discharge of his ordinary duties. Had he married a more amiable woman, he might have been made still more respectable than he was. Our moral and economic system is based on individual responsibility. Its based on the idea that people have to live with the consequences of their decisions.This makes them more careful deciders.This means that society tends toward justice people get what they deserve as much as possible. Doyle, Sherlock Holmes,A Scandal in Bohemia, 1891 Brooks, NewYork Times, Apr 2, 2009 Brooks, NewYork Times, Feb 19, 2009 Doyle, Sherlock Holmes,A Case of Identity, 1891 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Author comparison: match texts on left to those by same author on the right 115 His manner was not effusive. It seldom was; but he was glad, I think, to see me.With hardly a word spoken, but with a kindly eye, he waved me to an armchair, threw across his case of cigars, and indicated a spirit case and a gasogene in the corner. Then he stood before the re and looked me over in his singular introspective fashion. For all the preposterous hat and the vacuous face, there was something noble in the simple faith of our visitor which compelled our respect. She laid her little bundle of papers upon the table and went her way, with a promise to come again whenever she might be summoned. He was invited to Kellynch Hall; he was talked of and expected all the rest of the year; but he never came.The following spring he was seen again in town, found equally agreeable, again encouraged, invited, and expected, and again he did not come; and the next tidings were that he was married. There are many theories about what happened, but two general narratives seem to be gaining prominence, which we will call the greed narrative and the stupidity narrative.The two overlap, but they lead to different ways of thinking about where we go from here. He was not an ill-disposed young man, unless to be rather cold hearted and rather selsh is to be ill-disposed: but he was, in general, well respected; for he conducted himself with propriety in the discharge of his ordinary duties. Had he married a more amiable woman, he might have been made still more respectable than he was. Our moral and economic system is based on individual responsibility. Its based on the idea that people have to live with the consequences of their decisions.This makes them more careful deciders.This means that society tends toward justice people get what they deserve as much as possible. Doyle, Sherlock Holmes,A Scandal in Bohemia, 1891 Brooks, NewYork Times, Apr 2, 2009 Austen, Persuasion, 1818 Brooks, NewYork Times, Feb 19, 2009 Austen, Sense and Sensibility, 1811 Doyle, Sherlock Holmes,A Case of Identity, 1891 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Quantitative features that help discriminate the authors Grammatical person: 1st (we/us/our, I/me/my) Grammatical tense: present, past Word frequencies: frequent use of he Punctuation: use of colons and semi-colons Average word and sentence length Syntax: prepositional adverbial phrases (With hardly..., For all the...) These must be counted in all texts. The texts of unknown authorship should then have values most similar to those of the texts of one of the known authors. 116 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Forensic linguistics Forensic linguistics is a branch of applied linguistics that applies linguistic theory, research and principles to real life language in the legal context. Even more generally, it can be viewed as analyzing examples of language to discover properties that reveal more than just what is said. authorship (same as other examples?, plagiarism) psychological attributes of the author (deception, depression) similarity to other examples (e.g., trademark disputes) 117 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Linguistic ngerprinting and identication There has been much interest in nding linguistic ngerprints, but there are problems with the concept: language acquisition: language is learned and continually changing linguistic homogeneity: education, mass media register: the same person speaks differently in different contexts, with different people No accepted denition of general linguistic ngerprint has so far been proposed, nor are we likely to see one for these reasons. Nonetheless, people do exhibit regularities in their speech and writing that could distinguish them from others. This allows us to compare a limited set of authors/speakers in certain restricted conditions, just as we did with the rst page of these slides. 118 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Identifying style Every speaker uses language differently, leading to a unique style. Style is both: a collection of markers which can be observed and measured a set of unconscious habits which can be observed and measured Quantifying style: word usage: presence/absence of words, relative word frequencies type/token ratios average word and sentence length the number of unique words (hapax legomena) 119 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Authorship attribution with machine learning Machine learning provides a class of algorithms that perform unsupervised clustering. They dont have labels for any of the data points (e.g., documents). Based on properties measured from the data points, coherent clusters of documents with similar properties can be identied. A cluster can correspond to many different things, including collections of documents by the same author. Mixture models: a popular class of probabilistic algorithms for clustering. collections of probability distributions over the data soft cluster membership: points are proportionally part of multiple clusters a mixture of Gaussian distributions (a.k.a. the normal distribution) are one of the most commonly used type of mixture model The K-means algorithm is a related hard clustering algorithm that is simple and easy to understand. 120 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Hard clustering into k groups Assume you can measure various attributes for each data point, e.g.: the weight and top speed of various vehicles the average sentence length and average word length of various authors. Next, you want to identify k groups of similar items based on these attributes. How many groups? How to nd them using an algorithm? 121 weight top speed Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Hard clustering into k groups Assume you can measure various attributes for each data point, e.g.: the weight and top speed of various vehicles the average sentence length and average word length of various authors. Next, you want to identify k groups of similar items based on these attributes. How many groups? How to nd them using an algorithm? 121 weight top speed k=2? Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Hard clustering into k groups Assume you can measure various attributes for each data point, e.g.: the weight and top speed of various vehicles the average sentence length and average word length of various authors. Next, you want to identify k groups of similar items based on these attributes. How many groups? How to nd them using an algorithm? 121 weight top speed k=2? Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Hard clustering into k groups Assume you can measure various attributes for each data point, e.g.: the weight and top speed of various vehicles the average sentence length and average word length of various authors. Next, you want to identify k groups of similar items based on these attributes. How many groups? How to nd them using an algorithm? 121 weight top speed k=3?k=2? weight top speed Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Hard clustering into k groups Assume you can measure various attributes for each data point, e.g.: the weight and top speed of various vehicles the average sentence length and average word length of various authors. Next, you want to identify k groups of similar items based on these attributes. How many groups? How to nd them using an algorithm? 121 weight top speed k=3?k=2? weight top speed Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 An authorship clustering problem Texts from three authors (ve documents each) Arthur Conan Doyle (obtained from Project Gutenberg) Jane Austen (obtained from Project Gutenberg) Paul Krugman (obtained from New York Times website) Measure the relative frequency of the words I and the in each document. 122 Document I the Emma 1.8 3.2 Manseld 1.5 3.9 Persuasion 1.3 4.0 Pride 1.7 3.6 Sense 1.6 3.4 Document I the City 2.1 4.7 Gerard 3.6 6.1 Holmes 2.8 5.3 Hound 2.5 5.6 Polestar 2.3 6.2 Document I the 12-01-2008 0.0 6.7 12-07-2008 0.0 5.6 12-15-2008 0.3 6.7 12-19-2008 0.1 6.4 12-22-2008 0.4 6.3 Austen Doyle Krugman Now, forgetting who the authors are, lets see if they fall into distinct clusters based on these attributes. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Plot the attributes against each other 123 0 1.75 3.5 5.25 7 0 1 2 3 4 Austen Doyle Krugman Relative frequency of I Relativefrequencyofthe Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 K-means: intuition We see the clusters quite clearly, but a computer doesnt and we need to specify an algorithm that allows it to identify them. The K-means algorithm is a simple algorithm for such tasks. The basic idea: the values for the attributes in each dimension will be similar for each document of the same author each author is represented as the averages for the attributes of all the documents he or she wrote but: we dont know those averages since we forgot the authors! so, we take a guess at the average for each author, and then see which documents each of our hypothesized authors were likely to have produced these guesses will probably be wrong, but we can x that by iteratively re- estimating them 124 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 K-means: algorithm We are given N documents: D = d1, d2, ..., dN We need to output K centroids: C = c1, c2, ..., cK These centroids partition the documents into clusters based on which centroid is closest to each document. 125 K-means(D,K) C SelectRandomCenters(D,K) while C does change for k 1 to K gk {} for n 1 to N j argmini distance(ci,dn) gj gj {dn} for k 1 to K ck return C d d in gk 1 |gk| [Based on Manning, Raghavan, and Schutze 2008] Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 K-means: algorithm We are given N documents: D = d1, d2, ..., dN We need to output K centroids: C = c1, c2, ..., cK These centroids partition the documents into clusters based on which centroid is closest to each document. 125 K-means(D,K) C SelectRandomCenters(D,K) while C does change for k 1 to K gk {} for n 1 to N j argmini distance(ci,dn) gj gj {dn} for k 1 to K ck return C d d in gk 1 |gk| Pick K random points (could be some of the data points in D) [Based on Manning, Raghavan, and Schutze 2008] Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 K-means: algorithm We are given N documents: D = d1, d2, ..., dN We need to output K centroids: C = c1, c2, ..., cK These centroids partition the documents into clusters based on which centroid is closest to each document. 125 K-means(D,K) C SelectRandomCenters(D,K) while C does change for k 1 to K gk {} for n 1 to N j argmini distance(ci,dn) gj gj {dn} for k 1 to K ck return C d d in gk 1 |gk| Pick K random points (could be some of the data points in D) Stopping criteria (when does the algorithm stop?) [Based on Manning, Raghavan, and Schutze 2008] Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 K-means: algorithm We are given N documents: D = d1, d2, ..., dN We need to output K centroids: C = c1, c2, ..., cK These centroids partition the documents into clusters based on which centroid is closest to each document. 125 K-means(D,K) C SelectRandomCenters(D,K) while C does change for k 1 to K gk {} for n 1 to N j argmini distance(ci,dn) gj gj {dn} for k 1 to K ck return C d d in gk 1 |gk| Pick K random points (could be some of the data points in D) Stopping criteria (when does the algorithm stop?) (Re)initialize the document clusters. [Based on Manning, Raghavan, and Schutze 2008] Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 K-means: algorithm We are given N documents: D = d1, d2, ..., dN We need to output K centroids: C = c1, c2, ..., cK These centroids partition the documents into clusters based on which centroid is closest to each document. 125 K-means(D,K) C SelectRandomCenters(D,K) while C does change for k 1 to K gk {} for n 1 to N j argmini distance(ci,dn) gj gj {dn} for k 1 to K ck return C d d in gk 1 |gk| Pick K random points (could be some of the data points in D) Stopping criteria (when does the algorithm stop?) Find the closest centroid for each document; put the document in that group. (Re)initialize the document clusters. [Based on Manning, Raghavan, and Schutze 2008] Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 K-means: algorithm We are given N documents: D = d1, d2, ..., dN We need to output K centroids: C = c1, c2, ..., cK These centroids partition the documents into clusters based on which centroid is closest to each document. 125 K-means(D,K) C SelectRandomCenters(D,K) while C does change for k 1 to K gk {} for n 1 to N j argmini distance(ci,dn) gj gj {dn} for k 1 to K ck return C d d in gk 1 |gk| Pick K random points (could be some of the data points in D) Stopping criteria (when does the algorithm stop?) Find the closest centroid for each document; put the document in that group. Recompute centroids based on the new document clusters (the gks). (Re)initialize the document clusters. [Based on Manning, Raghavan, and Schutze 2008] Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Calculating distance The documents are data points in some (possibly high- dimensional) space. Well work with 2D here. Recall the Pythagorean theorem: c2 = a2 + b2 Here, the a is the distance on the x-axis and the b is the distance on the y-axis between points di and dj. distance(di,dj) = (xi - xj)2 + (yi - yj)2 Consider two data points d1 = (5,4) and d2 = (1,2). distance(d1, d2) = (x1 - x2)2 + (y1 - y2)2 = (5-1)2 + (4-2)2 = 42 + 22 = 20 Note: we could take the square root, but it doesnt matter since we are just comparing a bunch of squared distances. 126 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Simplied problem: just two authors and four documents Lets apply the K-means algorithm to four documents Keep in mind that we are acting like we dont know who is the author of each document. 127 Document I the Manseld 1.5 3.9 Persuasion 1.3 4.0 Document I the Gerard 3.6 6.1 Holmes 2.8 5.3 Austen Doyle D = {d1, d2, d3, d4} = { (1.5,3.9), (1.3,4.0), (3.6,6.1), (2.8,5.3) } Choose K = 2 (i.e., 2 authors) Choose C = {c1, c2} = { (3.6,6.1), (2.8,5.3) } as initial seed centroids. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Heres what it looks like 128 3 4 5 6 7 1 1.75 2.5 3.25 4 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Heres what it looks like 128 3 4 5 6 7 1 1.75 2.5 3.25 4 d1 d2 c1 d3 c2 d4 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Computing the groups Calculate the nearest centroid for each document and put it in the group for that centroid. d1: distance(c1,d1) = (3.6-1.5)2 + (6.1-3.9)2 = 9.25 distance(c2,d1) = (2.8-1.5)2 + (5.3-3.9)2 = 3.65 c2 is closer, so d1 is in g2 Doing this for d2, d3, and d4, we nd that: g1 = {d3} and g2 = {d1,d2,d4} 129 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Heres what it looks like 130 3 4 5 6 7 1 1.75 2.5 3.25 4 d1 d2 c1 d3 c2 d4 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 New centroids Next, we need to compute the new centroids based on these groups. g2 has multiple elements: sum of the x-values: 1.5+1.3+2.8 = 5.6 sum of the y-values: 3.9+4.0+5.3 = 13.2 size of g2 is 3, so c2 = (5.6/3, 13.2/3) = (1.9, 4.4) g1 stays the same: size of g1 is 1, so we have c1 = (3.6/1, 6.1/1) = (3.6,6.1) 131 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Heres what it looks like 132 3 4 5 6 7 1 1.75 2.5 3.25 4 d1 d2 c1 d3 c2 d4 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Re-assign groups based on new centroids We then keep iterating until the centroids stay the same calculate nearest centroid for each document, put in in the group recalculate centroids for new groups Notice that d4 is now closer to c1 distance(c1,d4) = (3.6-2.8)2 + (6.1-5.3)2 = 1.28 distance(c2,d4) = (1.9-2.8)2 + (4.4-5.3)2 = 1.62 133 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Heres what it looks like 134 3 4 5 6 7 1 1.75 2.5 3.25 4 d1 d2 c1 d3 c2 d4 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 The next round would be... 135 3 4 5 6 7 1 1.75 2.5 3.25 4 d1 d2 c1 d3 c2 d4 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 With the right groups 136 3 4 5 6 7 1 1.75 2.5 3.25 4 d1 d2 c1 d3 c2 d4 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Running k-means on all the documents 137 3 4 5 6 7 0 1 2 3 4 Relative frequency of I Relativefrequencyofthe Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Running k-means on all the documents 137 3 4 5 6 7 0 1 2 3 4 Relative frequency of I Relativefrequencyofthe Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Running k-means on all the documents 137 3 4 5 6 7 0 1 2 3 4 Relative frequency of I Relativefrequencyofthe Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Running k-means on all the documents 137 3 4 5 6 7 0 1 2 3 4 Relative frequency of I Relativefrequencyofthe Wrong cluster! Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Back to authorship identication Features were extracted from the documents and used as values for plotting each document in a multi-dimensional space. Documents were then clustered according to K-means (other algorithms could be used). K-means gave us a set of centroids, so we can plot other documents into the same multi-dimensional space and compute which one is closest. 138 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 We are given new documents of known authorship 139 3 4 5 6 7 0 1 2 3 4 Relative frequency of I Relativefrequencyofthe Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 We are given new documents of known authorship 139 3 4 5 6 7 0 1 2 3 4 Relative frequency of I Relativefrequencyofthe Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 We are given new documents of known authorship 139 3 4 5 6 7 0 1 2 3 4 Relative frequency of I Relativefrequencyofthe Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 We are given new documents of known authorship 139 3 4 5 6 7 0 1 2 3 4 Relative frequency of I Relativefrequencyofthe (Austen) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Attribution The known documents provide evidence that the clusters we found are sets of documents produced by the author(s) of the known documents. We can estimate the condence in our authorship assignments based on how close the known documents are to each center. 140 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 A case study: The Federalist Papers 85 essays written in 1787 and 1788 arguing for the ratication of the new US constitution by the individual states. Three authors, Alexander Hamilton, John Jay and James Madison, all writing under the pseudonym Publius. Later, Hamilton and Madison both claimed to have written a number of the same articles. Scholarship in the 20th century revealed most of them to be Madisons. 141 Alexander Hamilton 1st US Secretary of theTreasury. 51 articles (nos. 1, 69, 1113, 15 17, 2136, 5961, and 6585); co- authored 18, 19 & 20 w/ Madison. James Madison 4th US President,Father of the Constitution 29 articles (nos. 10, 14, 3758 and 6263); co-authored 18, 19 & 20 w/ Hamilton. John Jay 1st Chief Justice of the US. 5 articles (nos. 2-5 and 64) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Authorship of the Federalist Papers Historian Douglas Adair in 1944 argued that Madison was the author of many of the disputed papers. This was conrmed by Mosteller and Wallace in 1964 using a Bayesian classication model. Adairs authorship determinations are still generally accepted, though twelve essays are still disputed over by some scholars. Experiment: cluster the documents based on all words that occur 5 or more times and k-means. (With some principal components analysis in between). 142 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Extracting features: frequent words and their counts 143 FEDERALIST No. 1 General Introduction For the Independent Journal. Saturday, October 27, 1787 HAMILTON To the People of the State of New York: AFTER an unequivocal experience of the inefficacy of the subsisting federal government, you are called upon to deliberate on a new Constitution for the United States of America. The subject speaks its own importance; comprehending in its consequences nothing less than the existence of the UNION, the safety and welfare of the parts of which it is composed, the fate of an empire in many respects the most interesting in the world. It has been frequently remarked that it seems to have been reserved to the people of this country, by their conduct and example, to decide the important question, whether societies of men are really capable or not of establishing good government from reflection and choice, or whether they are forever destined to depend for their political constitutions on accident and force. If there be any truth in the remark, the crisis at which we are arrived may with propriety be regarded as the era in which that decision is to be made; and a wrong election of the part we shall act may, in this view, deserve to be considered as the general misfortune of mankind. .... .... ID Author NumWords . people for jury macedon one power with an as at to more states its be by this upon government them they has the not that than a ; but state courts , is it in if may have executive would been no constitution and any on of or there all his are from their which will other 1 HAMILTON 1771 49 5 12 0 0 4 2 6 11 10 8 71 7 2 10 34 14 14 6 9 2 6 6 130 14 28 11 25 7 2 5 0 105 13 20 27 4 11 10 0 2 3 3 8 40 6 9 104 6 2 9 0 12 11 14 18 25 3 2 JAY 1841 40 21 13 0 0 10 1 13 1 15 10 52 5 2 5 15 10 14 1 9 4 22 6 105 10 44 5 29 11 8 0 0 122 16 38 34 3 4 17 0 5 8 1 0 83 1 8 81 10 0 4 0 6 4 21 11 2 4 3 JAY 1604 37 7 11 0 0 8 3 10 3 24 1 55 13 11 1 31 18 6 0 16 8 5 5 91 13 20 8 13 7 7 7 3 117 7 21 25 6 6 7 1 2 2 2 0 60 5 6 60 32 1 4 2 8 15 11 11 24 7 4 JAY 1806 32 7 12 0 0 13 2 12 3 20 2 50 13 1 9 26 14 1 0 16 12 17 1 84 14 17 9 16 10 10 5 0 125 10 28 24 12 10 9 0 17 2 1 0 90 5 11 70 24 3 4 2 11 8 19 10 15 11 5 JAY 1475 35 2 7 0 0 10 1 11 3 3 4 44 11 1 4 31 10 6 0 2 11 11 0 64 8 23 9 9 8 4 1 0 86 7 20 28 3 2 1 0 37 0 2 0 72 3 5 51 10 0 4 0 3 11 11 10 6 4 6 HAMILTON 2528 77 3 13 0 0 3 4 15 12 19 6 60 10 7 1 18 12 11 4 2 3 13 10 187 17 24 6 52 10 5 5 0 174 8 11 60 5 3 27 0 6 13 0 1 81 0 5 166 19 8 6 3 18 16 15 24 6 10 7 HAMILTON 2562 81 0 14 0 0 4 2 12 15 19 11 81 6 31 2 47 28 22 11 2 12 7 4 205 14 27 5 48 16 3 7 0 158 14 17 43 12 3 23 0 51 8 3 1 51 7 13 156 10 9 12 0 7 11 18 24 1 9 8 HAMILTON 2306 72 8 9 0 0 6 5 13 13 16 11 79 8 10 10 35 11 19 3 3 11 13 8 156 12 18 3 46 17 10 6 0 164 21 21 42 7 8 18 2 27 16 4 4 54 2 11 133 17 2 9 0 14 9 16 26 11 6 9 HAMILTON 2213 68 2 10 0 0 12 3 14 12 18 10 70 6 12 5 26 13 14 4 17 5 20 12 168 13 24 1 46 14 6 6 1 133 16 23 37 6 5 24 0 8 15 2 3 45 4 9 153 11 3 8 4 14 7 14 25 7 5 10 MADISON 3337 79 4 18 0 0 8 1 11 14 20 8 99 12 3 9 61 39 11 0 13 8 11 4 259 14 31 13 78 32 11 2 0 221 41 47 63 6 16 16 0 6 9 5 2 121 4 18 155 22 6 4 8 26 12 21 39 30 22 11 HAMILTON 2766 78 0 18 0 0 5 3 21 15 21 11 82 9 13 6 44 20 24 6 5 6 10 8 186 20 24 6 69 8 8 4 0 173 9 21 66 6 9 15 0 50 3 5 0 70 4 5 178 12 8 13 1 9 19 11 25 17 10 12 HAMILTON 2410 72 3 8 0 0 5 1 18 11 17 13 81 5 10 8 40 15 17 7 8 6 8 13 174 6 27 4 47 15 8 11 0 161 22 26 54 7 3 13 0 22 8 2 0 62 7 12 139 7 9 5 2 15 19 14 23 11 14 13 HAMILTON 1045 28 2 1 0 0 8 5 10 3 8 1 42 6 9 2 25 5 5 2 7 1 3 2 72 5 18 10 26 6 2 3 0 53 9 6 14 6 8 4 0 14 2 4 0 17 3 3 56 3 9 3 0 4 5 1 11 9 1 14 MADISON 2357 53 5 20 0 0 2 3 7 9 24 7 71 8 12 8 45 18 13 0 9 4 17 8 200 13 33 5 37 19 4 1 0 144 25 31 40 6 16 17 0 5 9 10 2 60 3 17 122 11 0 6 0 6 11 19 40 26 7 15 HAMILTON 3395 83 4 18 0 0 1 9 19 18 15 24 116 6 11 8 44 32 24 10 14 4 15 14 251 16 38 10 69 25 8 5 1 186 48 30 73 7 7 31 0 13 14 6 4 74 3 10 194 38 18 21 1 22 22 7 55 19 6 < ... 70 more lines for the other essays ...> Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Dimensionality reduction: principal components analysis 144 Author Color Hamilton green Madison purple Jay cyan Hamilton & Madison red Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 K-means clustering 145 Three clusters (k=3): the model thinks the collaborations were most similar to Madison, plus three of Hamiltons. Author Cluster COLLABMH HAMILTON JAY MADISON 1 3 3 0 26 2 0 48 0 0 3 0 0 5 0 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 K-means clustering 145 With no changing of the features or parameters, amazingly close to the authorship attributions given by Adair. Three clusters (k=3): the model thinks the collaborations were most similar to Madison, plus three of Hamiltons. Author Cluster COLLABMH HAMILTON JAY MADISON 1 3 3 0 26 2 0 48 0 0 3 0 0 5 0 Author Cluster COLLABMH HAMILTON JAY MADISON 1 3 0 0 1 2 0 0 5 0 3 0 0 0 25 4 0 51 0 0 Four clusters (k=4): Cleanly clustered. One article (#47) attributed to Madison is astray, but it sensibly goes with the collaborative articles. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Web-scale authorship attribution Narayanan et al. (2012) On the Feasibility of Internet-Scale Author Identication The paper shows, for the rst time, that large-scale authorship attribution is feasible (in their case, testing with 100,000 authors) and accurate enough to be useful in authorship attribution use cases. For example, in their experiments, in 35% of their test cases, the correct author is in the top 20 authors predicted by their model. Given that this is out of 100,000 authors, that is quite signicant. 146 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Supervised learning and clustering Simple note: clusters are often used as features for learning supervised classiers. E.g. cluster words according to their syntactic contexts and use the cluster ids as the features. This can help with some word-sense disambiguation without using a predened set of word senses. 147 He paid a lot of interest after he took out that bank loan to buy his house. C2 He paid a lot of interest to the lecture he saw onYouTube. C2 C2 C5C8 C6 C6 C42 C12 Clusters can also themselves be labeled and used for training! C31 Wednesday, March 5, 14
  • Relative status: who has the lead? Communicative behaviors are patterned and coordinated, like a dance[Niederhoffer and Pennebaker, 02] http://minimalmovieposters.tumblr.com/post/16082323317/pulp-ction-by-ana-balderramas Slide by Lillian Lee Wednesday, March 5, 14
  • Relative status: who has the lead? Communicative behaviors are patterned and coordinated, like a dance[Niederhoffer and Pennebaker, 02] http://minimalmovieposters.tumblr.com/post/16082323317/pulp-ction-by-ana-balderramas adah ja ad to the adajkj the adah ja ad at a adajkj the adah ja ad of adajkj the adah ja ad of adajkj the adah toja ad an adajkj gh adah ja ad the adajkj forhgh Slide by Lillian Lee Wednesday, March 5, 14
  • Relative status: who has the lead? Communicative behaviors are patterned and coordinated, like a dance[Niederhoffer and Pennebaker, 02] http://minimalmovieposters.tumblr.com/post/16082323317/pulp-ction-by-ana-balderramas adah ja ad to the adajkj the adah ja ad at a adajkj the adah ja ad of adajkj the adah ja ad of adajkj the adah toja ad an adajkj gh adah ja ad the adajkj forhgh Those with less power tend to immediately match the function-word choices of those with more power. [C. Danescu-Niculescu-Mizil et al.WWW 2012] Slide by Lillian Lee Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Content words still matter: detecting dodging by debaters 149 Slide by Philip Resnik Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Demographics Predict categories such as age, race, and gender based on what a person writes. Mixes both content and stylistics features E.g. men and women tend to talk about different topics and they also use function words at different rates. (See James Pennebakers work) Usually also relies on other inputs, such as a persons name, when they write (e.g. tweet times), and their social network. 150 http://www.tweetolife.com/gender/ [no longer working] Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Example: People Pattern audience demographics (3.8k accounts) 151 Wednesday, March 5, 14
  • Beyond text Why NLP is hard Sentiment analysis overview Document classication Aspect-based sentiment analysis Visualization Semi-supervised learning Stylistics & author modeling Wrap up Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 What does barbecue mean? 153 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 What does barbecue mean? Barbecue 153 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 What does barbecue mean? Barbecue 153 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 What does barbecue mean? Barbecue 153 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 What does barbecue mean? Barbecue 153 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 What does barbecue mean? Barbecue 153 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 What does barbecue mean? Barbecue 153 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 What I thought semantics was before 2005 154 From: John Enrico and Jason Baldridge. 2011. Possessor Raising, Demonstrative Raising, Quantier Float and Number Float in Haida.International Journal of American Linguistics.77(2):185-218 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Updated perspective a la Ray Mooney (UT Austin CS) 155 http://www.cs.utexas.edu/users/ml/slides/chen-icml08.ppt Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 http://www.lib.utexas.edu/books/travel/index.htmlTravel at the Turn of the 20th Century 156 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Motivation: Google Lit Trips [http://www.googlelittrips.com/] 157 Grapes of Wrath in Google Earth Text http://www.googlelittrips.com/GoogleLit/9-12/Entries/2006/11/1_The_Grapes_of_Wrath_by_John_Steinbeck.html Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Motivation: Google Lit Trips [http://www.googlelittrips.com/] 157 Grapes of Wrath in Google Earth Text http://www.googlelittrips.com/GoogleLit/9-12/Entries/2006/11/1_The_Grapes_of_Wrath_by_John_Steinbeck.html Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Crisis response: Haiti earthquake 158 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Crisis response: Haiti earthquake 158 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Look, Mom, no hands! (Err, um... no metadata.) 159 Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Look, Mom, no hands! (Err, um... no metadata.) 159 Topics with a clear, circumscribed geographic focus emerge! Wednesday, March 5, 14
  • 2013 Jason M Baldridge Sentiment Analysis Symposium, March 2014 But, of course, certain kinds of metadata are now plentiful. 160 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Geotagged Wikipedia 161 3017N 9744W Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 01:55:55 RT @USER_dc5e5498: Drop and give me 50.... 05:09:29 I said u got a swisher from redmond!? He said nah kirkland! Lol..ooooooooOkay! 05:57:35 Lmao!:) havin a good ol time after work! Unexpected! #goodtimes 06:00:09 RT @USER_d5d93fec: #letsbereal .. No seriously, #letsbereal>>lol. Don't start. 06:00:37 On my way to get @USER_60939380 yeee! She want some of this strawberry! Sexy! ... 473141 N 1221152 W 162 Geotagged Twitter Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 01:55:55 RT @USER_dc5e5498: Drop and give me 50.... 05:09:29 I said u got a swisher from redmond!? He said nah kirkland! Lol..ooooooooOkay! 05:57:35 Lmao!:) havin a good ol time after work! Unexpected! #goodtimes 06:00:09 RT @USER_d5d93fec: #letsbereal .. No seriously, #letsbereal>>lol. Don't start. 06:00:37 On my way to get @USER_60939380 yeee! She want some of this strawberry! Sexy! ... 473141 N 1221152 W 162 Geotagged Twitter Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Document geolocation: where is this person? 163 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Document geolocation Language-model-based Information Retrieval (LMIR) [Ponte and Croft 1998, Zhai and Lafferty 2001] Given texts annotated with locations, construct language models for points or regions on the earths surface. (There are many choices for doing this.) Given a new (unlabeled) text, rank all locations w.r.t. how good a match they are. (There are many choices for doing this.) Give the ranking of locations, choose a single coordinate as the location for the unlabeled text. (There are many choices for doing this.) 164 [Serdyukov, Murdock, & van Zwol 2009; Cheng, Caverlee, & Lee 2010;Wing & Baldridge 2011] Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 165 Amsterdam, Zaandam,Amstelveen, Diemen, Landsmeer ... Frankfurt, Frechen, Hrth, Brhl,Wesseling, ... Construct pseudo-documents from a geodesic grid Grid: equal degree/area cells; rectangular or otherwise? Pseudoc location is cell center or centroid of all documents in cell. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 165 Amsterdam, Zaandam,Amstelveen, Diemen, Landsmeer ... Frankfurt, Frechen, Hrth, Brhl,Wesseling, ... Construct pseudo-documents from a geodesic grid Grid: equal degree/area cells; rectangular or otherwise? Pseudoc location is cell center or centroid of all documents in cell. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 166 Amsterdam, Zaandam,Amstelveen, Diemen, Landsmeer ... Frankfurt, Frechen, Hrth, Brhl,Wesseling, ... Generate a language model for each pseudo-document Interpolate or smooth against the LM for the entire collection, neighboring pseudo-documents, or both. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 167 Generate a language model for a query document May optionally be smoothed, depending on the ranking method. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 longitude 15 16 17 18 19 20 latitude 50 51 52 53 54 55 log(rank) 10 8 6 4 2 0 168 Rank all pseudo-documents w.r.t. query text Kullback-Leibler divergence Query Likelihood Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 longitude 15 16 17 18 19 20 latitude 50 51 52 53 54 55 log(rank) 10 8 6 4 2 0 169 Choose the best matching pseudo-document Highest ranking document Mean shift over top-K Reranking (Learning to Rank) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Locations of Twitter users are not uniformly distributed! 170 (Small) GeoUT (Twitter) plotted on Google Earth, one pin per user. Density of (all) documents in GeoUT over the USA (390 million tweets) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 k-d tree for geotagged Wikipedia, looking at N. America 171 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 k-d tree for geotagged Wikipedia, looking at N. America 171 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Pre-grid clustering [Erik Skiles, MA thesis, UT Austin, Ling] 172 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Four clusters on GeoUT (390 million tweets) 173 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Four clusters on GeoUT (390 million tweets) 173 West coast East coast Midwest & South Spanish language All tweets Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Performance (kd-tree with clustering) 174 Wikipedia (entire world) Half of documents geotagged within 12 km of truth Percent of documents within 166km (100 miles): 91% Twitter (USA) Half of users geotagged within 330 km of truth Percent of documents within 166km (100 miles): 40% For better or worse, it soon might not matter whether you have location turned on or not... what you say is where you are / are from. (Also, other factors, e.g. who you are linked to, of course.) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Geographic meaning of words (based on Wikipedia) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Geographic meaning of words (based on Wikipedia) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Geographic meaning of words (based on Wikipedia) beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Geographic meaning of words (based on Wikipedia) beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Geographic meaning of words (based on Wikipedia) mountainbeach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Geographic meaning of words (based on Wikipedia) mountainbeach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Geographic meaning of words (based on Wikipedia) mountainbeach wine Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Geographic meaning of words (based on Wikipedia) mountainbeach wine Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Geographic meaning of words (based on Wikipedia) mountainbeach wine barbecue Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Image geo-location: http://graphics.cs.cmu.edu/projects/im2gps/ Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Entity linking It is one thing to identify that a span of text is a named entity, and another thing entirely to pick the unique entity in the world that it refers to. 177 John Smith went to London to visit Apple last year. Which John Smith? London in UK, Canada, or elsewhere? Apple Inc. or Apple Records? 2013? 2012? 1999? 1850? Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Toponym (place name) resolution: geographic entity linking 178 They visit Portland every year. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Toponym (place name) resolution: geographic entity linking 178 They visit Portland every year. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Which Portland? (Also: Canada,Australia, Ireland...) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Toponym resolution in context 179 Although Elisha Newman made the rst land entry in the township of Portland (June, 1833), he did not become a settler until three years later, by which time a few settlers had located in the town. From Mr. Newman's story, it appears that early in 1833, he was visiting friends in Ann Arbor, and during an evening conversation discussed with others the subject of unlocated lands lying west of Ann Arbor. One of the company (Joseph Wood) remarked that he had been out with the party sent to survey Ionia and other counties, and that the surveyors were struck by the valuable water-power at the mouth of the Looking Glass River, saying there would surely be a village there some day. Mr. Newman was at once taken with the idea of locating lands at the mouth of the Looking Glass. Following up his impulse, he made ready to start at once, and, accompanied by James Newman and Joseph Wood, went out to the Looking Glass on a tour of inspection. Being satised with the location, he returned Eastward with his companions, and at White Pigeon made his land entry. Newman did not return for a permanent settlement until the spring of 1836, and meanwhile, in November, 1833, Philo Bogue bought a piece of land on section 28, in the bend of the Grand River, where he proposed to set up a trading post. Unaided he rolled up a log cabin near where the Detroit, Lansing, and Northern depot was located, and when he brought the house into decent shape went over to Hunt's at Lyons for his family, whom he had left there against such time as he should have affairs prepared for their comfort. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Spatial minimality 180 Although Elisha Newman made the rst land entry in the township of Portland (June, 1833), he did not become a settler until three years later, by which time a few settlers had located in the town. From Mr. Newman's story, it appears that early in 1833, he was visiting friends in Ann Arbor, and during an evening conversation discussed with others the subject of unlocated lands lying west of Ann Arbor. One of the company (Joseph Wood) remarked that he had been out with the party sent to survey Ionia and other counties, and that the surveyors were struck by the valuable water-power at the mouth of the Looking Glass River, saying there would surely be a village there some day. Mr. Newman was at once taken with the idea of locating lands at the mouth of the Looking Glass. Following up his impulse, he made ready to start at once, and, accompanied by James Newman and Joseph Wood, went out to the Looking Glass on a tour of inspection. Being satised with the location, he returned Eastward with his companions, and at White Pigeon made his land entry. Newman did not return for a permanent settlement until the spring of 1836, and meanwhile, in November, 1833, Philo Bogue bought a piece of land on section 28, in the bend of the Grand River, where he proposed to set up a trading post. Unaided he rolled up a log cabin near where the Detroit, Lansing, and Northern depot was located, and when he brought the house into decent shape went over to Hunt's at Lyons for his family, whom he had left there against such time as he should have affairs prepared for their comfort. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 GeoNames 4048392 Portland Mills Portland Mills 39.7781 -87.00918 P PPL US IN 133 0 223 218 America/Indiana/Indianapolis 2010-02-15 4084605 Portland Portland 32.15459 -87.1686 P PPL US AL 047 0 30 41 America/Chicago 2006-01-15 4127143 Portland Portland Portlend,33.2379 -91.51151 P PPL US AR 003 430 38 39 America/Chicago 2011-05-14 4169227 Portland Portland 30.51242 -86.19578 P PPL US FL 131 0 8 14 America/Chicago 2006-01-15 4217115 Portland Portland 34.05732 -85.03634 P PPL US GA 233 0 229 228 America/New_York 2010-09-05 4277586 Portland Portland 37.0778 -97.31227 P PPL US KS 191 0 362 364 America/Chicago 2006-01-15 4305000 Portland Portland 37.12062 -85.44608 P PPL US KY 001 0 220 223 America/Chicago 2006-01-15 4305001 Portland Portland 38.26924 -85.8108 P PPL US KY 111 0 135 138 America/Kentucky/Louisville 2006-01-15 4305002 Portland Portland 38.74812 -84.44772 P PPL US KY 191 0 265 266 America/New_York 2006-01-15 404289 Portland Portland Portlend,38.71088 -91.71767 P PPL US MO 027 0 170 172 America/Chicago 2010-01-29 4521811 Portland Portland Portlend,39.00341 -81.77124 P PPL US OH 105 0 187 188 America/New_York 2010-01-29 4650946 Portland Portland Portlend,36.58171 -86.51638 P PPL US TN 165 11480 244 245 America/Chicago 2011-05-14 4720131 Portland Portland Portlend,27.87725 -97.32388 P PPL US TX 409 15099 13 11 America/Chicago 2011-05-14 4841001 Portland Portland Portlend,41.57288 -72.64065 P PPL US CT 007 5862 24 27 America/New_York 2011-05-14 4871855 Portland Portland 43.12858 -93.12354 P PPL US IA 033 35 327 330 America/Chicago 2011-05-14 4906524 Portland Portland 41.66253 -89.98012 P PPL US IL 195 0 190 190 America/Chicago 2006-01-15 5006314 Portland Portland Portlend,42.8692 -84.90305 P PPL US MI 067 3883 221 223 America/Detroit 2011-05-14 5746545 Portland Portland 45.52345 -122.67621 P PPLA2 US OR 051 583776 12 15 America/Los_Angeles 2011-05-14 Spatial minimality 180 Although Elisha Newman made the rst land entry in the township of Portland (June, 1833), he did not become a settler until three years later, by which time a few settlers had located in the town. From Mr. Newman's story, it appears that early in 1833, he was visiting friends in Ann Arbor, and during an evening conversation discussed with others the subject of unlocated lands lying west of Ann Arbor. One of the company (Joseph Wood) remarked that he had been out with the party sent to survey Ionia and other counties, and that the surveyors were struck by the valuable water-power at the mouth of the Looking Glass River, saying there would surely be a village there some day. Mr. Newman was at once taken with the idea of locating lands at the mouth of the Looking Glass. Following up his impulse, he made ready to start at once, and, accompanied by James Newman and Joseph Wood, went out to the Looking Glass on a tour of inspection. Being satised with the location, he returned Eastward with his companions, and at White Pigeon made his land entry. Newman did not return for a permanent settlement until the spring of 1836, and meanwhile, in November, 1833, Philo Bogue bought a piece of land on section 28, in the bend of the Grand River, where he proposed to set up a trading post. Unaided he rolled up a log cabin near where the Detroit, Lansing, and Northern depot was located, and when he brought the house into decent shape went over to Hunt's at Lyons for his family, whom he had left there against such time as he should have affairs prepared for their comfort. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 GeoNames 4048392 Portland Mills Portland Mills 39.7781 -87.00918 P PPL US IN 133 0 223 218 America/Indiana/Indianapolis 2010-02-15 4084605 Portland Portland 32.15459 -87.1686 P PPL US AL 047 0 30 41 America/Chicago 2006-01-15 4127143 Portland Portland Portlend,33.2379 -91.51151 P PPL US AR 003 430 38 39 America/Chicago 2011-05-14 4169227 Portland Portland 30.51242 -86.19578 P PPL US FL 131 0 8 14 America/Chicago 2006-01-15 4217115 Portland Portland 34.05732 -85.03634 P PPL US GA 233 0 229 228 America/New_York 2010-09-05 4277586 Portland Portland 37.0778 -97.31227 P PPL US KS 191 0 362 364 America/Chicago 2006-01-15 4305000 Portland Portland 37.12062 -85.44608 P PPL US KY 001 0 220 223 America/Chicago 2006-01-15 4305001 Portland Portland 38.26924 -85.8108 P PPL US KY 111 0 135 138 America/Kentucky/Louisville 2006-01-15 4305002 Portland Portland 38.74812 -84.44772 P PPL US KY 191 0 265 266 America/New_York 2006-01-15 404289 Portland Portland Portlend,38.71088 -91.71767 P PPL US MO 027 0 170 172 America/Chicago 2010-01-29 4521811 Portland Portland Portlend,39.00341 -81.77124 P PPL US OH 105 0 187 188 America/New_York 2010-01-29 4650946 Portland Portland Portlend,36.58171 -86.51638 P PPL US TN 165 11480 244 245 America/Chicago 2011-05-14 4720131 Portland Portland Portlend,27.87725 -97.32388 P PPL US TX 409 15099 13 11 America/Chicago 2011-05-14 4841001 Portland Portland Portlend,41.57288 -72.64065 P PPL US CT 007 5862 24 27 America/New_York 2011-05-14 4871855 Portland Portland 43.12858 -93.12354 P PPL US IA 033 35 327 330 America/Chicago 2011-05-14 4906524 Portland Portland 41.66253 -89.98012 P PPL US IL 195 0 190 190 America/Chicago 2006-01-15 5006314 Portland Portland Portlend,42.8692 -84.90305 P PPL US MI 067 3883 221 223 America/Detroit 2011-05-14 5746545 Portland Portland 45.52345 -122.67621 P PPLA2 US OR 051 583776 12 15 America/Los_Angeles 2011-05-14 Spatial minimality 180 Ann Arbor Detroit Ionia Lyons Portland White Pigeon 1 >7 >4 >15 >17 1 # LocationsToponym Although Elisha Newman made the rst land entry in the township of Portland (June, 1833), he did not become a settler until three years later, by which time a few settlers had located in the town. From Mr. Newman's story, it appears that early in 1833, he was visiting friends in Ann Arbor, and during an evening conversation discussed with others the subject of unlocated lands lying west of Ann Arbor. One of the company (Joseph Wood) remarked that he had been out with the party sent to survey Ionia and other counties, and that the surveyors were struck by the valuable water-power at the mouth of the Looking Glass River, saying there would surely be a village there some day. Mr. Newman was at once taken with the idea of locating lands at the mouth of the Looking Glass. Following up his impulse, he made ready to start at once, and, accompanied by James Newman and Joseph Wood, went out to the Looking Glass on a tour of inspection. Being satised with the location, he returned Eastward with his companions, and at White Pigeon made his land entry. Newman did not return for a permanent settlement until the spring of 1836, and meanwhile, in November, 1833, Philo Bogue bought a piece of land on section 28, in the bend of the Grand River, where he proposed to set up a trading post. Unaided he rolled up a log cabin near where the Detroit, Lansing, and Northern depot was located, and when he brought the house into decent shape went over to Hunt's at Lyons for his family, whom he had left there against such time as he should have affairs prepared for their comfort. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 GeoNames 4048392 Portland Mills Portland Mills 39.7781 -87.00918 P PPL US IN 133 0 223 218 America/Indiana/Indianapolis 2010-02-15 4084605 Portland Portland 32.15459 -87.1686 P PPL US AL 047 0 30 41 America/Chicago 2006-01-15 4127143 Portland Portland Portlend,33.2379 -91.51151 P PPL US AR 003 430 38 39 America/Chicago 2011-05-14 4169227 Portland Portland 30.51242 -86.19578 P PPL US FL 131 0 8 14 America/Chicago 2006-01-15 4217115 Portland Portland 34.05732 -85.03634 P PPL US GA 233 0 229 228 America/New_York 2010-09-05 4277586 Portland Portland 37.0778 -97.31227 P PPL US KS 191 0 362 364 America/Chicago 2006-01-15 4305000 Portland Portland 37.12062 -85.44608 P PPL US KY 001 0 220 223 America/Chicago 2006-01-15 4305001 Portland Portland 38.26924 -85.8108 P PPL US KY 111 0 135 138 America/Kentucky/Louisville 2006-01-15 4305002 Portland Portland 38.74812 -84.44772 P PPL US KY 191 0 265 266 America/New_York 2006-01-15 404289 Portland Portland Portlend,38.71088 -91.71767 P PPL US MO 027 0 170 172 America/Chicago 2010-01-29 4521811 Portland Portland Portlend,39.00341 -81.77124 P PPL US OH 105 0 187 188 America/New_York 2010-01-29 4650946 Portland Portland Portlend,36.58171 -86.51638 P PPL US TN 165 11480 244 245 America/Chicago 2011-05-14 4720131 Portland Portland Portlend,27.87725 -97.32388 P PPL US TX 409 15099 13 11 America/Chicago 2011-05-14 4841001 Portland Portland Portlend,41.57288 -72.64065 P PPL US CT 007 5862 24 27 America/New_York 2011-05-14 4871855 Portland Portland 43.12858 -93.12354 P PPL US IA 033 35 327 330 America/Chicago 2011-05-14 4906524 Portland Portland 41.66253 -89.98012 P PPL US IL 195 0 190 190 America/Chicago 2006-01-15 5006314 Portland Portland Portlend,42.8692 -84.90305 P PPL US MI 067 3883 221 223 America/Detroit 2011-05-14 5746545 Portland Portland 45.52345 -122.67621 P PPLA2 US OR 051 583776 12 15 America/Los_Angeles 2011-05-14 Spatial minimality 180 Portland Lyons Ionia White Pigeon Ann Arbor Detroit Ionia Lyons Portland White Pigeon 1 >7 >4 >15 >17 1 # LocationsToponym Although Elisha Newman made the rst land entry in the township of Portland (June, 1833), he did not become a settler until three years later, by which time a few settlers had located in the town. From Mr. Newman's story, it appears that early in 1833, he was visiting friends in Ann Arbor, and during an evening conversation discussed with others the subject of unlocated lands lying west of Ann Arbor. One of the company (Joseph Wood) remarked that he had been out with the party sent to survey Ionia and other counties, and that the surveyors were struck by the valuable water-power at the mouth of the Looking Glass River, saying there would surely be a village there some day. Mr. Newman was at once taken with the idea of locating lands at the mouth of the Looking Glass. Following up his impulse, he made ready to start at once, and, accompanied by James Newman and Joseph Wood, went out to the Looking Glass on a tour of inspection. Being satised with the location, he returned Eastward with his companions, and at White Pigeon made his land entry. Newman did not return for a permanent settlement until the spring of 1836, and meanwhile, in November, 1833, Philo Bogue bought a piece of land on section 28, in the bend of the Grand River, where he proposed to set up a trading post. Unaided he rolled up a log cabin near where the Detroit, Lansing, and Northern depot was located, and when he brought the house into decent shape went over to Hunt's at Lyons for his family, whom he had left there against such time as he should have affairs prepared for their comfort. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Spatial minimality often fails 181 I moved from Encinitas, CA, a nice beach town in North San Diego County to Asheville, NC. By far, Ashville is more hip, especially West Asheville. Asheville has a lot in common with Portland. Austin, I've never been to so I cannot comment. But what makes a place cool and hip, in my opinion are that give a area "punch". There are a lot of ingredients. One is geography. Add a college or university (and all that they bring- and draw), good restaurants, a good music scene, a progressive attitude and tolerance. Hmmm. I'm sure there are many more to ponder. But that's my start. Oh, lots of bars! From: http://www.city-data.com/forum/austin/1694181-what-makes-city-like-austin-portland-3.html City-data.com incorrectly marks West and Portland as the cities in Texas -- presumably because of their textual and spatial proximity to Austin. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Spatial minimality often fails 181 I moved from Encinitas, CA, a nice beach town in North San Diego County to Asheville, NC. By far, Ashville is more hip, especially West Asheville. Asheville has a lot in common with Portland. Austin, I've never been to so I cannot comment. But what makes a place cool and hip, in my opinion are that give a area "punch". There are a lot of ingredients. One is geography. Add a college or university (and all that they bring- and draw), good restaurants, a good music scene, a progressive attitude and tolerance. Hmmm. I'm sure there are many more to ponder. But that's my start. Oh, lots of bars! From: http://www.city-data.com/forum/austin/1694181-what-makes-city-like-austin-portland-3.html City-data.com incorrectly marks West and Portland as the cities in Texas -- presumably because of their textual and spatial proximity to Austin. But: it is clear from the text that Portland, Oregon and Austin,Texas are the referents, though their states are never mentioned and are far from the other locations! I moved from Encinitas, CA, a nice beach town in North San Diego County to Asheville, NC. By far, Ashville is more hip, especially West Asheville. Asheville has a lot in common with Portland. Austin, I've never been to so I cannot comment. But what makes a place cool and hip, in my opinion are that give a area "punch". There are a lot of ingredients. One is geography. Add a college or university (and all that they bring- and draw), good restaurants, a good music scene, a progressive attitude and tolerance. Hmmm. I'm sure there are many more to ponder. But that's my start. Oh, lots of bars! Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Toponym classiers 182 Strategy: build a textual classier per toponym by obtaining indirectly labeled examples from Wikipedia. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Toponym classiers 182 Strategy: build a textual classier per toponym by obtaining indirectly labeled examples from Wikipedia. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Toponym classiers 182 Strategy: build a textual classier per toponym by obtaining indirectly labeled examples from Wikipedia. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Toponym classiers 182 Strategy: build a textual classier per toponym by obtaining indirectly labeled examples from Wikipedia. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Toponym classiers 182 Strategy: build a textual classier per toponym by obtaining indirectly labeled examples from Wikipedia. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Toponym classiers 182 Strategy: build a textual classier per toponym by obtaining indirectly labeled examples from Wikipedia. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Toponym classiers 182 Strategy: build a textual classier per toponym by obtaining indirectly labeled examples from Wikipedia. P(Portland-OR|music) > P(Portland-ME|music) P(Portland-OR|wharf ) < P(Portland-ME|wharf ) Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Results: disambiguating toponyms 183 Average error distance Accuracy Average error distance Accuracy Population SPIDER (spatial minimality) WISTR (Wiki supervised) SPIDER +WISTR 216 81.0 1749 59.7 2180 30.9 266 57.5 279 82.3 855 69.1 430 81.8 201 85.9 TR-CoNLL Reuters News Texts August 1996 Perseus Civil War Corpus Books Late 19th Century Take-home message: text classiers are very effective & can be boosted by spatial minimality algorithms. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Back to grounding 184 Grounding often involves connecting text to knowledge sources and other modalities (image, video) & bootstrapping. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Back to grounding 184 Grounding often involves connecting text to knowledge sources and other modalities (image, video) & bootstrapping. Semi-supervised learning methods such as label propagation help bridge the annotation gap. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Back to grounding 184 Grounding often involves connecting text to knowledge sources and other modalities (image, video) & bootstrapping. Semi-supervised learning methods such as label propagation help bridge the annotation gap. Also, they can help us create models for deeper aspects of language, such as syntactic structure and logical form. Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporality of words, by hour http://www.tweetolife.com/hour/ (no longer working) 185 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporality of words, by hour http://www.tweetolife.com/hour/ (no longer working) 185 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporality of expressions, by day: http://www.google.com/trends 186 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporality of expressions, by day: http://www.google.com/trends 186 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporality of expressions, by year: http://ngrams.googlelabs.com/ 187 slave trenches aircraft war Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporal resolution [Kumar, Lease, and Baldridge 2011] 188 2000BC 0AD 2000AD 4000BC Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporal resolution [Kumar, Lease, and Baldridge 2011] 188 2000BC 0AD 2000AD 4000BC Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporal resolution [Kumar, Lease, and Baldridge 2011] 188 2000BC 0AD 2000AD 4000BC Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporal resolution [Kumar, Lease, and Baldridge 2011] 188 2000BC 0AD 2000AD 4000BC Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporal resolution [Kumar, Lease, and Baldridge 2011] 188 2000BC 0AD 2000AD 4000BC Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Temporal resolution [Kumar, Lease, and Baldridge 2011] 188 2000BC 0AD 2000AD 4000BC Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Lexical brain decoding [Yarkoni, Poldrack, Nichols, Van Essen & Wager (2011)] 189 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Lexical brain decoding [Yarkoni, Poldrack, Nichols, Van Essen & Wager (2011)] 189 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 More modalities: videos [Motwani & Mooney, 2012] 190 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Beyond word co-occurences for vector-space models 191 bear boat car cow hadoop snow water wrench 3 234 42 4 1 2 325 0 beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Beyond word co-occurences for vector-space models 191 bear boat car cow hadoop snow water wrench 3 234 42 4 1 2 325 0 beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Beyond word co-occurences for vector-space models 191 bear boat car cow hadoop snow water wrench 3 234 42 4 1 2 325 0 beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Beyond word co-occurences for vector-space models 191 bear boat car cow hadoop snow water wrench 3 234 42 4 1 2 325 0 beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Beyond word co-occurences for vector-space models 191 bear boat car cow hadoop snow water wrench 3 234 42 4 1 2 325 0 beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Beyond word co-occurences for vector-space models 191 bear boat car cow hadoop snow water wrench 3 234 42 4 1 2 325 0 beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Beyond word co-occurences for vector-space models 191 bear boat car cow hadoop snow water wrench 3 234 42 4 1 2 325 0 beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Beyond word co-occurences for vector-space models 191 bear boat car cow hadoop snow water wrench 3 234 42 4 1 2 325 0 beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Beyond word co-occurences for vector-space models 191 bear boat car cow hadoop snow water wrench 3 234 42 4 1 2 325 0 beach Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Combining distributional models with logics 192 Erk (2013):Towards a semantics for distributional representations. Garrette et al (2012):A formal approach to linking logical form and vector-space lexical semantics Beltagy et al (2013):Montague Meets Markov: Deep Semantics with Probabilistic Logical Form Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Multi-component structured vector-space models 193 beachchildren visit the children visit the beach Agent Patient Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Language learning in context [Kim & Mooney, 2013] 194 Wednesday, March 5, 14
  • 2014 Jason M Baldridge Sentiment Analysis Symposium, March 2014 Language learning in context [Kim & Mooney, 2013] 194 Wednesday, March 5, 14
  • Wrap up Why NLP is hard Sentiment analysis overview Document classication Aspect-based sentiment analysis Visualization Semi-supervised learning Stylistics & author modeling Beyond text Wednesday, March 5, 14
  • All your meaning are belong to us Wednesday, March 5, 14
  • All your meaning are belong to us Wednesday, March 5, 14
  • All your meaning are belong to us Wednesday, March 5, 14
  • ALPAC yer bags Wednesday, March 5, 14
  • Its cold out there, after the hype Wednesday, March 5, 14
  • The Hype Cycle (Gartner) Wednesday, March 5, 14
  • The Hype Cycle (Gartner, 2013) http://www.gartner.com/newsroom/id/2575515 Wednesday, March 5, 14
  • The Hype Cycle (Gartner, 2013) http://www.gartner.com/newsroom/id/2575515 Wednesday, March 5, 14
  • http://davidrothman.net/2009/09/02/all-your-healthbase-are-belong-to-us-want-em-back/ Grounding matters Wednesday, March 5, 14
  • Being right can be consequential too Wednesday, March 5, 14
  • Being right can be consequential too Wednesday, March 5, 14
  • Being right can be consequential too Wednesday, March 5, 14
  • Being right can be consequential too Wednesday, March 5, 14
  • Being right can be consequential too Wednesday, March 5, 14
  • Being right can be consequential too Wednesday, March 5, 14
  • Being right can be consequential too Narayanan et al 2012:On the Feasibility of Internet-Scale Author Identication Wednesday, March 5, 14
  • Being right can be consequential too Narayanan et al 2012:On the Feasibility of Internet-Scale Author Identication Wednesday, March 5, 14
  • Tremendous challenges and tremendous opportunities It is possible to create automated methods for opinion mining tasks that make reasonably accurate predictions that are useful for summarizing attitudes and opinions at a glance. The deeper you go, the better you can discriminate detailed opinions, but also the harder it is to have the necessary data to build accurate models. (The problem is NLP-complete.) Semi-supervised methods provide a lot of promise for these endeavors. Methods that reduce the burden of feature engineering, such as deep learning, are also important. A big component of this is grounding texts in the real world. Wednesday, March 5, 14

Recommended

View more >