Text Analytics Past, Present & Future: An Industry View

  • Published on
    21-Apr-2017

  • View
    2.680

  • Download
    1

Transcript

Text Analytics AppliedText Analytics Past, Present & Future: An Industry View Seth GrimesAlta Plana Corporation@sethgrimesJune 5, 2014Copyright 2002 Alta Plana Corporation1Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Analytics is the systematic application of algorithmic methods that derive and deliver information, typically expressed quantitatively, whether in the form of indicators, tables, visualizations, or models.Systematic means formal & repeatable.Algorithmic contrasts with heuristic.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Text analytics past: PioneersText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Document input and processingKnowledge handling is keyDesk Set (1957): Computer engineer Richard Sumner (Spencer Tracy) and television network librarian Bunny Watson (Katherine Hepburn) and the "electronic brain" EMERAC.Hans Peter Luhn A Business Intelligence SystemIBM Journal, October 1958Copyright 2002 Alta Plana Corporation1Statistical information derived from word frequency and distribution is used by the machine to compute a relative measure of significance, first for individual words and then for sentences. Sentences scoring highest in significance are extracted and printed out to become the auto-abstract.H.P. Luhn, The Automatic Creation of Literature Abstracts, IBM Journal, 1958.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Pipelines and patternsIBMs MedTAKMI, 1997-http://www.research.ibm.com/trl/projects/textmining/index_e.htmText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Exhaustive extractionAn (old) Attensity example NLP to identify roles and relationships, for a law-enforcement application .Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Language engineeringGATE: General Architecture for Text Engineering.http://gate.ac.uk/Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Text analytics present: Business, technology, applications, and solutionsText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Organizations embracing text analytics all report having an epiphany moment when they suddenly knew more than before. -- Philip Russom, the Data Warehousing Institute, 2007http://tdwi.org/articles/2007/05/09-what-works/bi-search-and-text-analytics.aspxText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Linguistics, statistics, and semanticsText analytics (typically) involves linguistic modelling, statistical characterization, learned patterns, and semantic understanding of text-derived features Named entities: people, companies, places, etc.Pattern-based features: e-mail addresses, phone numbers, etc.Concepts: abstractions of entities.Facts and relationships.Events.Concrete and abstract attributes (e.g., expensive & comfortable) including measure-value pairs.Subjectivity in the forms of opinions, sentiments, and emotions: attitudinal data. applied to business ends.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1SourcesIts a truism that 80% of enterprise-relevant information originates in unstructured form:E-mail and messages.Web pages, online news & blogs, forum postings, and other social media.Contact-center notes and transcripts.Surveys, feedback forms, warranty claims.Scientific literature, books, legal documents....Non-text unstructured content?ImagesAudio including speechVideoValue derives from patterns.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1ValueWhat do we do with text, whether online, on-social, or in the enterprise?Post/Publish, Manage, and Archive.Index and Search.Categorize and Classify according to metadata & contents.Extract information and Analyze.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Semantics, analytics, and IRText analytics generates semantics to bridge search, BI, and applications, enabling next-generation information systems.SearchBI/Big DataApplica-tionsSearch based applications (search + text + apps)Information access (search + analytics)Synthesis (text + BI)/(big data)Text analytics (inner circle)Semantic search (search + text)NextGen CRM, EFM, MR, marketing, appsText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Content, composites, connections 1Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Content, Composites, Connections, 2Content, composites, connections 2Text Analytics: An Industry ViewJADT June 5, 2014#ApplicationsText analytics has applications in:Intelligence & law enforcement.Life sciences & clinical medicine.Media & publishing including social-media analysis and contextual advertizing.Competitive intelligence.Voice of the Customer: CRM, product management & marketing.Public administration & policy.Legal, tax & regulatory (LTR) including compliance.Recruiting.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Opinion, sentiment & emotionText Analytics: An Industry ViewJADT June 5, 2014#Sentiment analysisA specialization, of relevance to:Brand/reputation management.Customer experience management (CEM).Competitive intelligence.Survey analysis (EFM = Enterprise Feedback Management).Market research.Product design/quality.Trend spotting.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Data exploration via dashboards and workbenches.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Text analytics present: The marketText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1http://altaplana.com/TA2014Text Analytics: An Industry ViewJADT June 5, 2014#Text Analytics: An Industry ViewJADT June 5, 2014#Voice of the CustomerText analytics is applied to improve customer service and boost satisfaction and loyalty.Analyze customer interactions and opinions E-mail, contact-center notes, survey responses.Forum & blog posting and other social media. to Address customer product & service issues.Improve quality.Manage brand & reputation.Assessment of qualitative information from text helps users Gain feedback on interactions.Assess customer value.Understand root causes.Mine data for measures such as churn likelihood.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1The commercial sceneText Analytics: An Industry ViewJADT June 5, 2014#Online commerceText analytics is applied for marketing, search optimization, competitive intelligence.Analyze social media and enterprise feedback to understand the Voice of the Market: OpportunitiesThreatsTrendsCategorize product and service offerings for on-site search and faceted navigation and to enrich content delivery.Annotate pages to enhance Web-search findability, ranking.Scrape competitor sites for offers and pricing.Analyze social and news media for competitive information.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1E-Discovery and complianceText analytics is applied for compliance, fraud and risk, and e-discovery.Regulatory mandates and corporate practices dictate Monitoring corporate communicationsManaging electronic stored information for production in event of litigationSources include e-mail (!!), news, social mediaRisk avoidance and fraud detection are key to effective decision makingText analytics mines critical data from unstructured sourcesIntegrated text-transactional analytics provides rich insightsText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Text Analytics: An Industry ViewJADT June 5, 2014#Text Analytics: An Industry ViewJADT June 5, 2014#Text Analytics: An Industry ViewJADT June 5, 2014#The share rise in users who selected Arabiccoincided with much of the civil unrest in Middle Eastern countries.http://bits.blogs.nytimes.com/2014/03/09/the-languages-of-twitter-users/Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Non-English language support?Text Analytics: An Industry ViewJADT June 5, 2014#Software & platform optionsText-analytics options may be grouped in general classes.Installed text-analysis application, whether desktop or server or deployed in-database.Data mining workbench.Hosted.Programming tool.As-a-service, via an application programming interface (API).Code library or component of a business/vertical application, for instance for CRM, e-discovery, search.Text analytics is frequently embedded in search or other end-user applications.The slides that follow next will present leading options in each category except HostedText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Text Analytics: An Industry ViewJADT June 5, 2014#User decision criteriaPrimary considerations include Adaptation or specialization: To a business or cultural domain, language, information type (e.g., text, speech, images) & source (e.g., Twitter, e-mail, online news).By-user customization possibilities: For instance, via custom taxonomies, rules, lexicons.Sentiment resolution: Aggregate, message, or feature level. (What features? Topics, coreferenced entities?)What sentiment? Valence & what else? Emotion? Intent?Outputs: E.g., annotated text, models, indicators, dashboards, exploratory data interfaces.Usage mode: As-a-service (API), installed, or hosted/cloud.Capacity: Volume, performance, throughput, latency.Cost.Text Analytics: An Industry ViewJADT June 5, 2014#A few French companiesText Analytics: An Industry ViewJADT June 5, 2014#Academic spin-offsPeople PatternText Analytics: An Industry ViewJADT June 5, 2014#Text analytics future: Synthesis and sensemaking.Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1New York Times,September 8, 1957Text Analytics: An Industry ViewJADT June 5, 2014#Emotion in textText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Emotion and outcomesText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Audio including speech.Images.Video.http://www.geekosystem.com/facebook-face-recognition/http://www.sciencedirect.com/science/article/pii/S0167639312000118http://flylib.com/books/en/2.495.1.54/1/Beyond TextText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1The world of big dataMachine data (e.g., logs, sensor outputs, clickstreams).Actions, interactions, and transactions: geolocation and time.Profiles: individual, demographic & behavioral.Text, audio, images, and video.Facts and feelings.Text Analytics: An Industry ViewJADT June 5, 2014#(Accessible) data everywhereText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1http://www.geeklawblog.com/2011/12/lexis-advance-platform-launch-two.htmlA big data analytics architecture (example)Text Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1http://searchuserinterfaces.com/It is convenient to divide the entire information access process into two main components: information retrieval through searching and browsing, and analysis and synthesis of results. This broader process is often referred to in the literature as sensemaking. Sensemaking refers to an iterative process of formulating a conceptual representation from of a large volume of information. Marti Hearst, 2009SensemakingText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1http://www.businessweek.com/magazine/content/04_19/b3882029_mz072.htmEn routeText Analytics: An Industry ViewJADT June 5, 2014#Copyright 2002 Alta Plana Corporation1Text Analytics Past, Present & Future: An Industry View Seth GrimesAlta Plana Corporation@sethgrimesJune 5, 2014Copyright 2002 Alta Plana Corporation1

Recommended

View more >