Please enable JavaScript.

Coggle requires JavaScript to display documents.

WebM (Text Mining (Applications (Application area (->Information…

- - - - ->Information extraction: identification of key phrases and relationships within text by looking for patterns
      - ->Topic tracking: Based on a user profile and documents, text mining can predict other documents of interest to the user
      - ->Summarization: to save time for the reader
      - ->Categorization: identifying the main themes of a document and placing it into a predefined set of categories
      - ->Clustering: group similar documents together
      - ->Concept linking: connects related documents by identifying their shared concepts
      - ->Question answering: finding best answer to a given question through knowledge-driven pattern matching
  - - - Application
      - Voice of the customer
      - ->Get data from full set of customer touch points (email survey etc.)
      - Key element of customer experience management initiates
      - Voice of the market (Understanding aggregate opinions and trends
      - Knowing what stakeholdings are saying about product and services
    - - Web content mining
        Extract useful information from web pages (Data collection done via web crawlers) (Used for competitive intelligence, information, sentiment analysis etc. (Source unstructured textual content of the web pages)
      - Web Structure Mining
        
        Develop useful information from the links included in the web documents
        
        (Source: Unified Resource Locator (URL)
        
        Hyperlinks ->Authoritative pages
        
        Hubs: List of recommended links to authoritative pages
        
        Hyperlink-induced topic search algorithm
      - Web Usage Mining (Web Analytics)
        
        Extract information from clickstream analysis of web server logs generated via web page visits and transactions
        
        (Data stored in server access logs, agent log, referrer log, client-side cookies user characteristics, page attributes, metadata etc.
- - - - Old method(Bag of words)
      - New METHOD (NLP)
      - Critical component of text mining
      - The understanding of the natural human language
      - Consider grammatical and semantic constraints as well as context
    - - Part-of-speech tagging: depends not only on definition of term but also on the context used
      - Text segmentation: analysis of free-form text found in e-mails and recorded telephone transcripts Syntactic ambiguities: grammar ambiguity
      - Text contains acronyms, abbreviations, misspellings. e.g. customer, cust, customar, csmr
      - Imperfect or irregular input e.g. foreign accents
      - Speech acts and semantic analysis: understanding the meaning of words (i.e. book = to reserve vs book = a manual)
- - - - Business Understanding Plausible goals for web-based mining: e.g. Improve usability of web site Determine product for sale at website that tend of be purchased of viewed together Personalised web page for customers
        2.Data Understanding and Preparation for web-based mining Sources (LOGS,Flat files, Operational DB ->Data Preparation -> Session File ->Data Mining algorithm ->Learner Model (LOG File consist the referring page, user's host address, date and time, request type, status of request, bytes, browser type/agent)3.Data Preparation**4. Modeling**
      - Associative rule mining Unsupervised clustering (Usage profiles for clusters, Form clusters of similar user session instances to personalise pages viewed by web site users) Session file
      - Data Preparation Extracts relevant data from web server logs and create a session file suitable for data mining 5)Evaluation (Interpret results of web-based mining session using associative rule mining and clustering)
        Associative rule generated Confidence Support Interpretation Clustering Technique Using summary statistics of website activities in interpreting clusters formed
      - 6.Deployment Possible actions to take based on results of web-based mining Targeted Marketing Communications Webpage usability optimisation Personlisation of web pages based on user profiles
    - - Issues
      - Data preparation
      - Differentiate individual user sessions (Host address not useful as multiple users may access the site from same host. Cookies is the solution)
      - Identify unwanted log file entries (generated from multiple log entries generated by users)
      - Data Transformation
      - New attributes can be added to user session records to improve prediction of outcome
- - - - 1. (Establish the Corpus)
      - ->Collect all the relevant unstructured data(Text, HTML, XML etc.)
      - -> Digitise and standardise the collection (all in ASCII text file etc.)
      - ->Place the collection in a common place (a flat file/directory)
      - (Output of this task is a collection of documents in digitised format for computer processing)
      - 2. (Create the Term-Document Matrix)
        
        (introduce structure to the corpus)
        
        (Output for this stage is a flat file called term-document matrix where the cells are populated with the term frequencies/APPROPRIATE INDICES)
        
        (E.G. Document 1 Term Investment Risk appears once, Document 2 appears twice. Populated 3 occurrence)
      - 3. Extract pattern/knowledge
        
        ->Classification (Text categorisation)
        
        ->Clustering (Natural Groupings of Text)
        
        Improves search recall and precision
        
        Association (A=>C)
        
        CONFIDENCE % of documents that include concept of C
        
        SUPPORT % of documents that include bot h A&C
        
        Trend analysis