Please enable JavaScript.

Coggle requires JavaScript to display documents.

Chapter 10: Representing and Mining Text (The Relationship of IDF to…

- - - - term frequency representation. --> use the word count
- - - - A very popular representation for text is the product of Term Frequency (TF) and Inverse Document Frequency (IDF), commonly referred to as TFIDF. The TFIDF value of a term t in a given document d is thus:
        TFIDF(t, d) = TF(t, d) × IDF(t)
- - - - As presented, the bag-of-words representation treats every individual word as a term, discarding word order entirely.A next step up in com‐ plexity is to include sequences of adjacent words as terms.
        “The quick brown fox jumps.” it would be transformed into the set of its constitutent words {quick, brown, fox, jumps}, plus the tokens quick_brown, brown_fox, and fox_jumps.
      - N-grams are useful when particular phrases are significant but their component words may not be. they require no linguistic knowledge or complex parsing algorithm.
        
        The main disadvantage of n-grams is that they greatly increase the size of the feature set.