Please enable JavaScript.

Coggle requires JavaScript to display documents.

Statistical Natural Language Processing, Vectorization Techniques - Coggle…

- - - - Vocabulary size
      - Sparse
      - Meaning of sentence is lost. Text data is sequential & syntatic
  - - - Value in resulting vector corresponding to each word is product of the term frequency and the inverse document frequency
        
        Term Frequency
        
        Ratio of number of times term X appears in raw text and the total number of terms in raw text
        
        Inverse Document Frequency
        
        Logarithm of the ratio of total number of documents and the number of documents where the term X appears
        
        Works as a dampening factor to reduce importance of terms that are common
        
        Terms that appear only in small number of documents will get a high relevance
- - - - CBOW
        
        CBOW model is trained to predict the target word from a fixed window size of context words
        
        Sliding Window Concept
        
        Each word transformed into a one hot encoding form
        Only words that are in the window are taken into consideration
        
        Consider 3 words in a sentence for sliding window of size 3
        
        Middle word is predicted
        2 surrounding words provide context to neural network
        
        Finally after training the network by sliding the window, we have weights which we use to compute the embeddings
      - Skip-gram
        
        word to predict the target context
    - - BERT
        
        Fin BERT
        
        Bio BERT
      - CoVe