Please enable JavaScript.

Coggle requires JavaScript to display documents.

Data Science Overview - Coggle Diagram

- - - - Language Translation
      - Text/Speech Generation
      - Summarisation
      - Relation / Named Entity Classification or Extraction
      - Chatbots
    - - Encoder
        Intuitively, takes our "understanding" and captures that in an embedding
        
        Typically an LSTM or GRU
        
        The encoder sees the original text token by token
        
        It outputs an encoded vector
      - Decoder
        Intuitively, emits a series of tokens that represents a soft-max (over the vocabulary)
        
        Like the encoder, uses an LSTM or GRU
        
        Receives:
        
        An encoded vector from the Encoder
        
        Start token
        
        Continues sequentially until it emits a stop token
  - - - Context-Free
        Words with multiple meanings get the same embedding values
        (e.g. topics and embeddings)
      - Context-Aware
        Position in the sentence and neighbouring words affect the word, enabling it to derive different meaning
        (e.g. language models)
    - - Vector
        
        Bag of Words
        
        TF-IDF
        Term Frequency-Inverse Document Frequency
        
        Topic Model
        
        Embedding
      - Common Classifications
        
        Naive Bayes
        
        Bagging / Boosting
        
        MLP
      - Sequence-based Classification
        
        LSTM / GRU
        
        CNN
- - - - AR = Auto Regressive
        Current value is a weighted sum of past values
      - I = Integrated
        Differencing
        
        To make a time series stationary, we need to change it. Customarily we use the difference transformation.
      - MA = Moving Average
        White noise or shocks to a system gradually reduced with time
    - - ARIMA(p, d, q)
        
        p = number of auto regressive terms
        
        d = number of differences
        
        q = number of moving average terms
    - - Linear models need stationarity
        
        To achieve stationarity we take a difference of the time series itself
        
        Taking differences removes the trend (I term)
        
        Sometimes we over difference and need to add back (MA term)
        
        Sometimes differencing doesn't go far enough (AR term)
      - Stationarity
        Constant mean (no trend), and constant variance
        
        Top = Stationarity
        Bottom = Non-stationarity
        
        Augmented Dickey-Fuller Test
        Tests for stationarity through the presence of a unit root
        
        A unit root is a stochastic trend (random walk with drift)
        
        If a series has a unit root, it shows a systematic, unpredictable pattern
    - - Heuristic using ACF and PACF plots
        
        Auto-Correlation
        The correlation of a time series to a delayed copy of itself, ie t-x where t is the current time and x is the lag
        
        ACF
        Auto Correlation Function. Plots the autocorrelation of a time series by lag. Values between 1.00 (perfect correlation), 0 (no correlation) and -1 (inverse correlation)
        
        PACF
        Partial Auto Correlation Function. Same as ACF but plots partial auto-correlation
        
        Partial Auto-Correlation
        The autocorrelation for an observation and an observation at a prior time step is comprised of both the direct correlation and indirect correlations. These indirect correlations are a linear function of the correlation of the observation, with observations at intervening time steps. Partial auto-correlation seeks to remove these indirect correlations
      - Grid search with loss (AIC)
      - Automated Search with Pyramid (AutoARIMA)