Please enable JavaScript.

Coggle requires JavaScript to display documents.

Deep Learning (Machine Learning (Machine Learning Vs Traditional…

- - - - uses an algorithm to get the output
      - the results are given based on the algorithm that is written by the programmer
      - the algorithm will "recognizes" the what is ask
    - - uses an algorithm to get the ouput
      - uses the old data as a preference to predict the output (to get the result)
      - the algorithm will "Learn" base from the data
- - - - Neurons in an ANN is organized into layers
        
        Layers
        
        layers within an ANN (all but the input and output layers) are called Hidden Layers.
        
        Hidden Layer
        
        if an ANN has more than one hidden layer, it is said to be a deep ANN
        
        3 types of layers :
        
        input layers
        
        hidden layers
        
        output layers
        
        also known as: prediction classes
        
        Classification of Layers
        
        Dense Layer (fully connected)
        
        Convolutional Layer
        
        eg. image datas
        
        Pooling Layer
        
        Recurrent Layer
        
        eg. work with time series data
        
        Normalization Layer
        
        Layer Weights
        
        each connection between two nodes consist of weight, which is just a number.
        
        each weight represents the strength of the connection between two nodes
      - also known as:
        
        Nodes
  - - - neuron can be 0 and 1
        
        the closer to one the more activated the neuron
        
        the closer to zero the less activated the neuron is
    - - also known as: Rectified Linear Unit
      - if input < 0
        
        then the output is 0
      - if input > 0
        
        then the output is given input
  - - - Popular optimizer is SGD or Stochastic Gradient Descent
      - SGD
        
        Minimize some function that we call Loss Function, it makes it so the value will remain minimum as possible.
        
        Makes the prediction as accurate as ever
    - - is the error or the difference of what the inetwork is predicting vs the true label of the input
        
        error is what you call difference between the predicted output and the true label
        
        error = predictedOutput - trueLabel (error = .02 - 0 = 0.02)
        
        MSE (Mean Squared Error) - loss function the is commonly used
        
        if we passed multiple sample at once we would get the mean of the squared errors over all these samples
        
        mse = (output - label)^2 .... / noOfsample
        
        mse = (output - label)^2
    - - epoch
        
        refers to a single pass of the entire dataset to the network during training
      - gradient = (d(loss)/d(weight))*learningRate(0.01-0.0001 range)
        
        gradient is just a word derivative of a function of several variables
        
        gradient loss functions
        
        tell us which direction will move the loss towards the minimum
      - learning rate tells us how large is the step we should take in the direction of the minimum(loss)
        
        Learning Rate
        
        step size
        
        hyperparameter
        
        (gradient * learningRate)
        
        now we can the new weight by
        
        newWeight = oldWeight - ( gradient * learningRate );
        
        small number, usually has the range of 0.01-0.0001
      - How the weights updates?
        
        newWeight = oldWeight - (learingRate * gradient)
    - - trains faster because we got many samples
      - the larger the batch the faster the model will learn
      - a.k.a mini-batch
      - number of samples that is pass through a network at one time
      - batches in epoch = training set size / batch size
      - is another hyperparameter
    - - Transfer Learning
        
        is when we use the knowledge that was gained from one problem and apply it to the new but related problem
- - - - Overfitting
        
        when our model is predicting well on our training set data but not on data that it haven't seen.
        
        when our model isnt generalizing well
        
        when our validation data is not doing well in contrast of our training data.
        
        How to reduce Overfitting?
        
        Have more data we can train on our model.
        
        Data Augmentation
        
        cropping, rotating, flipping, zooming, color variations
        
        eg. when we got a lot of dogs facing right, we can flip it so it'll face left.
        
        when we create a new data based on our older data.
        
        Dropout
        
        When if ignores some of the of the nodes.
        
        By Reducing the complexity of our model.
        
        eg. removing some layers from the model
        
        Regularization
        
        dropout is a regularization technique
        
        L2 Regularization is the most common regularization technique
        
        L2 Regularization Term
        
        Norms are positive numbers
        
        loss+(n∑j=1∥∥w[j]∥∥2)λ/2m
        
        Loss will be summed by the sums of squared matrix norms
        
        which multiplied by regularization parameter(λ)
        
        n
        
        1 more item...
        
        w[j] layer
        
        1 more item...
        
        m
        
        1 more item...
        
        λ
        
        1 more item...
        
        Regularization is a technique that helps reduce overfitting or reduce the variance in our network by penalizing complexity
      - Underfitting
        
        When it isn't able to classify the data it was trained on or let alone the data that it wasn't train before
        
        Reducing Underfitting
        
        Increase complexity of the model
        
        eg. increase the number of layers in our model
        
        increase the number of neurons in our layers
        
        changing what type of layers were using and where
        
        Add more features to the input samples
        
        eg. addivng more variables(term)
        
        Reduce Dropout
        
        eg. we got 50% drop rate we can make it as 20% and see if the model is improving.
- - - - if given a picture of a cat or a dog, it will give an answer whether the picture has a cat or a dog
- - - - if given a picture of a cat or a dog it will not know whether the picture has a cat or a dog , instead it learns about the differences between the pictures