Please enable JavaScript.

Coggle requires JavaScript to display documents.

ML2 - Coggle Diagram

- - - - Eg. credit card frauds 10000 good and 100 bad examples
  - - - 2xy/x+y
    - - when we need more bias to precision we can use F0.5 score
      - when we need more bias to recall we can use f2 score
  - - - how much the model is better than the just guessing the average
        
        if R2 score is 1 then model is good and if 0 then it just guessing the average between points
- - - - wi → wi - a d/dwi (Error)
      - Types
        
        stockastic
        
        consider only one point for the error farther most point
        
        Batch
        
        consider all points for the error function and minimize accordingly
        
        mini batch gradient descent
        
        stockastic and batch is computationally slow hence mini batch is used
        
        divide the data into many batches having roughly same no of points and use each batch to update your weights.
    - - 3 dimensions will be fitting a plane through the points
      - for n dimensions
        
        n dimensional hyperplane
    - - based on MSE we can have 2 equations having 2 unknowns for the line
      - But for n parameter we shall have n equations and n unknowns which will take infinite time to solve.
        
        hence gradient descent
    - - when lines is not enough we use polynomial of degree n for fitting the data points
    - - We take account for the coefficient of the line for the error
      - Types
        
        L1 Regularization
        
        adds absolute values of the all the coefficients to the error
        
        L2 Regularization
        
        adds square of the coefficient of the equation to the error
        
        Lambda
        
        some time we want simpler model or complex based on the requirement then we can multiply the calculated regulation values to lambda to get the result
  - - - university acceptance
    - - boundary will be a plane
      - for n dimensions we have n-1 dimensional hyperplane
    - - building block of NN
      - Node with the modeled line for prediction
      - having bias from outside or within a node
      - step function
      - linear function
      - Perceptron learning
        
        start with random move closer to the all points
  - - - how much chance is for the particles to move ice has low entropy water vapor has high entropy
      - high knowledge is low entropy
      - Formula Entropy
        
        formula : -prob of red * log2(probab of red ball) - prob of yellow log2(probab of yellow ball)
        
        multiplication is bad to convert it into sum we use logarithm
        
        Total probability is multiplication of all 4 probabilities as they are independent events.
        
        Game of creating initial configuration from random balls in bucket
      - MultiClass Entropy
        
        For 2 probabilities
        
        entropy=−p1log2(p1)−p2log2(p2)
        
        Hence for n probabilities
        
        negative summation of probability * log2(probability)
      - Information Gain
        
        Change in Entropy
        
        we check the entropy of the parent and then check the average entropy of the children and the difference between this two will get us the information gain
    - - doesn't generalize well
    - - Decision Trees are tend to overfit the data
      - Hence we can create n different trees based on choosing random columns from the data then for prediction we can take vote from all the trees predict the output which is most occuring.
    - - maximum depth = k
        
        hence 2^k leaves in tree
      - min_sample_split
      - min_samples_leaf
  - - - prior and posterior probabilities
      - P(A|R) = P(R|A)P(A) / P(R|A)P(A) + P(R|B)P(B)
      - False Positive and calculation of being sick in 99% accurate test and 1 in 10K patient rate
    - - Applying bayes theorem for spam classification
      - calculating probabilities
    - - We make naive assumption that events which are occurring are independent from each other
      - probabilities are independent
      - P('money'| spam )
        
        probability of word money given that it is spam
    - - start with some random words as features P(spam|'cheap','money','easy'..) then flip the events P('cheap','money','easy'| spam)*P(spam) then based on naive assumption split it with product of simple factors p('cheap"|spam)P('easy'|spam) ..P(spam) that we can calculate from our data.
      - We do this for both spam and ham then we get two values that don't add up to one and then we normalize to get probability of each being spam or not spam.
  - - - Classification error + Margin Error
    - - similiar to L2 regularization
    - - it is constant which multiplied to the classification error
      - if c is larger then we have small margin but better classification
      - if c is small then we have large margin but less better classification
    - - change data dimension to higher solve a problem there then bring it back to original
    - - project the data in higher dimensions and find the separating hyperplane then project it back to original dimension
    - - based on normal distribution
      - we draw mountains over every point then separate the points on the mountains then plot back the intersection to original
      - gamma parameter
        
        decides if the mountations is narrow or wider
  - - - Boostrap Aggregating
        
        take random subsets of data and train weak learners on them then take a vote for future prediction
    - - AdaBoost
        
        Train a weak model complete data and then next time next learner will focus on mistakes made by the last one. subsequently all models will try to minimize the errors.
        
        weighting the points half 50% hence the next learner will focus on that points more.#correct 11 #incorrect 3 then assign 11/3 3.66 to the wrong onces.
        
        weight = ln(accuracy/ 1- accuracy )
        
        Simply weight = ln(no of correct points / no of incorrect points )
        
        Combining the models
        
        calculate each models weight then add then up if it is positive then red orElse blue.
        
        from sklearn.ensemble import AdaBoostClassifier
        
        model = AdaBoostClassifier(base_estimator = DecisionTreeClassifier(max_depth=2), n_estimators = 4)
- - - - Does not do well in training set
      - Error due to bias
      - Over simplify the problem
    - - Does well in the training set but it tends to memorize it instead of learning characteristics from it.
      - Error due to variance
      - Over complicate problem
- - - - Even if the data is same and no of centroids are same KMeans will not give the same clusters
      - It works as hill climbing algorithm and does rely on the fact where the initial centroids are
      - Also if 3 centroid can give 2 clusters
        
        local minima
  - - - Single link clustering and tree diagram
        
        Dendrograms
      - single link checks for smallest link between clusters
    - - DBSCAN
        
        density based special clustering of applications with noise
        
        robust against noise
        
        doesn't require no of clusters