Please enable JavaScript.

Coggle requires JavaScript to display documents.

Linear Regression (Problems (Collinearity (What, Multi Collinearity…

- - - - Variance Inflation
        factor
- - - - Beta 1 shows that keeping
        all the other coefficients FIXED
        the increase in Y by unit change
        in the corresponding X1. While
        simple regression simply IGNORE
        the presence of other variables.
      - If a pair or more than
        a pair of predictors
        are highly correlated
        then there is a chance of
        redundancy, we might need
        to remove a less significant
        variable. that significant
        can be chosen from the simple
        linear regression parameters.
  - - - Ho: All Betas are zero
        Ha: Atleast one in non Zero
        Check: IF F-statistic> 1 and
        corresponding F-value at
        given significance level
      - If q of predictors are not significant
        Let say we fit the line with p-q predictors
        and then find the RSS which will
        be New RSS. The updated F statistic
        for Ho=All last q predictors are
        insignificant and their coeff. is zero.
      - We can't just focus on individual
        p values of coefficients. If p is
        large than there is 5% or 2% or 1%
        chance that , that percentage of
        predictor coefficient will have p value
        higher than level of significance
  - - - Start with all p variables
        
        Remove the variables
        for which p values is max
        
        again fit with p-1 var.
        
        remove one which has
        max p-value
    - - start with null model
        
        Do single regression
        with each p predictors?
        
        Choose the one with
        lowest RSS.
        
        Select next predictor
        whom RSS gets lowest?
    - - Start with null model
        
        Add predictors basis
        forward selection.
        
        if at the time of adding
        the p-value goes above
        thresold value then
        remove that variable.
  - - - Models with high
        no. of variables
        have high RSE if
        the decrease in
        RSS is small
        relative to increase
        in p.
    - - For SLR it equals
        square of Corr(X,Y)
        while for MLR it is
        square of Corr(Y,Ypred)
      - Will always increase with
        addition of new predictor
        
        least square eqn
        allow us to fit the
        training data more
        accurately
  - - - Prediction error includes
        not only model bias
        which comes due to
        estimation of true coefficients
        but also due to irreducible error
- - - - These estimates are
        based on sample
        data and estimator of
        population regression line
  - - - 1.t-Test to check if value=0
        ......p value = value not zero
        
        Confidence interval depend
        upon standard error. SE depends
        upon number of points and spread
        of dataset.
        More spread=More leverage
  - - - Equals the variance of error
        
        Absolute measure of the lack of fit of model
        
        Drawback - Measured in units of Y and also even if
        one point is not significantly close the value gets high.
        
        High RSE = higher standard error of coefficients.
    - - Measures variability in
        Y that can be explained by X.
        
        Equals Correlation sq .
        Correlation=
        cov(x,y) / sd(x)*sd(y)
        
        TSS-RSS/TSS
        
        It only shows the linear
        relation though.
- - - - Hierarchial principle:
        Even if p value of individual
        predictor is large and with
        interaction term model is
        better, than keep the predictor.