Please enable JavaScript.

Coggle requires JavaScript to display documents.

14 Linear Least Squares (The Matrix Approach to Linear Least Squares…

- - - - ex. Arrhenius-ekvationen
- - - - \[y_{i}=\beta_{0}+\beta_{1}x_{i}+e_{i}\]
      - (The simplest model: y is a linear function of x plus random noise)
      - \(e_i\) independent with
        \(E(e_{i})=0\qquad\textrm{Var}(e_{i})=\sigma^{2}\)
      - :link::scroll:\[E(\hat{\beta}_{j})=\beta_{j},\quad j=0,1\]
      - \(x_i\) antas vara fixed
      - :link::scroll:\[\textrm{Var }\hat{\beta}_{0}=\frac{\sigma^{2}\sum_{i=1}^{n}x_{i}^{2}}{n\sum_{i=1}^{n}x_{i}^{2}-\left(\sum_{i=1}^{n}x_{i}\right)^{2}}\]\[\textrm{Var }\hat{\beta}_{1}=\frac{n\sigma^{2}}{n\sum_{i=1}^{n}x_{i}^{2}-\left(\sum_{i=1}^{n}x_{i}\right)^{2}}\]\[\textrm{Cov}(\hat{\beta}_{0},\hat{\beta}_{1})=\frac{-\sigma^{2}\sum_{i=1}^{n}x_{i}}{n\sum_{i=1}^{n}x_{i}^{2}-\left(\sum_{i=1}^{n}x_{i}\right)^{2}}\]
      - Vi ser att varianserna beror på \(x_i\) (som ju är kända) och \(\sigma^2\) (som vi alltså då behöver estimera för att kunna uppskatta varianserna)
        
        :pushpin: residual sum of squares (RSS)\[RSS=\sum_{i=1}^{n}(y_{i}-\hat{\beta}_{0}-\hat{\beta}_{1}x_{i})^{2}\]
        
        Kommer visa sen att\[s^{2}=\frac{RSS}{n-2}\] är unbiased estimat av \(\sigma^2\) #
        
        När är \(\hat{\beta}_0\) och \(\hat{\beta}_1\) normalfördelade?
        
        Om \(e_i\) är normala
        
        eller approximativt för stora \(n\): Om \(e_i\) är oberoende
        
        \[\frac{\hat{\beta}_{i}-\beta_{i}}{s_{\hat{\beta}_{i}}}\sim t_{n-2}\]
  - - - \[s_{yy}=\frac{1}{n}\sum_{i=1}^n(y_{i}-\overline{y})^{2}\]
      - \[s_{xx}=\frac{1}{n}\sum_{i=1}^{n}(x_{i}-\overline{x})^{2}\]
      - \[s_{xy}=\frac{1}{n}\sum_{i=1}^{n}(x_{i}-\overline{x})(y_{i}-\overline{y})\]
      - the correlation coefficient
        between the x's and y's
        
        \[r=\frac{s_{xy}}{\sqrt{s_{xx}s_{yy}}}\]
      - The slope of the least squares line is\[\hat{\beta}_{1}=\frac{s_{xy}}{s_{xx}}\]
        
        and therefore\[r=\hat{\beta}_{1}\sqrt{\frac{s_{xx}}{s_{yy}}}\]
        
        In particular, the correlation is zero if and only if the slope is zero
        
        Insättning i # ger\[\frac{\hat{y}-\overline{y}}{\sqrt{s_{yy}}}=r\frac{x-\overline{x}}{\sqrt{s_{xx}}}\]
        
        Interpretation: Suppose that \(r>0\) and that \(x\), the predictor variable, is one standard deviation greater than its average; then the predicted value of \(y\) is \(r\) standard deviations bigger than its average, \(r\leq1\). The predicted value thus deviates from its average by fewer standard deviations than does the predictor. In units of standard deviations, it is closer to its average than is the predictor.
        
        [Sir Francis Galton "regression towards mediocrity" This is exactly what the regression line predicts, as in the previous paragraph]
- - - - the elements are jointly distributed random variables with
        \(E(Y_{i})=\mu_{i}\)
        \(\textrm{Cov}(Y_{i},\,Y_{j})=\sigma_{ij}\)
      - The mean vector = the vector of means:\[E(\mathbf{Y})=\boldsymbol{\mu}_{Y}=\left[\begin{array}{c}\mu_{1}\\\mu_{2}\\\vdots\\\mu_{n}\end{array}\right]\]
      - The covariance matrix of Y, denoted \(\boldsymbol{\Sigma}\), is defined to be an \(n\times n\) matrix with the \(ij\) element \(\sigma_{ij}\), which is the covariance of \(Y_i\) and \(Y_j\). Note that \(\boldsymbol{\Sigma}\) is a symmetric matrix.
      - Suppose that Z is formed from a fixed vector c and a fixed linear transformation A of Y:
      - För A (n x n) och x (n) kallas\[\mathbf{x}^{T}\mathbf{A}\mathbf{x}=\sum_{i=1}^{n}\sum_{j=1}^{n}x_{i}a_{ij}x_{j}\]en quadratic form
      - If \(\mathbf{Y}_{\textrm{p}\times1}\) och \(\mathbf{Z}_{m\times1}\) är slumpvektorer så har vi att
        
        cross-covariance matrix of Y and Z är \(p\times m\)-matrisen \(\boldsymbol{\Sigma}_{YZ}\) med ij:te elementet \(\sigma_{ij}=\textrm{Cov}(Y_{i},\,Z_{j})\)
        
        quantifies the relationships between the elements of Y and Z
        
        The covariance of \(Y_i\) and \(Z_j\) can be converted to a correlation coefficient by dividing by the product of the standard deviations of \(Y_i\) and \(Z_j\)
    - - \[Y_{i}=\beta_{0}+\sum_{j=1}^{p-1}\beta_{j}x_{ij}+e_{i}\]
      - where \(e_i\) are random errors with
        
        \(E(e_{i})=0\)
        \(\textrm{Var}(e_{i})=\sigma^{2}\)
        \(\textrm{Cov}(e_{i},\,e_{j})=0,\quad i\neq j\)
      - In matrix notation:
        
        and
        \(E(\mathbf{e})=\boldsymbol{0}\)
        \(\boldsymbol{\Sigma}_{ee}=\boldsymbol{\sigma}^{2}\mathbf{I}\)
  - - - From #, we can compute the expected value of this quadratic form:
  - - - \(\hat{\beta}_{i}\sim N(\beta_{i},\sigma^{2}c_{ii})\)
    - - \[\frac{\hat{\beta}_{i}-\beta_{i}}{s_{\hat{\beta}_{i}}}\sim t_{n-p}\]so that to test \(H_{0}:\beta_{i}=\beta_{i0}\), we can use the test statistic\[t=\frac{\hat{\beta_{i}}-\beta_{i0}}{s_{\hat{\beta}_{i}}}\]
        
        CI for \(\beta_{i}\):\[\hat{\beta}_{i}\pm t_{n-p}(\alpha/2)s_{\hat{\beta}_{i}}\]
        
        The most commonly tested null hypothesis is \(H_{0}:\beta_{i}=0\), which states that \(x_{i}\) has no predictive value.
  - - - The \(i\)th standardized residual is\[\frac{Y_{i}-\hat{Y}_{i}}{s\sqrt{1-p_{ii}}}\]where \(p_{ii}\) is the \(i\)th diagonal element of P.
- - - - \(E(Y|\boldsymbol{\xi}=\mathbf{x})=\mathbf{x}\beta\)
      - \(\textrm{Var}(Y|\boldsymbol{\xi}=\mathbf{x})=\sigma^{2}\)
    - - ...