Please enable JavaScript.

Coggle requires JavaScript to display documents.

Linear Algebra: Inner Products MathTheBeautiful (Chapter 1 (Motivation (2…

- - - - We need to define somhow the concept of length. Very practical when talking about aproximation
    - - Dot product = inner product for geometric vectors
        lengh and angles are given
      - First exapmle of use in projections
        \( P_b (a)= \frac{\langle a,b \rangle}{\langle b, b \rangle} b \)
      - \(lengh(a)=\sqrt{\langle a,a\rangle }\)
      - \( \langle a, b\rangle=lengh(a)lengh(b)\cos (a,b) \)
    - - For ortonormal basis \( {e_i} \) we have
        \( v=v^ie_i \rightarrow v^i=\langle v, e_i\rangle\)
      - \( a \perp b \leftrightarrow \langle a,b\rangle =0\)
      - Inner products comes first,
        lengh and angles comes from inner products
  - - - Geometric vectors
        \( \langle a, b\rangle=lengh(a)lengh(b)\cos (a,b) \)
      - Polynomials
        \( \langle p,q\rangle=\int_{-1}^1 p(x)q(x)dx \)
        you can thin of this def. as infinite sum in
        standart inner product
      - standart inner product in\( \mathbb{R}^n \)
        \( \langle x,y\rangle =x_1y_1+..+x_ny_n\)
  - - - \( u_i=v_i-\sum\limits_k \frac{\langle v_i, u_k\rangle}{\langle u_k ,u_k \rangle}u_k\)
      - Legendre Polynomials polynomials that are
        produced by ortogonalization procedure of the standart basie
        12.Gram-Schmidt Orthogonalization and Legendre Polynomials
        13,Decomposition with Respect to Legendre Polynomials
        14.Why {1,x,x²} Is a Terrible Basis
        
        From definition after normalization
        \( \int_{-1}^1 P_i(x)P_j(x)dx=\sigma_i^j\)
        
        Rodrigues' formula:
        \(P_n (x)=\frac{1}{2^n n!}\frac{d}{dx}[(x^2-1)^n]\)
        
        ortogonality of the basis implay better aproximations
        small error in input implay small error in outputs
        in order to measure error you need lengh that tells you
        your inner product
        
        Gaussian Quadrature :warning:
        
        17.Gaussian Quadrature 1: Summary of Legendre Polynomials
        18.Gaussian Quadrature 2: How to Determine the Weights :warning:
        20.Gaussian Quadrature Explained :warning:
        
        we need to figure out how to choose wights smart
        and how to choose smart \(x_i\)
        
        Most numerical integration procedure looks like
        \( \int_{-1}^1 f dz= w_1 f(x_1)+w_2f(x_2)+...+w_nf(x_n)\)
        
        this problem gives motivation why we are considering
        an "integral" inner product, and wy Legendre polynomials
        are mode to be ortogonal with respect to this inner product
        
        Step1. we devide 2n-1 polynomial by Lagrange polynomial
        \( \int_{-1}^1 p(x)dx=\int_{-1}^1q(x)L_n(x)dx+\int_{-1}^1r(x)dx \)
        
        Step2. We choose \(x_i\) as zeros of \(L_n \) to make first integral exacly, we choose weights \(w_i\)to make second integral exaclly
        
        First integral is equal zero becouse \(L_n \) is perpendicular
        to polynomials of degre n-1 or less (and \( deg q \leqslant n-1 \) )
    - - QR Decomposition=
        the matrix representation
        of Gram-Schmit ortogonaization :warning:
      - QR decomposition of a matrix A into a product A = QR of an orthogonal matrix Q and an upper triangular matrix R. QR decomposition is often used to solve the linear least squares problem, and is the basis for a particular eigenvalue algorithm, the QR algorithm.
- - - - Transport apear when you use inner
        product in matrix form :warning:
      - Inner product implays that matrix has to be symetric
      - Linearity of inner product implays marix representation
      - Positive defnite of inner product implay positive def of a matrix
        thats just synonim
    - - If matrix is positive than the diagonal coefficient have to be positive
      - For a 2 by 2 matrix you can consider a quadratic function
        of 1 variable (can always skale such that y=1) and can came
        up with general condition
        \(a_{11}>0 \) , \(\det A>0 \)
      - Sylvester's criterion states that a Hermitian matrix M is positive-definite if and only if all of the leading principal minors must be positive.
      - 25.The LDU and LDLᵀ Decompositions and the
        Implications for Positive Definiteness
        26.Eigenvectors of Symmetric Matrices Are Orthogonal
        27.A Symmetric Eigenvalue Decomposition Example
        in under Three Minutes!
        28.A 3x3 Symmetric Eigenvalue Decomposition
        in under 3 Minutes
        
        LDU =LDLᵀ if and only
        if matrix is symetric
        
        For a matrix to be positive definite, all the pivots
        of the matrix should be positive :warning:
        
        Pivots are the first non-zero element in each row of a matrix that is in Row-Echelon form. Row-Echelon form of a matrix is the final resultant matrix of Gaussian Elimination technique.
        
        Eigenvectors of Symmetric Matrices Are Orthogonal
        
        If all the Eigen values of the symmetric matrix are positive, then it is a positive definite matrix.
        
        positive deff. can be define for non symetric/hermitian matrix
  - - - \(f''(x)>0 \)
        is equivalent in multivariable case is
        matrix \(H\) (hessian) is positive define
        
        that comes from "\(f''(t)>0\)"
        for all lines that goes from x
      - Critical point occures when \(f'(x)=0\)
        is equivalent in multivariable case is
        \(\nabla f(x)=0\)
        
        that comes from "\(f'(t)=0\)"
        for all lines that goes from x
    - - Least squares formula for
        overdetermined system \(Ax=B\)
        \(x=(A^{T}A)^{-1}A^{t}B\)
        
        Simp. to \(x=A^{-1}B\) if \(A^{-1}\) exists
        
        comes from minimalizing the length \(r^{T}r\),
        where \(r=B-Ax\) is an "error" vector
      - You can solve aproximatly system
        \(Ax=B\)
        for positive def matrix A, by minimalieze function
        \(f(x)=1/2x^{T}Ax-x^{T}B\)
- - - - 36.Fourier Series Is Nothing but a Least Squares Problem! :warning:
        37.A Few World about the Infinite Fourier Series
    - - 34.Decomposition by Inner Product with General Bases :warning:
        35.Decomposition by Inner Product in the
        Underdetermined Case and Least Squares