Please enable JavaScript.

Coggle requires JavaScript to display documents.

Mathematics (:banana:Linear Algebra (:green_apple:scalar (:explode:a…

- - - - :explode:变量的值是随机的,这个变量就是随机变量
    - - :pencil2:概率 = P(随机变量)
      - :icecream:P(X=0) = 0.4; P(X=1) = 0.6
    - - :pencil2:\( P(x|y) = \dfrac{P(x)P(y|x)}{P(y)} \)
      - :heavy_dollar_sign:when we know \( P(y|x) \) and need to know \( P(x|y) \)
        
        :fire:it is usually feasible to compute \( P(y) = \sum_x{P(y|x)P(x)} \), so we do not need to begin with the knowledge of \( P(y) \)
    - - :one:伯努利分布
      - :two:高斯分布
  - - - :one:假设存在N个点\( (x_i, y_i) \),我们要拟合的直线为\( y = wx + b \)
      - :two:目的是要让每个点和直线的误差最小,所以损失函数是\( L = \sum_i{ (y^{(i)} - w x^{(i)} - b)^2 } \)
        
        :warning:~~点到直线的距离~~
        
        :zap:计算的是y和\(y^*\)之间的误差
        
        :warning:~~除以N~~
        
        :bread:我觉得其实除以不除以差别不大
      - :three:因为是凸函数,求最小值就是解决下面的方程\( \dfrac{\partial L}{\partial w} =0; \dfrac{\partial L}{\partial b} = 0 \)
      - :four:上面的方程可以求解出来为\( w = \dfrac{\sum_i{x^{(i)} (y^{(i)} - b) }}{\sum_i { (x^{(i)} )^2 } }; b = \dfrac{1}{N} \sum_i{( y^{(i)} - w x^{(i)} )}; \) 之后只要联立这两个方程就可以了
    - - :one:假定直线为\( y = w_0 + w_1 x + w_2 x^2 + ... + w_N x^N \), 所以直线的参数可以是\( (w_0, w_1, w_2, ... , w_N) \)
      - :link:详细参考b站视频最小二乘线性回归从入门到放弃 :-)
- - - - :heavy_dollar_sign:discriminate between elements that are exactly zero and elements that are small but nonzero
    - - :explode:largest magnitude in the vector
      - :pencil2:\( \lVert x \rVert_\infty = \max_i \lvert x_i \rvert \)
  - - - :one:unit vector
        
        :explode:a vector with \( L^2 \) norm equals to one. (\( \lVert x \rVert_2 = 1 \))
      - :two:orthogonal
        
        :explode:a vector \( \textbf{x} \) and a vector \( \textbf{y} \) are orthogonal to each other if \( \textbf{x}^T\textbf{y} = 0 \)
        
        :bread:\( \textbf{x}^T\textbf{y} \) <=> dot production
        
        :pill:orthogonal matrix
        
        :explode:every row or column vector is orthogonal to other vectors & unit
        
        :fire:\( \textbf{A}^T\textbf{A} = \textbf{A}\textbf{A}^T = \textbf{I} \)
        
        :arrow_right:\( \textbf{A}^{-1} = \textbf{A}^T \)
        
        :fire:the inverse is cheap to compute
    - - \(y = \sum{A \times B^T}\)
        
        \(\frac{\partial y}{\partial A} = \frac{\partial y}{\partial u} \times \frac{\partial u}{\partial A} = \frac{\partial y}{\partial u} \times B, \text{where } u = A \times B^T\)
        
        \(\frac{\partial y}{\partial B} = \frac{\partial y}{\partial u} \times \frac{\partial u}{\partial B} = \frac{\partial y}{\partial u}^T \times A, \text{where } u = A \times B^T\)
      - \(y = \sum{A \times B}\)
        
        \(\frac{\partial y}{\partial A} = \frac{\partial y}{\partial u} \times \frac{\partial u}{\partial A} = \frac{\partial y}{\partial u} \times B^T, \text{where } u = A \times B\)
        
        \(\frac{\partial y}{\partial B} = \frac{\partial y}{\partial u} \times \frac{\partial u}{\partial B} = A^T \times \frac{\partial y}{\partial u}, \text{where } u = A \times B\)
  - - - :one:Principal Components Analysis, PCA
      - :two:Linea Discriminant Analysis, LDA
      - :three:Classical Multidimensional Scaling, MDS
    - - :one:Isometric Mapping, Iosmap
      - :two:Locally Linear Embedding, LLe
      - :three:Laplacian Eigenmaps, LLE
    - - :heavy_dollar_sign:比较2个向量的相似度
- - - - :explode:-log2(p(x))
      - :bread:x是随机变量,而p(x)表示某个取值发生的概率
      - :baguette_bread:这个公式不是直接得到的,而是通过一些性质最终选择这个函数的,这些性质分别是
        
        :one:f(1)=0
        
        :two:f(0.5) = 1, 归一化使用
        
        :three:如果p(x) < p(y), 那么f(p(x)) > f(p(y))
        
        :four:函数f在输入空间内是连续的
        
        :five:2个事件同时发生的"惊喜度"等于各自事件发生的"惊喜度"的和
      - :arrow_right:随机变量整体的惊喜度
        
        :explode:-Σp(x)log2(p(x))
        
        :bread:Σ(每个时间发生的概率 * 该事件的惊喜度)
        
        :baguette_bread:乘上发生的概率,是作为权重,因为不同事件的比重肯定不同
        
        :arrow_right:扔硬币问题(假设正面向上概率为p)
        
        :explode:- p * log2(p) - (1 - p) * log2(1 - p)
- - - - 凸函数
        
        偏导为0的时候
        
        计算得到w和b
        
        多维情况下就是矩阵运算了