Please enable JavaScript.

Coggle requires JavaScript to display documents.

Machine Learning (Optimization (条件最优化 (拉格朗日乘子), Gradient Descent, 牛顿法与拟牛顿法…

- - - - a: 从样本训练集中选择K个与新样本最相似的实例(instance);
      - b: K个数据中出现次数最多的分类，作为新样本的分类
    - - 精度高、对异常值不敏感、无特殊假设
      - 计算复杂度高、空间复杂度高
    - - 特殊的数据存储结构
      - 二叉树，表示对空间的划分
    - - 距离度量
      - K值选择
      - 分类决策规则
  - - - 特征之间相互独立
      - 特征同等重要
- - - - Standardization; Z-score normalization--归一化后的特征将服从标准正态分布 (均值和标准差)
      - Min-Max Scaling; Min-Max normalization--x’ = (x-min(x))/(max(x)-min(x))
      - Nonlinear Scaling--Log
      - PowerTransformer
      - QuantileTransformer
    - - Batch Normalization
        
        BN 比较适用的场景是：每个 mini-batch 比较大，数据分布比较接近。在进行训练之前，要做好充分的 shuffle. 否则效果会差很多。
        
        由于 BN 需要在运行过程中统计每个 mini-batch 的一阶统计量和二阶统计量，因此不适用于动态的网络结构和 RNN 网络
      - Layer Normalization
        
        Layer normalization 对于recurrent neural networks 的帮助最大
        
        Despite its simplicity, batch normalization requires running averages of the summed input statistics. In feed-forward networks with fixed depth, it is straightforward to store the statistics separately for each hidden layer. However, the summed inputs to the recurrent neurons in a recurrent neural network (RNN) often vary with the length of the sequence so applying batch normalization to RNNs appears to require different statistics for different time-steps.
        
        under layer normalization, all the hidden units in a layer share the same normalization terms µ and σ, but different training cases have different normalization terms.
      - Weight Normalization
        
        work well for RNNs