Please enable JavaScript.

Coggle requires JavaScript to display documents.

Classification (Supervised learning), SVM v.s. Neural Network, instance…

- - - - 特性：
        
        Select the attribute with the highest information gain （gain越大，鑑別力越大）
        
        Attribute Selection Measures
        
        Information gain：attribute的value越多bias越大
        
        Gian ratio：prefer unbalanced splits
        
        Gini index：每一個partition 內data含量比較平均、class 種類較多時處理不好、attribute的value越多bias越大
        
        Attributes are categorical (if continuous-valued, they are discretized in advance)
      - 缺點：
        
        Tree的高度太高、包含太多分支的話會 overfitting
      - 優點：
        
        Allow for continuous-valued attributes （離散化、concept hierarchy）
        
        Handle missing attribute values
        
        Attribute construction，建構新的attribute做decision tree
        
        Scalability: Classifying data sets with millions of examples and hundreds of attributes with reasonable speed
        
        建構出來的tree結構與rule很容易了解、解釋分類的前因後果
  - - - Bayesian Classification
        
        種類
        
        naïve Bayesian classifier （independent attributes ）
        
        優點：
        
        Easy to implement
        
        缺點：
        
        Assumption: class conditional independence
        
        Bayesian Belief Networks （dependent attributes）
        
        特性
        
        假設資料之間的attribute是independent，利用貝氏定理來計算每個資料屬於各個類別的機率，，哪一個類別的機率最高就assign給他
    - - 特性：
        
        for both linear and nonlinear data
        
        利用nonlinear mapping做高維度的轉換，原本的data在低維度是nonlinear問題，用Kernel function轉換到高維度後就變成linear
        
        SVM finds the hyperplane to split data, using support vectors (“essential” training tuples) and margins (defined by the support vectors). Find hyperplane maximizes the margin
        
        如果問題無法完整切分乾淨 -> slack variables （做最佳化的時候允許誤差）
      - 缺點：
        
        Gaussian RBF overfits badly 會造成“Christmas Tree” effect
        
        效能較不佳，由於時間複雜度為O(n²)當有超過一萬筆資料時，運算速度會慢上許多
  - - - McCulloch-Pitts Model
        
        特性：
        
        很常用來預測股價趨勢，以及會不會買、買的機率是多少
        
        不只預測class label，更可以預測「值」
        
        如果是linear separable 的值用一個node就可以分開
      - A Neuron = a Perceptron
        
        缺點
        
        poorly behaved when patterns are non-linear separable
    - - 優點：
        
        Overcome the limitations of linear networks for
        function approximation
        
        Overcome the limitations of perceptrons for pattern recognition
      - Convolutional Neural Network（CNN）
        
        特性：
        
        透過convolutional 函數學習資料特徵進行分類
        
        a feed-forward network that can extract topological properties from an image
        
        優點
        
        can recognize patterns with extreme variability (such as handwritten characters). :arrow_right: 透過convolution operation來擷取不同特徵，這些特徵跟大小、位置無關
        
        recognize visual patterns directly from pixel images with minimal preprocessing.
        
        將擷取特徵與分類二合一
      - 缺點：
        
        Vanishing gradient問題：用sigmoid 或hyperbolic當activation function時，當層數越來越多用gradient decent調整參數時，越後面層數的參數調整幅度會越來越小
      - Recurrent Neural Network（RNN）
        
        Long Short-Term Memory（LSTM）
        
        a form of RNN. It fixes the vanishing gradient problem of the original RNN.
        
        除了紀錄output以外也會紀錄 cell state
        
        Input data : sequence, a form of neural networks that feed outputs back to the inputs during operation
        
        例子：每一天的data 形成一個sequence來預測股市的漲跌
        
        缺點：
        
        gradient vanishing problem：RNN only produces output h at every time update, so the gradient vanishing problem may exist
    - - Long training time（尤其是當input data 大、複雜網路、很多參數要調整）
      - Require a number of parameters typically best determined（需要domain knowledge）
      - Poor interpretability（很難解釋結果）
    - - High tolerance to noisy data
      - 􏲎􏷉􏳤􏱋􏷋􏶌􏰒􏰄􏱞􏰦􏱞􏲁􏰕􏰖􏲶􏰪􏷣􏰒􏰌即使沒有看過的data 也可以做不錯的分類
      - Well-suited for continuous-valued inputs and outputs
      - 實務上很多領域都可以使用
      - Algorithms are inherently parallel
- - - - Returns the mean values of the k nearest neighbors
    - - Robust to noisy data by averaging k-nearest neighbors
    - - 維度越高點跟點的距離被放大，比較相關的attributes在用距離表現時算不出來
  - - - Reproduction
      - Crossover
      - Mutation
- - - - A classifier model Ci is learned for each training set Di
    - - 將Ｘ給各個Ci做分類，用多數決的方式來assign X
    - - Often significant better than a single classifier
      - more robust for noisy data
      - Improve accuracy in prediction
  - - - 訓練時會不斷調整每個classifier model的權重。分類效果好的權重較高，且資料如果被分錯，就把分錯的data權重加重
      - 優點：
        
        非常容易implement
        
        Does feature selection resulting in relatively simple
        classifier
        
        Fairly good generalization
      - 缺點：
        
        Suboptimal solution
        
        Sensitive to noisy data and outliers
- - - - a variation of holdout
      - 做很多次holdout取平均值
  - - - 一次留一個做testing，其他做training