Please enable JavaScript.

Coggle requires JavaScript to display documents.

Complex Systems (Data Mining (Prediction: Some attributes can be used to…

- - - - Decision Trees
      - Support Vector Machines (SVM)
        
        Uses
        
        2-Class Prediction
      - Neural Networks
      - Rule-Based Systems
      - Random Forests
      - Bayesian Predictors
        
        Assumptions
        
        All attributes are equally important
        
        All attributes are statistically independent (given the class value)
      - k-Nearest Neighbours
      - 1R (one-level decision tree)
        
        Rules all test one particular attribute
        
        Algorithm
        
        For a single given categorical/nominal attribute, create one branch for each value that the attribute can take. 2. Label each branch with the most frequent class (target attribute) that occurs when the attribute being examined is that value. 3. Set the error rate of each branch to be the proportion of instances that don't belong to the value of that particular branch. 4. Choose the branch with the lowest error rate.
    - - Prediction Accuracy
      - Confusion Matrix
      - ROC Curve
    - - Classification: Finite number of target attribute values
      - Regression: Continuous target attribute values
      - Transparent Predictor: You can tell how the predictor makes its decision
      - Opaque Predictor: You can't tell how the predictor makes its decision
    - - Ensemble Methods
        
        Bagging
        
        Instead of using one big training set, use a bunch of smaller training sets by sampling with replacement. Build a predictor from each sample. Deploy all of the predictors on a test record and let them vote their predictions, which leads to one overall prediction based on the weights of the different votes.
        
        Voting
        
        Purpose of voting is to reduce variance, which increases prediction accuracy
        
        Weighting
        
        Equal
        
        By prediction accuracy
        
        By size for decision trees
        
        By margin for SVMs
        
        By self-declared confidence for a NN
        
        Boosting
        
        Take the results of each predictor into account when building the next one. Start by labeling each training record with a probability of being selected for a sample of 1/n, so they are all equally likely to be chosen. Build a predictor using the whole set, then use it on the whole set, then modify the probability of being selected for each one. If the predictor performed poorly on a record, increase the probability of that record being chosen again. Repeat so that the hardest records to predict correctly are used the most often for training. Similarly to bagging, each predictor votes.
        
        Gradient Boosting
        
        First model makes predictions. Compare predictions to the correct answers to calculate vector of errors. Create a second model to try to predict the errors of the first model. Combine the predictions of M1 and M2 to create a new set of more accurate predictions. Calculate the error of this new set of predictions and try to predict that using a third model M3. Continue this forever.
      - Monolithic Method
        
        Take one set of training data and use that for one predictor
    - - Techniques
        
        Forward Selection
        
        Start with one attribute. Build a predictor and remember the accuracy. Add another attribute, retrain, repredict, and see if the accuracy increases. If so, keep the new attribute, else discard it and move on to the next one.
        
        Backward Selection
        
        Start with all attributes. Train, predict, record accuracy. Try dropping an attribute. If the accuracy decreases, add it back, otherwise leave it out. Continue for all attributes until you only have the ones that affect prediction accuracy left
      - Goals
        
        Get rid of attributes that do not correlate with the target attribute
        
        Get rid of attributes that are correlated but do not offer new information (ie: there's already another attribute that's correlated and gives the same information)
  - - - Distance-Based: Similarity = Closeness
      - Distribution-based: Similarity = Frequency
      - Hierarchical
        
        Approaches
        
        Top-down
        
        Assume all objects are part of one cluster. Find a way to split it. Repeat
        
        Bottom-up
        
        Assume each object is it's own cluster. Join the two most similar clusters. Repeat. Preferred over top-down because there are faster algorithms for it.
      - Matrix Decomposition
        
        Think of the original dataset as a matrix. Now think of it as the product of two other matrices. We can imagine that one of these matrices has some underlying distribution.
        
        Decompositions
        
        Singular value decomposition (SVD)
        
        Semidiscrete decomposition (SDD)
        
        Independent component analysis (ICA)
        
        Nonnegative matrix factorization (NNMF)
        
        Ways to interpret a matrix decomposition
        
        Factor interpretation
        
        One of the matrices being multiplied represents a set of factors
        
        Geometric interpretation
        
        One of the matrices being multiplied represents axes, and the other matrix gives the coordinates of each record w.r.t. these axes
        
        Component interpretation
        
        The dataset is the result of adding together a set of component matrices. Each component matrix represents an underlying process.
      - Deep Learning
    - - Much harder than prediction
      - Want clusters to be tight and well-separated
  - - - Important because
        
        Some prediction and clustering techniques do better or worse depending on how attributes are scaled
        
        Correlation between attributes can be lost if one is scaled and the other isn't
      - Approaches
        
        Ranking
        
        z-score
        
        Linear scaling
  - - - Adjacency matrix. Row and column labels are the same. Values represent connections between people in the network.
      - Edges
        
        Connected or unconnected
        
        Can be typed
        
        Can be signed (positive or negative)
        
        Can be weighted