Please enable JavaScript.

Coggle requires JavaScript to display documents.

Summary 5 - all - Coggle Diagram

- - - - Main task: find out which items are connected to occurrences of other items
      - inputs: transaction dataset where a transaction is an instance of an item set
      - outputs: association rules
        
        "if a customer buys x, they are likely to buy y"
        
        ** x and y cannot have overlapping items
      - evaluate good output
        
        support: how frequent is an association rule
        
        confidence: how often items in Y appear in transactions that contain X
        
        lift: how much more likely an item-set occurs than pure chance
      - Process: apriori algorithm
        
        keep only item sets where support >= minimum support
        
        "if an itemset X is NOT frequent, then any larger itemset containing X cannot be frequent"
    - - Clustering (groups)
        
        Outputs
        
        Clusters
        
        What are good clusters?
        
        Low inter-similarity: data points in different clusters should be different from each other
        
        distance between clusters via Linkage methods
        
        High intra-similarity: data points in the same cluster should be similar to each other
        
        measures of similarity
        
        4 more items...
        
        Inputs
        
        A dataset of N columns (features) and M rows
        
        Applications
        
        Facilitate the analysis of large datasets
        
        Discover natural groups and patterns in the data
        
        Definition: the grouping of observations on the basis of their features; organizing data points into meaningful groups called clusters
        
        Hierarchical clustering
        
        Agglomerative/bottom-up clustering; start from individual data points or smaller clusters
        
        forming larger clusters from smaller ones in a hierarchical manner
        
        Procedure
        
        Preparation: compute the distance between points based on the distance measure of choice and create a distance matrix
        
        1 more item...
        
        consider each data point individually as a one-point cluster
        
        merge two one-point clusters that are nearest to each other and form new clusters
        
        continue by merging 2 clusters of any size closer to each other
        
        repeat until there is only one or k clusters left
        
        output
        
        main output: dendrogram: diagram that shows cluster hierarchy
        
        1 more item...
        
        pros
        
        data-driven, no need to pre-specify the number of clusters, gives you solutions to any number of clusters
        
        cons
        
        computationally demanding
        
        Partitioning-based clustering
        
        K-means clustering
        
        directly partition the data into K-groups, where k is the number of clusters (pre-specified)
        
        partition recursively, re-assigning points to the closest cluster
        
        procedure
        
        7 more items...
        
        pros
        
        1 more item...
        
        cons
        
        1 more item...
        
        How to choose K
        
        Subjective and depends on needs, business goal, domain knowledge
  - - - Definition: predicts continuoous/numeric values of dependent variable for new observation
      - input: observation/record defined by collection of available attributes
        
        Dependent/outcome variable Y, Independent/predictor variable X
      - steps
        
        model construction (training)
        
        randomly split data into training data to build model, validation data to tune performance, and test data to perform the model
        
        regression trees: try to classify data into predefined classes based on a set of "decision rules" organized as a tree
        
        pros: no parametric assumptions, good for variable selection, robust to outliers
        
        cons: unstable, can miss relationships between attributes
        
        k-NN; identify nearest neighbors, numeric prediction for new observation is made using the average of the nearest neighbors' outcome values, may be weighted average with weight decreasing distance
        
        pick a distance metric and k value. pick to minimize RMSE
        
        normalize data if needed (only x variables)
        
        for every new observation, identify k-nearest existing observations based on computed distance
        
        predict outcome for new observation by taking average of nearest neighbor outcome values
        
        pros: simple, effective at capturing complex relationships, does not make assumptions about data distribution
        
        cons: curse of dimensionality, not good for real time predictions
        
        model testing
        
        evaluating performance:
        
        mean error: average of prediction errors
        
        mean absolute percentage error
        
        mean absolute error: use absolute value of error to give an idea of magnitude of average error
        
        total sum of squared error: penalizes larger errors
        
        root mean squared error: square the prediction errors; sum, take the average and then the square-root; penalizes larger errors
        
        prediction error: difference between predicted outcome and actual outcome
    - - objective: identify to which "class" a new observation belongs to, based on data containing observations whose class variable is already known
      - Steps
        
        Model construction (training): construct a classification model based on training data
        
        1.1 randomly split the data into training data, validation data, test data
        
        1.2 choose classification model and train it on training data
        
        K-NN (nearest neighbor): for a given record, identify "nearest" (most similar) observation
        
        use distance metrics seen in clustering (e.g. Euclidean) to know if an observation is "near" another
        
        k-NN will classify observation according to the predominant class (outcome variable) among the identified K nearest neighbor
        
        must decide k, how many neighbors we want to consider
        
        Procedure
        
        pick K, how many neighbors we want to consider
        
        1 more item...
        
        pick a distance measure; determine whether you need to normalize the data
        
        3 more items...
        
        after standardization, compute pairwise Euclidian distance between new observation and existing ones
        
        pick the 1-nearest neighbor, aka the point with the shortest distance
        
        1 more item...
        
        trade-offs
        
        pros: easy to use, does not require statistical assumptions
        
        cons: known as the 'lazy' classifier because there is no real model building, we use labeled data available in our dataset as model; bad for high-dimension datasets; not good for real-time prediction
        
        Decision tree: combines numeric and binary data, starting from a top node and go down to the leaf nodes
        
        terminology
        
        Root node/root: top of the tree
        
        internal/decision nodes/branches: subsequent nodes after root node but not the last nodes
        
        branches: arrows connecting nodes showing direction of tree flow
        
        leaf nodes/leaves: last nodes each contain a class label
        
        purity: measured through the concept of entropy; the lower the entropy, the purer the given data partition, the more the observations in that partition have the same value for the outcome variable
        
        1 more item...
        
        information gain: how to partition data and which should be the root?; information gain = entropy(parent node) — weighted entropy (children node)
        
        stopping criteria: when to stop splitting nodes and create a leaf node
        
        when all data points in a node are from the same class; there are no remaining attributes to further split the data
        
        trade offs
        
        pros: good for variable selection; does not require assumptions about data, robust to outliers
        
        cons: unstable from slight changes in data; it can miss interesting relationships between predictors
        
        Model validation: refine classification model on validation data (sometimes skipped)
        
        Model testing: measure accuracy of final model using test data
        
        3.1 assess "how accurate" the model is by testing on test data
        
        determine unbalanced dataset through confusion matrix
        
        error rate: how many did the model get wrong?
        
        accuracy: how many did the model get right?
        
        recall: for each actual class, how many were recovered?
        
        sensitivity
        
        specificity
        
        precision: for predicted class how many did the model get right?
        
        F1-score: if we care about both recall and precision; harmonic mean of precision and recall; f1 is preferred over accuracy when data is unbalanced
        
        3.2 apply model to classify new data if it is "good enough"