Please enable JavaScript.

Coggle requires JavaScript to display documents.

Similarity, Neighbors, and Clusters (Fundamental concepts: Calculating…

- - - - if we have two (numeric) features, then each object is a point in a two-dimensional space
        
        (xA - xB)2 + (yA - yB)2. Essentially, we can compute the overall distance by computing the distances of the individual dimensions—the in‐ dividual features in our setting. This is called the Euclidean distance 1 between two points, and it’s probably the most common geometric distance measure.
        
        they could be represented by points in three-dimensional space and their positions would then be represented as (xA, yA, zA) and (xB, yB, zB). The distance between A and B would then include the term (zA–zB)2.
- - - - This concept generalizes to other sorts of prediction tasks, for example regression and class probability estimation.
        
        Consequently, when using weighted scoring the exact value of k is much less critical than with majority voting or unweighted averaging. Some methods avoiding committing to a k by retrieving a very large number of instances (e.g., all instances, k = n) and depend upon distance weighting to moderate the influences.
        
        Geometric Interpretation, Overfitting, and Complexity Control
        
        As with other models we’ve seen, it is instructive to visualize the classification regions created by a nearest-neighbor method.
        These regions can be calculated by systematically probing points in the instance space, determining each point’s classi‐ fication, and constructing the boundary where classifications change.
        
        iirregular concept bound‐ aries are characteristic of all nearest-neighbor classifiers, because they do not impose any particular geometric form on the classifier. Instead, they form boundaries in in‐ stance space tailored to the specific data used for training.
        
        Thus, in terms of overfitting and its avoidance, the k in a k-NN classifier is a complexity parameter. At one extreme, we can set k = n and we do not allow much complexity at all in our model.the n-NN model (ignoring similarity weight‐ ing) simply predicts the average value in the dataset for each case. At the other extreme, we can set k = 1, and we will get an extremely complex model, which places complicated boundaries such that every training example will be in a region labeled by its own class.
        
        1 more item...
      - Issues with Nearest-Neighbor Methods
        -Intelligibility
        -Dimensionality and domain knowledge
        -Computational efficiency
- - - - Euclidean distance is probably the most widely used distance met‐ ric in data science.
        
        Equation 6-2. Euclidean distance (L2 norm)
        dEuclidean(X,Y)= ∥X-Y∥2 = (x1 -y1)2 +(x2 -y2)2 + ⋯
        
        Equation 6-3. Manhattan distance (L1 norm)
        dManhattan(X,Y)= ∥X-Y∥1 = |x1 -y1 | + |x2 -y2 | + ⋯
        
        edit distance or the Levenshtein metric. This metric counts the minimum number of edit operations required to convert one string into the other, where an edit operation consists of either inserting, deleting, or replacing a character (one could choose other edit operators)