Please enable JavaScript.

Coggle requires JavaScript to display documents.

Support Vector Machines (Support Vector (image (Dots outside dotted lines…

- - - - Constrained Optimization Problem
        
        Solve
        
        Lagrange
        
        Maximize
        
        Minimize
        
        Largest margin
      - General
        
        \(m=\frac{2}{\mid \mid w \mid \mid}\)
        w = normal to hyperplane
        
        Maximizing the margin = minimizing the weights
        
        The smaller the magnitude of w the larger the margin is
        
        \(\gamma = \frac{1}{\mid \mid w \mid \mid }\)
    - - Line through smallest points (of each classification)
- - - - Slack
        
        Minimization
        
        Inverse of margin
        
        Equation
        
        \( min_{w,b,\xi}\frac{1}{\gamma(\mathbf w, b)} + C\sum_n\xi_n\)
        
        \(\gamma(\mathbf w, b)\) = large margin
        \(C\sum_n\xi_n\) = small slack
        
        Best > Perfect
        
        Too much = bad
        
        Penalized for slack
- - - - Hingle loss = upperbound 0-1 loss
  - - - When greater than 1, loss = 0
        Growing penalty the more negative ya(x) is
      - Black line = 0-1 Loss
        Red = Hinge Loss
- - - - Sum of equality constraints = 0 since
        h(x) = 0
  - - - Objective function + inequality constraints + equality constraints
        
        Why does this work
        
        for(each value of x)
        for each valueof alpha and beta
        calculate Lagrangian
      - Minimize x then maximize over constraints
    - - Maximize constraints then minimize over x
        
        We have it in terms of the inner product
        
        We can use kernel function with dual
    - - Weak duality
        
        Dual solution = lower bound
        \(d^* \leq p^*\)
      - Strong duality
        
        \(p^* = d^*\) under certain conditions
        
        Holds for SVM
- - - - High \(\gamma\) = low spread
        Low High \(\gamma\) = far spread
      - Decision boundary
        
        High \(\gamma\) = straight boundary
        Low High \(\gamma\) = curvy boundary
- - - - \(min_\theta max_\alpha L(\theta,\alpha)\)
    - - \( max_\alpha min_\theta L(\theta,\alpha)\)
    - - L = Lagrangian
  - - - Find derivative with respect to \(\theta\)
        Gives us \(\theta\) in terms of \(\alpha\)
        
        Plug in solution for \(\theta\)
        
        Find derivative with respect to \(\theta\)
        Gives us \(\alpha\)
        
        Now that we have \(\alpha\) we can plug it into the result of the first derivative