Please enable JavaScript.

Coggle requires JavaScript to display documents.

Data Methods (Regression (Linear (Goodness of fit (R²) (Estimated Sums of…

- - - - ß1 = cov(x,y) / var(x)
      - ß0 = y-ß1*x
    - - Estimated Sums of Squares
        (Regression line - Sample Mean)²
      - Residual Sums of Squares
        (Point - Regression line)²
      - Number of expl. variables
      - Adjusted R²
    - - Table
    - - Dummy variables
        
        Types
        
        Slope + Intercept
        
        Intercept
        
        Slope
        
        Regression
        
        Logistic
      - Replacement
        
        Quadratic: z=x²
        
        Interaction terms: z=x1*x2
      - Log for exp. terms
    - - Best subset
      - Backward elimination
      - Forward selection
      - Stepwise regression (Top-Down + Bottom-Up)
  - - - ß0 = Constant, moves curve horizontally
      - ß1 = Slope, steepness)
        
        ß1 < 0 -> Odds go down
        
        ß1 > 0 -> Odds go up
        
        e^(ß1) -> odds ratio
      - ß0/ß1 = Threshold of 0,5 success
    - - Likelihood function (L) measures prob
      - Log of L (LL) to be maximized
    - - Null deviance
        (Explained by only intercept)
      - Residual deviance
        (Explained by fitted model
        (parameters + intercept))
      - Likelihood ratio test (Model is explanatory value)
      - A Wald test (test if coefficient =0)
      - McFadden R² (>0.2 acceptable, >0.4 ok)
  - - - Mean = Variance (Model)
        Mean < Variance (Real)
        -> Overdispersion
        -> Over-Confidence in model
      - Zero-Inflation underpredicted
- - - - Training
      - Testing
  - - - Leave-One-Out Holdout (n-fold)
      - k-Fold
    - - Error rate
      - Recall
      - Precision
      - Specificity
      - False alarm rate
  - - - Probably approx. correct (PAC) learning
        (Finite Hypothesis Space)
      - Vapnik-Chervonenkis (VC) theory
        (Infinite Hypothesis Space)
      - Concepts
        
        Sample Complexity
        
        Computational Complexity
        
        Mistake Bound
    - - Kolmogorov Complexity
      - MInimum Description Length Principle
- - - - Bias
      - Signal(s)
      - Weights
    - - Classification function
        [Perceptron]
        
        Perceptron Error
      - Logistic
        [Logistic neuron]
        
        Classification Error
      - ReLU activation
        [ReLU neuron]
      - None
        [Linear neuron]
  - - - Avg. misclassification
      - Learning = Minimize cost function
      - Logistic Neuron Optimization
    - - Learning rate (small)
        
        Oscillation risk if too large
        
        Stall if gradient = 0
      - Random start values
      - Update value
    - - Weights
      - Node input
      - Node output
  - - - Hidden layers
      - Nodes per Layer
      - Activation function
      - Learning rate
      - Iterations & Desired Error Level
- - - - Orthogonal (Varimax)
      - Oblique (OBLIMIN)
- - - - Parametric tests
        
        t-Test
        
        One-Tailed
        
        Two-Tailed
        
        Purpose
        
        Single samples
        
        Paired samples
        
        Independent (unpaired) samples
        
        Possible errors
        
        Alpha error (H0 rejected although true)
        
        Beta error (H0 accepted although false)
        
        Significance
      - Probability Distribution
        
        Chi² test of independence
      - Non-parametric tests
        
        Wilcoxon signed-rank
        
        Mann-Whintey-U
        
        Kruskal-Wallis
- - - - Similarity
      - Distance
    - - Single linkage (smallest distance)
      - Complete linkage (largest distance)
      - Ward
- - - - Conditional Probability
        
        Frequency Tables [P(a)]
        
        Likelihood Tables [P(a|e)]
      - Bayes Rule/Theorem
      - Zero Frequency Problem
        
        Add 1 to each numerator
        
        Add constant
        
        Add constant * weight
      - Missing value problem
        
        Omitting variable from calculation
      - Numeric attributes
        
        Normal distr
        
        Probability density function
        
        Unknown distr
        
        Kernel density estimation
        
        K(x-xi, h) -> Kernel functions
        
        h -> bandwith
    - - Choose splitting attribute
        
        Information gain
        (Increase purity of subsets)
        
        Information entropy
        
        Information before - after split
        -> Select highest
        
        Entropy as purity measure
        
        Doesn't work well with highly branching attributes
        
        Gain ratio
        
        Reduced bias on high-branch attributes
        
        Takes intrinsic info about split into account (new info entropy)
        
        Gini index
        
        Select split that reduces Gini Index the most
      - Relational rules
        
        Attributes <>= Constant/Attribute
        
        More expressive in some cases
        
        DT as example of propositional learner
      - Numeric attributes
        
        Split binary by best possible split point (info gain)
        [Allow multi-way splits if better]
        
        Use numerical attributes multiple times in tree
      - Missing values
        
        Training
        
        Ignore attributes
        
        Treat as nominal value
        
        Estimate
        
        Ignore instances
        
        Classification
        
        Partition
        
        Follow leader
      - Building DT
        
        Top-Down construction
        
        Bottom-up Pruning to reduce overfitting
        [Good training accuracy, poor test accuracy]
        
        Total costs = mnlog(n)
        
        Scalability depends on amount of attributes/data
    - - Theory
        
        Describe independence among subsets
        
        Allow dependencies b/w variables within subsets
      - Probability Laws
        
        Chain rule
        (Base for network)
        
        Conditional independence
        (Base for trees)
      - Directed Acyclic Graph (DAG)
        
        Nodes hold Conditional Probability Tables (CPT)
        
        Node for each attribute
      - Inference of
        event prob.
        
        Topology
        
        Evidence
      - Learning Bayes Nets
        
        Method to evaluate goodness of network
        
        Method to search through possible network space
- - - - Error rate
      - C4.5 uses conf. interval
        of normal distribution
        
        Error estimate
  - - - Data cleaning
        
        Missing values
        
        Conversion
        
        Ordered to numeric
        
        Nominal to binary or ignore
      - Balanced training data
      - Discretization
        
        Equal frequency
        
        Class dependent
        
        Transform to binary attributes
      - Feature selection
        
        Best subset
        
        Backward elimination
        
        Forward selection
        
        Stepwise regression
    - - Visualize & Summarize data
      - Data understanding quantitative conditions
  - - - External
      - Internal
      - Sample selection bias
    - - Quasi-experiments
      - Observational studies
      - Randomized control trials
- - - - 8 No correlation b/w residuals
        
        Durbin-Watson d-test
        
        Model seasonality with dummy var.
      - 9 Covariance b/w X and u = 0
      - 10 Mean u für given X = 0
        
        Check of omitted var (Endogeneity).
        
        In panel data,
        individual effects
        need to be treated
        
        Fixed effects
        (Indiv. effect corr.
        to covariates)
        
        First-Difference
        
        Within estimator
        
        Least squares dummy estimator
        
        Random effects (Indiv. effects uncorr. to covariates)
        
        Hausman Test
      - 11 u normally distributed
      - 7 Homoskedasticity (Variance of Residuals constant)
        
        White test
    - - 5 Independent variable(s) is (are) deterministic
      - 4 Variance of indep. variables = 0
      - 6 No perfect multicollinearity
        
        Check correlations of predictor variables
        
        Variance inflation factor
        (Explaining predictor by other variables)
    - - 2 Model has to be linear in parameters
        
        Linearize!
      - 3 Number of observation > Number of independent variables
      - 1 Correct specification of regression model
- - - - Ranking of pairs
      - Rating
      - Anchoring
    - - Euklidian
      - City-Block
- - - - 5 Aggregation of utility values
        
        Normalize to most preferred stimuli
        
        Transform to same zero-point
      - 4 Estimation of utility values
        
        Metric ANOVA
        
        Monotonous ANOVA
      - 3 Evaluation of stimuli
      - 2 Survey design
        
        Full-Profile
        
        Latin square (Max 3 attributes, unlimited levels)
        
        Reduced design
        
        Two-Factor-Method/
        Trade-Off-Analysis
      - 1 Attributes & levels
    - - Good for low-involvement products
      - Choice-set with non-option
      - Estimation of partial utility values by multinomial logit model
- - - - Bagging
        
        Bias-Variance Decomposition
        
        Sampling training data from data set
      - Random Forests
      - Boosting
        
        Forward stagewise additive model
        
        Additive regression
      - Stacking
        
        Level 0 Models
        
        DT
        
        Naive Bayes
        
        Instance Bayes
        
        Level 1 Meta Learner
  - - - Probabilistic C. via
        Expectation Maximization (EM)
        
        Iterative
        
        Expectation Maximization
        
        Maximum Likelihood Estimator
        
        Extendable
        
        Multiple clusters
        
        Multiple attributes
        
        Nominal attributes
        (w/ prob. distr.)
      - Hierarchical C via
        Minimum Spanning Tree (MST)
        
        Top-Down
        
        Bottom-Up
        
        Kruskal algorithm
      - Partitional C. via
        K-Means
        
        Random number of cluster centers
        
        Iterative improvement
        
        1 Shifting to mean
        
        2 Re-assing items
        
        3 Repeat
        
        Distance measures
        
        Euclidian distance
        
        Manhatten distance (City-Block)
    - - Tables
      - Venn diagram
      - Cluster Dendogram
      - n-dim. Scatter plot
- - - - PC = Eigenvectors
      - PCA score = New coefficient on PC
      - Uncorrelated PC
      - Proportion of explained variance = Eigenvalue
    - - 1 Transform to zero mean data
      - 2 Calculate covariance matrix
        
        Coveriance
        (if same units & diff. important)
        
        Correlation matrix
        (if diff. units or diff. unimportant)
      - 3.1 Calculate Eigenvector & Eigenvalue
      - 3.2 Order Eigenvectors by Eigenvalue
      - 3.3 Rotate (Multiply data with Ordered Eigenvectors)
      - 4.1 Ignore PC with low significance
      - 4,2 Calculate only important EVs
      - 4.3 Rotate with important EV
      - 5 Reconstruct data
    - - Linear relationships
      - Continuous vars
    - - A = USV^t
        
        U S = Principal Components
        
        V = Principal Axes
        
        S = Singular Values = Eigenvalues of A A^t
  - - - Coefficient shrinkage
        (Regularization)
        
        Lasso
        
        Ridge regression
        
        Subset selection
      - Using derived input
        
        Principal Component Regression (PCR)
        
        Partial Least Squares (PLS)
    - - Multiple Linear Regression overfits data
      - Features > Observations
      - Multicollinearity
- - - - Support (supp)
      - Itemsets
        
        Item1
        
        Item2
        
        ...
      - Frequent itemsets (support >= minsupp)
      - Minimum support (minsupp)
      - Attributes -> Binary flags
    - - Rule support
      - Confidence
    - - 1 Find all frequent itemsets (>minsupp)
        [Build from subsets]
      - 2 Find rules with >minconf
        [First check simple rules, then build multi]
      - 3 Prune
  - - - Similarity weighting
      - Significance Weighting
      - Rating Prediction
      - SVD
    - - Product
      - User