Please enable JavaScript.
Coggle requires JavaScript to display documents.
AI-book-2-Chapter-6 - Coggle Diagram
AI-book-2-Chapter-6
Deep Feedforward Networks
Names
Multilayer Perceptrons
Deep Feedforward Networks
Feedforward Neural Networks
Structure
Directed Acyclic Graph
Chaining of functions
Length of Chain = Depth of Model
Layers
Output Layer
Hidden Layers
Criteria
Activation Function
Architecture
Function Approximation Machines
Statistical Generalization
Features φ
Manually Engineered
Learn
Hybrid
Use human knowledge to constraint
Generic Infinite Dimensional
Gradient Based Learning
Non-Convex Optimization
Due to Non-Linearity
Parameters
Initialization
Sensitive to Initialization
Initialize to small random values
Used for Linear, SVM
When dataset is too large
Cost Functions
Types of model output
Probability
Actual Value
Maximum Likelihood
Minimize Cross-Entropy
No Minimum
KL Divergence
Log Likelihood
Regularization
Weight Decay
Gradient
Large Predictable Gradient
Avoid Saturation