Please enable JavaScript.
Coggle requires JavaScript to display documents.
Sequence Modeling - Coggle Diagram
Sequence Modeling
-
Neurons with Recurrence
-
Sequentially update: capture the sequence and iteratively update from the observation in the sequence data
Output (y hat) = Input(Xt), Past memory(ht-1)
-
-
Design Criteria
- handle variable-length sequences (flexible of the model)
- track long-term dependencies (prior input have an impact to future prediction)
- maintain information about order
- share parameters across the sequence (updating)
-
-
-
-
-
Self-Attention with NN
- Encode position information
- Extract query, key, value for search
- Compute attention weighting