LSTM (Mini-Course (Foundations (Lesson 3: How to prepare data for LSTMs…
Lesson 1: What are LSTMs
Recurrent Neural Networks
How to Train Recurrent Neural Networks:BPTT.
How to Have Stable Gradients During Training:LSTM
[samples, time steps, features]
Lesson 2: How LSTMs are trained
Backpropagation Through Time
Truncated Backpropagation Through Time:TBPTT(k1,k2)
Prepare Sequence Prediction for TBPTT
How to Seed State for LSTMs
Stateful and Stateless LSTM ：
resetting/no-resetting and seeding/no-seeding
Lesson 3: How to prepare data for LSTMs
Normalize Series Data:scikit-learn MinMaxScaler.
Standardize Series Data:scikit-learn StandardScaler
One Hot Encode
Manual One Hot Encoding
One Hot Encode with scikit-learn: LabelEncoder and OneHotEncoder
One Hot Encode with Keras: to_categorical()
Handle Missing Timesteps
Remove Missing Sequence Data: dropna() Pandas
Replace Missing Sequence Data: fillna() Pandas
Mark Missing Values: replace() Pandas
Masking Missing Values:using a Masking layer
Impute Missing Values: Imputer() scikit-learn
Convert a Time Series to a Supervised
The series_to_supervised() Function:Pandas shift() Function
Handle Very Long Sequences with
Use Sequences As-Is:A reasonable limit of 250-500 time steps
Use Truncated Backpropagation Through Time
Use an Encoder-Decoder Architecture
Data Preparation for Variable Length Input
Sequence Padding: pad_sequences() function in the Keras
Lesson 4: How to develop LSTMs in Keras
5 Step Life-Cycle:
Lesson 5: How to develop Vanilla LSTMs
combine CNN with LSTM
Time Series Prediction
LSTM Network for Regression
LSTM for Regression Using the Window Method
LSTM for Regression with Time Steps
LSTM with Memory Between Batches
Stacked LSTMs with Memory Between Batches
Time Series Forecasting
Transform the dataset to make it suitable for the LSTM model, including:
1.Transforming the data to a supervised learning problem.
2.Transforming the data to be stationary.
3.Transforming the data so that it has the scale -1 to 1.
Fitting a stateful LSTM network model to the training data.
Evaluating the static LSTM model on the test data.
Report the performance of the forecasts.
Lesson 6: How to develop Stacked LSTMs
Lesson 7: How to develop CNN LSTMs
Lesson 8: How to develop Encoder-Decoder LSTMs
How to Use the TimeDistributed Layer
The input must be (at least) 3D
The output will be 3D
keeping the internal process for each time step separate.
Simplifies the network by requiring far fewer weights
Add Numbers with an Encoder-Decoder LSTM
use an Encoder-Decoder LSTM
Sequence Echo Problem
Generate Random Sequence: randint() function
One Hot Encode Random Sequence: one_hot_encode()
one_hot_decode(): argmax() NumPy function
converting sequences to supervised:
Pandas shift() function dropna() function.
Echo Whole Sequence (sequence-to-sequence model)
Echo Partial Sequence (encoder-decoder model)
Lesson 9: How to develop Bi-directional LSTMs
Compare LSTM to Bidirectional LSTM
LSTM with reversed input sequences (go_backward=True)
Comparing Bidirectional LSTM Merge Modes
Lesson 10: How to develop LSTMs with Attention
Attention is the idea of freeing the encoder-decoder architecture from the fixed-length internal representation.
Lesson 11: How to develop Generative LSTMs
Lesson 12: How to tune LSTM hyperparameters
Evaluate the Skill of Deep Learning Models
Estimating Model Skill
(Controlling for Model Variance)
Use a Train-Test Split
Use k-Fold Cross Validation
Estimating a Stochastic Model’s Skill
(Controlling for Model Stability)
Fix the Random Seed
Repeat Evaluation Experiments
Tune LSTM Hyperparameters
Tuning the Number of Epochs
Tuning the Batch Size
Tuning the Number of Neurons
Grid Search Hyperparameters
GridSearchCV class in the scikit-learn
Improve Deep Learning Performance
1.Improve Performance With Data.
Get More Data.
Invent More Data.
Rescale Your Data.
Transform Your Data.
2.Improve Performance With Algorithms.
Steal From Literature.
3.Improve Performance With Algorithm Tuning.
Batches and Epochs.
Optimization and Loss.
4.Improve Performance With Ensembles.