EEG 2D representation
Some approaches
Other input representations, e.g. transformed representations such as time-frequency decomposition, generally increase data dimensionality requiring more training data and/or regularization to learn meaningful features.
TF map using Source signals Mammone2020
Topographic maps (interpolations). Bashivan2016 Fadel2020 Collazos2020 Qiao2020
Spectogram Shajil2020
2D EEG Meshes Zhang2018
EEG signals as input to CNNs is to design a 2D input representation with the electrodes along one dimension and time steps along the other preserving the original EEG representation i.e. non-transformed representation. Farahat2019
CNNs are typically designed by stacking individual temporal and spatial convolutional layers or a single spatio-temporal convolutional layer, and eventually deeper convolutional layers that learn patterns on the filtered activations. Borra2020
CNNs do not need any a priori knowledge about the meaningful characteristics of the signals for the specific decoding task and have the potentiality to discover the relevant features by using all input information.
Transfer Learning
CNN Interpretability
Several efforts have been made to increase CNN interpretability via post-hoc interpretation techniques (i.e. techniques that analyse the trained model)
These techniques include temporal and spatial kernel visualizations Xu2020
Kernel ablation tests (i.e. selective removal of single
kernels)Lawhern2018EEGnet
Saliency maps (i.e. maps showing
the gradient of CNN prediction with respect to its input example) Farahat2019 - Evaluation Alqaraawi2020
Gradient-weighted class activation mapping Selvaraju2019
Correlation maps between input features and
outputs of given layers Liao2020
Other Architectures: Mane2020, Lawhern2018EEGnet
Saliency maps variant Borra2020
Transferring knowledge from one subject to another deteriorates the classification accuracy. For this reason, most of the studies usually perform intra-subject classification.
DL Interpretation for sequential data Shickel2020
However due to time-consuming calibration and re-training sessions, it’s been always a priority for BCI systems to transfer the knowledge learned from multiple subjects to the new target subject
Inter-subject transfer learning techniques as classification strategy Fahimi2020
Transfer learning using pre-trained models as the starting point (vgg19, alexnet, vgg16, googlenet, squeezenet, resnet50, googlenet, densenet201, resnet18, resnet101). Kant2020
Instance transfer subject-independent (ITSD) framework combined with a convolutional neural network (CNN) Zhang2020
Firstly, an instance transfer learning based on the perceptive Hash algorithm is proposed to measure similarity of spectrogram EEG signals between different subjects. Then, we develop a CNN to decode these signals after instance transfer learning. (Spectograms)
To deal with the EEG individualdifferences problem, transfer learning technique is implemented to fine-tune the followed fully connected (FC)layer to accommodate new subject with fewer training data. Zhang2021
The aim of the paper is to examine if the transfer learning approach allows to achieve higher classification performance even in the case of inexperienced BCI users who were never previously trained to generate motor imagery patterns in EEG signals. Saeed2020
In this paper, authors propose a novel privacy-preserving DL architecture named federated transfer learning (FTL) for EEG classification that is based on the federated learning framework. Ju2020
Working with the single-trial covariance matrix, the proposed architecture extracts common discriminative information from multi-subject EEG data with the help of domain adaptation techniques
Transfer Learning for EEG-Based Brain-Computer
Interfaces: A Review of Progress Made Since 2016. Wu2020
In one approach, the network learns a general model based on the data from a pool of subjects. Then, it transfers the knowledge to a new subject. In a more adaptive approach, the model will be updated based on a subset of new subject’s samples. In this way, the problems of time-consuming re-trainings and low intersubject
Domain adaptation methods to alleviate
the discrepancy across data from source and targe
A measure for joint distribution
discrepancy (JDD) between the source and target is proposed in Zhao2020
To overcome this limitation, several groups introduced machine learning, especially transfer learning methods for adapting BCIs to target subjects
Several groups have started explicitly modelling such variations to exploit the common structure that is shared between multiple subjects
Data from source subjects, in order to regularize common spatial patterns (CSP), ultimately to make the estimation of covariance matrix more unbiased and filters more effective for target subjects
Other works constructed filter bank to extract more abundant features, selected them according to some designed rules, and then ensembled them to obtain high performance
There are also researchers transforming features from different subjects into another space and making them more similar
It can be added to
the deep neural network as an effective regularized part
Recent studies reveal that a deep neural network with domain adaptation technique can learn both deeper and more transferable features
In Tzeng2014 a DDC model that adds an adaptation layer and a dataset shift loss to the deep CNN for learning a domain-invariant representation. While the performance was improved, DDC only adapts a single layer of the network
In Long2018
Multilevel features are matched utilizing multiple kernel MMD. Long et al. also exploited a better way to reduce the computation cost for MMD and obtained a better result
Jian et al. proposed an adversarial representation learning approach to learn highlevel representations that are both domain-invariant and target-discriminative, in order to tackle the cross-domain classification problem.
FOR BCI TASKS
Fahimi et al. in Fahimi2020 developed an end-to-end deep CNN to decode the attentional information from EEG time series. They also explored the consequence of input representations on the performance of deep CNN by feeding three different EEG representations into the network. Additionally, intersubject transfer learning techniques were performed as a classification strategy.
Farshchian et al. in Farshchian2019 implemented various domain adaptation methods to stabilize the interface over significantly long time, including canonical correlation analysis, minimizing the Kullback-Leibler divergence of the empirical probability distributions. These two methods provided a significant and comparable improvement in the performance of the interface
Tan et al. in Tan2018 modeled cognitive events by characterizing the data using EEG optical flow, which is designed to preserve multimodal EEG information in a uniform representation. After that, a deep transfer learning framework, which was suitable for transferring knowledge by joint training, was constructed. It contained an adversarial network and a special loss was designed.