Please enable JavaScript.
Coggle requires JavaScript to display documents.
Transformers的变体 (多模态方向 (LXMERT, VideoBERT, CBT, VLP, UNITER, LXMERT,…
Transformers的变体
多模态方向
LXMERT
VideoBERT
CBT
VLP
UNITER
LXMERT
VisualBERT
B2T2
VL-BERT
Unicoder-VL
ViLBERT
计算量过高怎么办？
Reformer
Span Transformer
Augmenting Self-attention with Persistent Memory
Large Memory Layers with Product Keys
Star-Transformer
BP-Transformer: Modelling Long-Range Context via Binary Partitioning
对超长文本不友好怎么办？
Transformer-XL
Adaptively Sparse Transformers
Incremental Transformer with Deliberation Decoder for Document Grounded Conversations
Compressive Transformer
Mogrifier LSTM (额外门控LSTM)
内存占用较大怎么办？
Masked Block Self-Attention
ALBERT
Reformer
Span Transformer
绝对位置信息如何改？
Transformer-XL
Self-Attention with Relative Position Representations
Self-Attention with Structural Position Representations
TENER: Adapting Transformer Encoder for Named Entity Recognition
如何达到稀疏注意力？
Generating Long Sequences with Sparse Transformers
Adaptively Sparse Transformers
Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
没了RNN从左到右的归纳偏置怎么办？
Universial Transformer
如何应用到与顺序无关的任务上？
Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks