Please enable JavaScript.
Coggle requires JavaScript to display documents.
PTMs (Extensions (Model Compression (Pruning (CompressingBERT),…
PTMs
Extensions
Multi-Lingual
Multi BERT, MASS, Unicoder, XLM, MultiFit
Multi-Modal
Image
ViLBERT, LXMERT, VisualBERT, B2T2, VL-BERT
-
-
Knowledge-Enriched
ERNIE, KnowBERT, K-BERT, KEPLER, WKLM
Domain-Specific
BioBERT, SciBERT, PatentBERT
Language-Specific
BERT-wwm, ZEN, NEZHA, ERNIE, BERTje, CamemBERT, FlauBERT, RobBERT
Model Compression
-
-
-
Distilation
DistiBERT, TyniBERT, MiniLM
Architecture
-
Transformer Enc.
BERT, SpanBERT, XLNet, RoBERTa
-
-
-
Task Types
-
self-Supervised
-
MLM
BERT, SpanBERT, XLM, MASS, T5
-
-