Please enable JavaScript.
Coggle requires JavaScript to display documents.
Deep Learning for Clusttering, Unsupervised Fine-tuning for Text…
Deep Learning for Clusttering
Model
FCN
CNN
AutoEncoder
To minimize reconstruction loss
Robustness: Adding noise
Restrictions on latent features
Architecture
DCN: AutoEncoder + KMeans (for clustering loss)
DEN
Deep Subspace Clustering
GAN/VAE
CDNN
Core concept: only clustering loss for training
Initialize methods to avoid overfitting
Architectures
Deep Embedded Clustering
Pretrain an autoencoder using reconstruction loss
Train the encoder using the cluster assignment hardening loss
VAE-based Deep Clustering
Loss functions
Principal Clustering Loss
Auxiliary Clustering Loss
Metrics
ACC
Neutralized Mutual Information
Training strategy
Loss = Loss(network, clustering)
Unsupervised Fine-tuning for Text Clustering
settings
Text data set X, number_samples=n
Number of clusters=K; centers are μ
encoder f(θ): making turning X to Z for better clustering properties
Trainables: f(θ) and μ
Loss function = Loss(masked language model loss, clustering loss)
Lm: BERT
Lc: KL-Divergence
Average pooling of all hidden states
More details in section 2.2
Training
Metrics: clustering purity (Manning et al., 2008)
Hyerparameters: section 3.2