Training

July 22, 2025
in Training, Text Diffusion, Mode Collapse, Regularization, Learning Rate Scheduling
5 min read

Mode Collapse Prevention: Comprehensive Training Improvements

Status: 🟡 In Progress - Training Deployed
Type: Implementation + Training Experiment

Objective

Address the catastrophic mode collapse discovered in the 1000-epoch training run, where the model generated only punctuation marks instead of coherent Shakespeare-style text. Implement comprehensive training improvements to prevent token frequency exploitation and ensure stable, quality text generation.

July 21, 2025
in Implementation, Text Diffusion, Diffusion-LM, Training
3 min read

Learned Rounding Function & Custom Embeddings Implementation

Status: ✅ Complete - Full Pipeline Validated
Type: Implementation + Experiment

Objective

Implement and validate the learned rounding function and custom embedding space improvements identified in our Diffusion-LM analysis, replacing cosine similarity decoding with trainable components for improved text generation quality.