Experimental GPT-2 scale (~124M param) LLM trained from scratch on Google Colab. Trained on C4, Cosmopedia/Alpaca/Python mix. Includes full training pipeline, mixed dataset loader with Colab-resilient checkpointing, and log analysis tools. Honest write-up of what went wrong.
nlp pytorch transformer language-model nlp-machine-learning c4 google-colab gpt2 train-from-scratch llm bitsandbytes cosmopedia
-
Updated
Feb 19, 2026 - TypeScript