Chasing 16MB: My Parameter Golf Journey and What I Learned the Hard Way

I saw what big companies and research labs were doing at massive scale and tried to adapt those ideas to extreme compression in tiny models. Here’s what happened. When OpenAI launched the Parameter Golf challenge, the rules were brutal: train a small language model that must fit inside a 16 megabyte compressed file and finish training in just 10 minutes on powerful hardware. Most participants focused on proven techniques that were already working on the leaderboard. I took a different appro...