Running 35B–400B LLMs on a GPU-less Cluster to Mine 10,000 Papers — and the 4 Bugs That Almost Ruined the Data

In a fascinating experiment, a team tackled the challenge of running large language models (LLMs) on an old cluster without GPUs, relying solely on CPU power. They employed significant quantization techniques to process up to 400 billion parameters across the system. While the project aimed to extract valuable scientific data from 10,000 papers, it encountered four critical bugs that nearly compromised the data's integrity. These bugs underscored the importance of rigorous testing in complex computational environments and highlighted the potential of CPU-only setups in large-scale data processing.