Transformer Attention Is Hopfield's 1982 Update Rule (And What That Tells Us About LLM Memory)

Hopfield's associative-memory equation from 1982 and the scaled dot-product attention from Vaswani 2017 are the same operation. One substitution turns one into the other. The 2024 Nobel Prize in Physics — to Hopfield and Hinton — is the academic acknowledgement that the mathematics behind today's LLMs was already written four decades ago, in a different vocabulary. This is a condensed write-up of the longer, interactive piece at ki-mathias.de/en/hopfield.html. Seven chapters there, five live M...