John Hopfield and Geoffrey Hinton win physics Nobel for pioneering work in machine learning

This Tuesday, the Royal Swedish Academy of Sciences published its decision to award the 2024 Nobel Prize in Physics to Geoffrey Hinton and John Hopfield “for foundational discoveries and inventions that enable machine learning with artificial neural networks”. Hopfield and Hinton's work is the basis of machine learning as an independent field of study, and it underpins some of the most popular machine applications, including automatic translation, face recognition, and generative AI.

Inspired by human associative memory, John Hopfield developed a recurring neural network, the Hopfield network, that can store and recall data by treating it as patterns. The Hopfield network consists of a single layer of neurons connected to every other neuron but themselves. The neurons only have two states, active or inactive. To train a Hopfield network, the neurons are exposed to several different patterns, and to represent each of these patterns, the connections between any two neurons in the same state (active/inactive) are strengthened. In contrast, the connections between neurons in different states remain unchanged or diminish. Thus, the neurons update their connections to match the closest learned pattern when the neural network is fed a new, possibly faulty, or incomplete pattern.

Geoffrey Hinton worked on the idea that neural networks such as the Hopfield network could be layered and made to incorporate probabilities rather than binary states. As a result of his research, Hinton developed a system that could be trained on the probabilities encoding the likelihood that the system would encounter a certain data pattern in the training set. Once the system is trained, it can analyze novel patterns to classify them under given categories or create new instances of patterns it has frequently seen.

Notably, Hinton quit Google in 2023, citing the inability to speak freely about his concerns with the risks posed by AI as the reason for his departure.