The Beatles' final AI-produced song snagged two Grammy nominations

Last year, Paul McCartney decided to use AI to clean up one of John Lennon's 1978 demos of 'Now and Then', which has since become the band's last song and (official) record. Inspired by the approach by which dialogue editor Emile de la Rey used AI to clean up the audio in archival footage from 1969 so it could be featured in Peter Jackson's documentary series "The Beatles: Get Back," McCartney decided to leverage AI to extract Lennon's voice from the 1978 'Now and Then' demo and transform it into a useable voice track for the final recording, which also features guitar recordings by George Harrison.

The approach used to clean up the archival material is similar to the technology that videoconferencing software leverages to suppress background noise in real-time. In the context of "The Beatles: Get Back", the team making the series consulted with Paris Smaragdis at the University of Chicago to create MAL, a neural network trained on high-quality audio data and snippets of the band's archival footage. First, the network was trained on generic clips of people talking, and sounds of individual instruments recorded by the documentary series' team members, and their friends and colleagues. Then, the dataset was expanded with footage fragments where The Beatles could be heard speaking individually or playing their instruments in isolation.

Eventually, the neural network was powerful enough that it could isolate the instruments and voice tracks from background noise, effectively breathing a new life into footage that would have been otherwise unsalvageable. MAL also enabled producer Giles Martin to make a new stereo mix of the band's 1966 album "Revolver."

The Record of the Year and Best Rock Performance nominations were possible because Paul McCartney's and Ringo Starr's contributions to the song count as newly recorded, enabling the band to break a record for the longest span between Grammy nominations, as the "Now and Then" nominations come 60 years after The Beatles received one such nomination in 1965 for "I Want To Hold Your Hand".