By Editorial Team Ten years ago, AI pioneer Geoffrey Hinton, with his grad students Alex Krizhevsky and Ilya Sutskever, revolutionized the world of artificial intelligence, with their seminal paper “ImageNet Classification with Deep Convolutional Neural Networks.” It proved a watershed moment for the field, triggering a decade of rapid innovations and world-changing applications. This week at NeurIPS, the world’s largest AI research conference, their paper was awarded the conference’s “Test of Time” award for its huge impact on the field. But in his keynote talk to an audience of thousands at the conference, Geoffrey made it clear that his work is not done, offering a vision for an entirely new form of computing built upon a new algorithm, Forward Forward. Radical Partner Aaron Brindle spoke with Geoffrey Hinton (who is an investor in Radical Ventures) after his talk. The following excerpt of their conversation was edited for length and clarity. Aaron Brindle (AB): So much of your work has been in the pursuit of better understanding the brain and how it works. Does Forward Forward bring you closer to that goal? Geoffrey Hinton (GH): That’s the hope. Although I may be the only person that thinks that. Forward Forward refers to two forward passes of data through the multi-layers of artificial neurons. One is real data and the other is negative data. I was inspired many years ago by this idea of contrastive learning and the powerful role negative data can play in tuning our capacity to build models of the world. With Forward Forward, this negative data can be run through the network when it’s offline – or asleep. Most people doing research on learning algorithms don’t think about sleep and may not wonder why we spend all that time sleeping. In the early 1980s, I was very focused on sleep and my work with Terry Sejnowski on Boltzmann Machines was inspired by the idea of what the brain is doing when it sleeps. AB: In the list of references in your paper you cite a Nature article from almost 40 years ago, on the function of dream sleep. Do the early results from Forward Forward provide further insight into the mystery of why we dream? GH: Why do dreams – which are always so interesting – just disappear? Francis Crick (who played an important role in deciphering the structure of DNA) and Graeme Mitchison had this idea that we dream in order to get rid of things that we tend to believe, but shouldn’t. This explains why you don’t remember dreams. Forward Forward builds on this idea of contrastive learning and processing real and negative data. The trick of Forward Forward, is you propagate activity forwards with real data and get one gradient. And then when you’re asleep, propagate activity forward again, starting from negative data with artificial data to get another gradient. Together, those two gradients accurately guide the weights in the neural network towards a better model of the world that produced the input data. AB: The Forward Forward algorithm does not rely on backpropagation, why is this important? GH: So the state-of-the-art in deep learning has been backpropagation, which works like this: You have an input which is maybe the pixels of an image, and you run a forward pass through the neural network and you get an output such as a categorization that says: “that’s a cat.” Then you do a backward pass through the network, sending information backwards through the same connections and change the neural activities to make the answer better or more accurate. If, for example, you input a picture of a cat and you go forward through the network and it says “that’s a dog,” backprop sends information backwards to tell the neurons how they ought to change their activities. So in future, when it sees that picture, it will say “cat” and it won’t say “dog.” The problem with backpropagation is it’s very hard to see how it would work in the brain. It interrupts what you’re doing. If I’m feeding you pictures very fast – like in a video – you don’t want to stop and run the network backwards in time which is what backpropagation would have to do. You want to just be able to keep going forwards as new data comes in. AB: What’s wrong with relying on backpropagation?