A neural network starts out knowing nothing — its internal parameters set randomly, its first guesses close to worthless. What turns that random starting point into something that can recognise a face or translate a sentence is a remarkably simple idea, repeated an enormous number of times: check how wrong the current guess was, and nudge every parameter a small amount in whichever direction would have made that guess slightly less wrong.
Working backward from the mistake
A network makes a prediction, and that prediction is compared against the correct answer using a loss function — a single number measuring how wrong the guess was. Backpropagation is the algorithm that calculates, for every individual parameter in the network — there can be billions of them — exactly how much that specific parameter contributed to the current error. It works backward, layer by layer, from the output toward the input, using calculus (specifically, the chain rule) to assign blame precisely rather than guessing at it.
Gradient descent: the actual nudge
Once the network knows how much each parameter contributed to the error, and in which direction, gradient descent is the step that actually adjusts every parameter a small amount, opposite to the direction that increases error, so the next guess on similar input should be slightly better. Repeat this cycle — guess, measure error, calculate blame, nudge — millions or billions of times across enormous amounts of training data, and a network that started out random gradually shapes itself into one whose parameters produce useful, accurate predictions.
What we're still unsure about
Backpropagation and gradient descent explain mechanically how a network's parameters get adjusted, but they don't fully explain why the resulting networks generalise as well as they often do to data they've never seen. A large enough network has, in principle, enough parameters to simply memorise its training data rather than learn any genuinely useful pattern — and yet in practice many networks generalise well anyway. Exactly why that happens, and under what conditions it stops happening, is still an active, unresolved area of machine learning theory, not a settled mathematical result.
This sits inside Backpropagation & Gradient Descent, one of eight topics in AI/ML, one of seven domains in Computer Science, one of seventeen subjects the app can quiz you on.