In 2012 a machine learned to tell a cat from a dog about as well as a careless person.
A decade later, machines write, translate, prove, and predict the shape of nearly every known protein.
We can build these systems far better than we can explain them. That is the problem worth our best minds.
For fifty years, artificial intelligence meant writing rules by hand. The breakthrough came from giving that up. A neural network is a large pile of adjustable numbers; show it examples, measure its error, and nudge every number a little in the direction that reduces the error. Repeat a few billion times.
On the standard image-recognition test, error fell from 28% in 2010 to under 3% in 2017, past the level of a trained human. The same recipe then learned speech, translation, the game of Go, and language itself. Nobody designed those abilities. They were found by the training process.
Seven years took the best system from clearly worse than a person to clearly better.
Source: ImageNet Large Scale Visual Recognition Challenge results, 2010 to 2017; Russakovsky et al. (2015) for human performance · cited from memory, to verify
There is no step where anyone tells the system the rule. It guesses, is told how wrong it was, and every one of its numbers is nudged slightly toward a better guess. The model lower on this page does exactly this, a few hundred times a second, in front of you.
Source: Rumelhart, Hinton and Williams, Learning representations by back-propagating errors, Nature (1986) · cited from memory, to verify
The ideas behind today's systems are mostly old. What changed is the amount of computation spent training them, which has doubled roughly every six months since 2010. That is far faster than chips improve; the rest came from building bigger machines and spending more money.
Scale buys capability, and it also buys a problem. A system trained this way holds its knowledge in billions of numbers that no person wrote and no person can read. We can test what it does. We are only beginning to be able to see why.
From AlexNet to GPT-4: about fifty million times more computation in eleven years.
Source: Epoch AI, Notable AI Models database; Sevilla et al., Compute Trends Across Three Eras of Machine Learning (2022) · cited from memory, to verify
Each mark is ten times the one before. The adjustable numbers in a network are loosely comparable to the connections between neurons, though a synapse does far more than store one number. On that rough measure the largest models are still several hundred times smaller than one human brain.
Source: LeCun et al. (1998); Krizhevsky et al. (2012); OpenAI (2019, 2020); Azevedo et al. (2009) for the brain · cited from memory, to verify
These systems state falsehoods with the same fluency as facts. They can be argued into things they should refuse. They need far more examples than a child does, and far more energy than a brain, which runs on about twenty watts.
Underneath those is a deeper gap. We do not have a science of what a trained network has learned, in the way we have a science of bridges. Without one, we cannot give guarantees: we can only test, and hope the tests covered the case that matters.
This is why Dhi exists. It sets the machinery of modern AI beside the machinery of the brain and is honest about which resemblances are real.
The brain does all of its learning, perceiving and reasoning on the power of a dim light bulb, from a few years of ordinary experience. Our best systems need a power station and most of the written record of humanity. Whatever the brain is doing, we have not yet found it. That gap is the subject of Dhi.
Source: Raichle and Gusnard (2002) for brain power; Azevedo et al. (2009) for neuron counts; training power is an order-of-magnitude estimate · cited from memory, to verify
We want AI that accelerates discovery: systems that read the literature no person can keep up with, propose experiments, and check each other's reasoning. The clearest proof that this is possible is protein structure, where a learned model turned a problem of decades into a lookup.
And we want it understood. Our commitments are to study these systems as natural objects, to publish what we find about how they work and fail, and to label our own AI plainly wherever it speaks. The research companion in the Agora already follows that rule: it is marked as AI, it cites only what it was given, and it says when it does not know.
Sixty years of experiments determined about 200,000 structures. A learned model predicted over 200 million.
Source: AlphaFold Protein Structure Database (2022 release); RCSB Protein Data Bank statistics · cited from memory, to verify
Filled and hollow dots are two kinds of thing. The shading shows what the network currently believes about every point on the plane. It starts knowing nothing and improves only by being corrected.
This is a statement of intent, not a record of achievement. Each step carries a label for how firm it is. Nothing marked proposed or speculative has been done yet.
Already live: an honest side-by-side of brains and machines, and an Agora assistant that is labelled, cites only what it is given, and admits uncertainty.
Explorable maps of the internal features of real models, written for researchers outside the large laboratories.
Systems that check a claim against its cited papers and flag what the evidence does not support, with every judgement open to challenge.
Systems that propose and help run experiments in the other five fields of this plan, with humans accountable for the result.
A working science of learned systems, so that guarantees can be given and not merely hoped for.
Every claim on this page is open to challenge in the Agora, in the Mind & Intelligence room.
Argue it in the Agora ↗