Now that we can no longer ignore AI, it is easy to forget how long and winding the road to artificial intelligence was. Experts draw from this a ‘bitter lesson‘: all their carefully constructed theories have been washed away by a tidal wave of data and brute computing power. If a computer is strong enough and is fed enough information, intelligence seems to arise spontaneously. Was it hubris to want to capture the complexity of our thinking in diagrams and models?
The magnitude of this double tidal wave is difficult to imagine. Thanks to Moore’s Law, which predicts that the number of components in a microchip will double every two years, computers are now a billion times more powerful than at the time of the moon landing. Nvidia’s latest GPU chips – the new gold of the AI economy – contain more than 300 billion transistors. The largest data centers link hundreds of thousands of these chips together.
The data flow is also growing exponentially. The Internet now contains about 100 trillion words, and the billions of videos on YouTube provide a similar amount of information. It would take a human being at least a hundred thousand years to read and watch all this.
But the question is whether this strategy can go much further. How much smaller can ASML make the structures on microchips within the limitations of physics? How much more data can we produce and process? The learning curve of current AI models appears to be flattening. Is the revolution reaching its natural limits?
Perhaps this is the time to shift our focus from the gigantic data centers to the playground. Because we humans can, in turn, teach AI something: the ‘sweet lesson’ of toddlers and preschoolers. While AI requires billions of data points, young children are incredibly good at learning from just a few examples.
A colleague told how his daughter saw a giraffe for the first time at the zoo. The next day they visited the natural history museum. In a hall full of skeletons, the girl pointed to one and said: “Giraffe!” A single data point was enough for her to recognize the pattern, while an AI model must first scan millions of images to discover the same connection.
What can preschoolers do that AI cannot? To do this, we need to dive deeper into how humans learn. Developmental psychologists talk about the exploration-exploitation dilemma. There are roughly two strategies for acquiring knowledge. The first method is exploration: freely investigating the environment, primarily driven by curiosity. The second method is exploitation: applying and refining already acquired knowledge, often with a practical goal in mind.
The question is what the right balance is between these two strategies. Think of a learning process as the search for the best place to pitch your tent. Do you first have to extensively map the area without the risk of getting lost? Or make a quick choice with the risk of a suboptimal location?
Young children behave like sophisticated mini-researchers
Evolution has found an elegant solution to this dilemma. We spend our first years mainly on exploration. Compared to other animal species, humans have a very long childhood and deal with it ‘sloppily’. Instead of first learning essential survival strategies like many young animals, we spend our childhood playing and exploring. In this way we discover all kinds of connections of which we do not know in advance whether they are useful. Only around the seventh year do we move to the exploitation phase. Useful patterns are ingrained and deepened, such as in language and arithmetic lessons.
Research shows that young children do not approach this exploratory phase haphazardly, but behave like sophisticated mini-researchers. They have been fitting since their first year of life simple version of the scientific method. Toddlers have intuitive theories about how the world works, for example about mechanics or gravity. They adjust them when the facts give reason to do so, just as scientists revise their hypotheses. And they investigate further if results are unclear, just as you design experiments to isolate variables.
Preschoolers are also better than adults at inferring complex or surprising causal relationships. They easily ignore obvious explanations and keep trying different options. Older children and adults, on the other hand, are more likely to cling to their prejudices and stop investigating sooner. For example, young children like to think of new uses for tools – something parents experience to their horror when their toddler gets their hands on a hammer.
That long period of exploration during childhood seems necessary for the formation of our complex intelligence. But crucially, we don’t do this by immersing ourselves in an ocean of data like AI models. There simply isn’t time to watch a hundred thousand years of videos. We explore the world with a focused strategy. It seems like something of the scientific method is already baked into the design of our brains. This offers hope for an approach to AI that ignores even more raw computing power and data.
But this lesson isn’t just for AI. Adults can also learn something from young children. If there is one thing we are good at, it is dealing with the unknown. Not bathing in a sea of information, but wandering around in a wide landscape with a stream of data here and there. This inner explorer, with which we are all born, deserves to be cherished. In the battle with machines devouring billions of data points, the ability to explore the unknown with almost nothing is our greatest asset.
The journalistic principles of NRC