Human intelligence involves many dimensions: we interact socially, learn quickly from other people, and determine how tasks fit together, as well as physically interact with and intervene in the real world.
“The way cognitive scientists think about intelligence is much broader than just solving tasks,” says SFI Professor Melanie Mitchell. In contrast, AI companies define intelligence as decomposable into small tasks, accounting for only a sliver of the full range of natural ability.
That narrow definition is an issue in light of AI companies’ prevailing narrative that “human innovation — which got us to the moon, which has given us antibiotics and Middlemarch — will be done just as well by AI,” says SFI External Professor John Krakauer, a professor of neurology and neuroscience at the Johns Hopkins University School of Medicine.
To explore this disconnect, Mitchell and Krakauer convened an SFI working group, “The Nature of Intelligence: Cognitive Science Perspectives on AGI,” on March 31–April 2. It was the third of six in-person meetings of leaders on the Nature of Intelligence project, funded by a gift from the Wicklow Foundation.
The participants, who have expertise in fields such as neuroscience, psychology, philosophy, and artificial intelligence, are all investigating the nature of intelligence, whether in biological systems or machines.
The working group explored the idea of artificial general intelligence from the perspective of cognitive science. The concept of AGI lacks an established, agreed-upon definition, which will make it difficult to know if and when it is ever achieved. AI companies track performance on benchmarks based on human achievements, some of which derive from the logic of IQ tests.
In contrast, cognitive science avoids an overarching definition of intelligence in favor of comparative experiments. Objective, quantitative measures of intelligence, such as IQ, have been disfavored as accounting for only a sliver of natural abilities.
The group set out to explore the definitions of AGI proposed by AI companies, whether those definitions were coherent, and the extent to which they overlap with how cognitive scientists think about intelligence.
While they sound impressive, benchmark-based achievements often don’t hold up in the real world, Mitchell says. Krakauer notes that open-ended, curiosity-driven activities such as art and science are particularly difficult to reduce to task-based problem solving.
Other themes the group explored include the link between language and intelligence and the existential threats that AI poses, ranging from near-term threats such as disinformation to speculative threats, such as AI systems developing their own desires and goals.
The group plans to synthesize its knowledge in a position paper. While they disagreed on some points, Mitchell says, “we agreed that the conception of intelligence that comes out in many discussions of AGI is not one that people who study natural intelligence necessarily focus on.”