Microsoft co-founder Bill Gates has been a key player in technological innovation for the last 40 years. Even though he no longer holds a management role at Microsoft, the billionaire remains at the forefront of innovation by sharing his insights on advances such as artificial intelligence, which he has described as a turning point.
A stochastic parrot ahead of its time. In a recent interview on the Next Big Idea Club podcast, Gates discussed the future of AI and its rapid evolution. He also expressed his excitement about the transformative potential of AI in various fields.
However, Gates also pointed out that current AI models like ChatGPT or Llama lack sophistication due to their non-deterministic behavior. This means that they won’t do anything unexpected and only do things they’ve been specifically taught, essentially functioning as repetitive “stochastic parrots” trained on millions of data points.
Metacognition is the future. Despite the significant advancements current AI models brings in terms of process automation, Gates acknowledges that these models lack the full dimensions of thinking that humans possess. As such, he believes the next frontier that will be crossed is “metacognition,” referring to a system that’s aware of its own thinking.
The self-awareness of their own knowledge is what enables humans to acquire knowledge and apply it in various ways to arrive at different results, rather than consistently reaching the same “learned” conclusion.
Instead of simply imitating knowledge, the goal is to learn to apply it. According to Gates, metacognition is “the Holy Grail” for AI to be much more than just an assistant that enhances our capabilities. In his eyes, it would make us “superhuman.”
The Microsoft co-founder also points out that current AI models focus on “generating through constant computation each token in sequence, and it’s mind-boggling that that works at all.” However, he believes that the current LLM cognitive strategy should “think about a problem in a broad sense.”
“[The current LLM cognitive strategy] does not step back like a human and think, ‘Okay, I’m going to write this paper and here’s what I want to cover. Okay, I’ll put some facts in. Here’s what I want to do for the summary…,’” Gates said during the interview.
A couple of evolutions more and we’ll have a problem. Gates believes that current AI models will continue to evolve and show a significant improvement in their answers. At the same time, he warns that this exponential improvement may lead to an overfitting of training data and computational power and won’t focus on the key issue: the reliability and accuracy of these models.
Gates envisions a future where AI possesses “metacognitive” abilities, enabling it to assess the importance of a task, verify its answers, and utilize external tools when necessary. This, according to Gates, is the “big frontier” for artificial general intelligence.
The “Aristotelization” of AI. In other words, according to Gates, artificial intelligence should be capable of comprehensively breaking down a problem, considering all its variables, and then reconstructing it to provide a new creative solution. This solution should be based on certainties rather than hallucinations and not necessarily taught to the AI.
This process resembles Aristotle’s approach to problem-solving through “first principles thinking.” It's a method of analysis that Elon Musk and Jeff Bezos have also employed to solve previously unsolved problems.
This article was written by Rubén Andrés and originally published in Spanish on Xataka.
Image | Viralyft via Unsplash | Wikimedia Commons (Lukasz Kobus)
View 0 comments