AI
Rethinking Intelligence: Beyond AI Scaling Limits
Why aren't our largest language models getting proportionally smarter? A deep dive into intelligence, brain architecture, and AI's future.
Dr. Gareth Roberts
Nov 19, 2024•12 min read
TABLE OF CONTENTS
Although this is a contentious topic*, and there are plenty of reasons why they may have appeared to plateau in performance, it's a question worth exploring as it takes us deep into the fascinating world of brains, artificial intelligence, and what intelligence really means.For a long time, we've had the idea that bigger equals better when it comes to intelligence. It's not completely wrong – studies do show that brain volume correlates with IQ, though perhaps not as strongly as you might think (several meta-analyses place it around r = 0.3 - 0.4). But here's where it gets interesting: **if more neurons were all that mattered, whales and elephants would be writing philosophy books instead of us.** What about if we take into body mass? Well, we'd all be living in a world worshipping our shrew overlords.Think about this: your brain, that 1.4-kilogram marvel sitting in your skull, regularly outperforms much larger brains found in nature. The secret lies not in its size but in its **remarkably efficient architecture**. Recent discoveries in neuroscience have revealed several mechanisms that make our brains so effective in allowing us to perform intelligent behaviours:The prefrontal cortex, notably its lateral surface, contains neurons with an extraordinary property – they respond to **complex combinations of stimuli** rather than single features. These neurons create a high-dimensional representation of information, allowing the brain to encode complex cognitive tasks rapidly and with remarkable efficiency.These specialised cells, found in key brain regions like the anterior cingulate cortex and frontal insula, act as **the brain's express highways**. Multiple studies have shown a direct correlation between VEN density and fluid intelligence scores. These neurons are particularly fascinating because they enable rapid communication between distant brain regions.Your brain is remarkably energy-efficient, using **~20 watts of power**. One influential account for this efficiency is a Bayesian inference mechanisms that help predict incoming sensory information, minimising the energy needed for processing. This influential work largely spearheaded by Karl Friston and his predictive coding framework explains how minimising free-energy allows the brain to focus its limited energy on processing unexpected or important information, rather than wasting resources on the predictable.Here's something that might surprise you: when we talk about LLMs "knowing" things, we're not really talking about knowledge in the way humans understand it. Instead, these models are **incredibly sophisticated pattern-matching machines**. Think of them less like a wise sage and more like a savant who's memorised the statistical relationships between words in virtually every book ever written.This distinction leads to some fascinating paradoxes:Just as there's a limit to how many times you can fold a piece of paper, there might be a **natural limit to how complex language patterns can get**. Some researchers suggest we might be approaching this ceiling in current models.When training these models, we've discovered something counterintuitive: sometimes, **less data of higher quality works better than more data of lower quality**. Why? Because authentic human writing contains richer, more meaningful patterns than AI-generated text.While we've been obsessing over making AI models bigger, we might have been overlooking something crucial: **speed**. The relationship between processing speed and intelligence is one of the most robust findings in cognitive science, and it's far more sophisticated than you might think.Early studies in humans revealed a fascinating pattern: the correlation between reaction time and IQ **increased as experimenters made the tasks more complex**. In simple reaction time tasks, the correlation is modest. But in choice reaction-time tasks, where participants must choose between multiple options, the correlation increases significantly with each additional choice. This suggests that processing speed becomes increasingly crucial as cognitive demands rise.Even more intriguing are the results from **inspection time tasks**, which eliminate the motor component of processing speed entirely. In these tasks, participants simply have to discriminate between two briefly presented stimuli – no quick button-pressing required. The correlation between performance on these tasks and fluid intelligence is remarkably robust. This tells us something profound: it's not just about how quickly you can react; it's about how rapidly your brain can process information at its most fundamental level.These findings suggest that our current approach to AI might need a significant rethink. Rather than building ever-larger models, we might need a combination of different architectures:#### Fast, Lean Models
Small but incredibly fast inference engines optimised for basic logical operations and rapid pattern matching, similar to how our brain handles fundamental processing tasks#### Deep, Slower Models
Larger models for storing and processing complex patterns and knowledge, albeit with slower access times#### Integration Layers/Coordination Systems
A system for rapidly combining outputs from both types of models. I'm increasingly seeing rapid development at this level with frameworks such as LangChain, Semantic Kernel, crewAI, etc that are combining multiple LLMs, use of agents and tools, Knowledge Graphs, and sophisticated formal logic systems.Early experiments with this kind of hybrid architecture have shown promising results. Models combining fast, shallow networks for basic logical operations with deeper networks for complex pattern matching have demonstrated **superior performance on reasoning tasks** compared to single, large models (see the work of Zhang et al., 2023).This brings us to the current challenge in AI development. Today's large language models are like massive pattern-matching engines running on relatively slow hardware. While they can process vast amounts of text, they lack the brain's ability to quickly adapt and respond to new situations. Let's break this down with some specific examples:When you ask GPT-4 to solve a math problem, it doesn't actually do the computation – it tries to **pattern-match to similar problems it's seen before**. This works brilliantly for common problems but falls apart for novel ones. Recent studies show that even the largest models struggle with basic arithmetic when the numbers fall outside their training patterns. Most commercial models invoke Python or JavaScript functions by default, and if pressed to do it themselves will perform the calculation using heavily fine-tuned step-by-step mathematical reasoning routines.While modern AI can eventually arrive at impressive answers, it does so through a relatively **slow, sequential process** (awesome prompts are usually very large and force the model to decompose the prompt). If you need another example, look at the impressive reasoning abilities of the O1 model series (and how slow they are). In contrast, your brain processes information through massive parallel pathways, allowing for instant recognition and rapid problem-solving. Humans may take their time and be more cautious than models - but the underlying neural computations are fast.Analysis of recent language models shows that increasing model size has led to **diminishing returns in performance**. With some researchers suggesting we're hitting a "pattern entropy ceiling" – a point where additional training data only adds noise rather than meaningful patterns.Looking at current AI development, we might be going about this all wrong—rather like trying to build a bird by making increasingly powerful slingshots. Yes, you might eventually get something into the air, but you're rather **missing the elegant engineering that makes natural flight possible**. Just as the Wright brothers succeeded by studying birds rather than building bigger catapults, perhaps we need to take a closer look at nature's own intelligence systems.This is where a century of proper scientific grafting becomes invaluable. They give us a blueprint for rethinking AI development entirely. These insights, backed by decades of researchers poking about in brains (ethically, of course), might just hold the key to developing artificial systems that can actually think, not just play an extremely sophisticated game of pattern matching.Perhaps it's time to stop asking **"How can we make it bigger?"** and start asking **"How can we make it think more like that monkey that stole my glasses?"** After all, the future of AI might not lie in building ever-larger pattern matchers, but in understanding what makes a 1.4-kilogram chunk of energetically stingy tissue so remarkably good at adapting to whatever the world throws at it.And if we manage that, we might finally create AI systems that can do more than just semi-memorise the internet—they might actually be able to understand why that joke about the slingshot was funny.---
Size Isn't Everything: The Brain's Clever Design
Mixed Selectivity Neurons
Von Economo Neurons (VENs)
Bayesian Energy Conservation
The Pattern Problem: Why Our AI Might Be Hitting a Wall
The Pattern Ceiling
The Quality Conundrum
Speed of Processing: The Hidden Superpower of Intelligence
The Speed-Intelligence Connection
Implications for AI Architectures
Current AI: A Question of Architecture
The Pattern-Matching Trap
The Speed Bottleneck
The Training Data Challenge
Learning from Nature's Best Design
Notes
TAGGED WITH
AI
Intelligence
Neuroscience
Related Articles
Discussion
3 commentsJoin the Discussion
Dr. Sarah Chen2 days ago
Alex Morgan1 day ago
Dr. Michael Thompson3 days ago
Elena Rodriguez4 days ago