4. We’re Role Models For Agents to Self-improve
Language models are hitting a data wall, forcing an engineering pivot from raw internet scraping to internal reasoning. Instead of relying on passive text intake, the incoming wave belongs to systems that flag their own logical inconsistencies, allowing models to self-improve without extra data inputs.
This means the technical race is no longer about the size of the crawl, but the aggressiveness of the filter. Forcing models to read unvetted text stalls their reasoning capabilities.
“Would you teach your child to read the diaries of serial killers?” asked computer scientist Geoffrey Hinton during a fireside chat with Hellermark on why labs must drop ubiquitous scaling. “We should curate that data a whole lot so you model decent behavior to it.”
Matching human capability also requires updating the core math behind digital computing architectures. Standard digital setups force transformers to freeze their weights during a sequence, a constraint that blocks real-time learning. To truly step up, hardware must mimic biological synapses that adapt and decay instantly.
“What they're missing at present is fast weights,” Hinton noted when placing his final bet on the future of neural nets. “Our brain manages to compete with transformers ... by having fast changes to the weights so that in that weight matrix, you've stored lots of information about recent history efficiently.”