Summary: A study demonstrates that learning in neural networks is driven primarily by adjusting the strength of existing connections rather than by continuously expanding or reconfiguring underlying network architecture.
The research team evaluated artificial neural networks trained on language-learning tasks across increasing volumes of training data. Despite marked improvements in performance, the trained networks maintained the capacity to lose a high proportion of synaptic connections without degrading functionality.
The findings suggest a unified computational principle across artificial and biological intelligence: cognitive learning and functional optimization rely on the refinement of existing synaptic weights and cooperative component dynamics rather than structural topological expansion.
Key Facts
- Weight Modification vs. Topological Expansion: Experimental results indicate that enhanced learning performance stems from adjusting internal connection strengths (synaptic weights) rather than adding new structural nodes or reconfiguring network topology.
- Network Pruning Resilience: Trained neural networks were able to withstand the loss of a significant proportion of synaptic connections without experiencing measurable performance degradation, demonstrating high functional redundancy and efficiency.
- Alignment with Biological Constraints: In adult biological brains, neuron counts remain approximately constant, and there is no established biological mechanism supporting continuous, large-scale structural rewiring of global network architecture during standard learning.
- Convergence of Biological and Artificial AI: The findings point toward a shared principle between biological brains and modern artificial neural networks: intelligence emerges from optimizing cooperative interaction among existing components.
- Computational Efficiency Implications: Demonstrating that architectural expansion is unnecessary for complex learning supports the development of more energy-efficient AI models and provides a physical model for biological cognitive reserve.
Source: Bar-Ilan University
How does the brain learn? Does it acquire new knowledge by creating new neural pathways, or by strengthening the connections that already exist?
A new study from Bar-Ilan University offers evidence in favor of the latter, suggesting that learning is driven primarily by changes in the strength of existing neural connections rather than by expanding the brain’s underlying architecture.
Published in Physica A, the study by Prof. Ido Kanter of Bar-Ilan University’s Department of Physics and the Gonda (Goldschmied) Multidisciplinary Brain Research Center explored this longstanding question using artificial neural networks trained on language-learning tasks.
As the amount of training data increased, the models became significantly better at learning. Surprisingly, however, the researchers found that the networks could still lose roughly the same proportion of connections (synapses) without any meaningful decline in performance. In other words, improved learning did not depend on building more complex networks. Instead, it resulted from more effective cooperation among the components that were already there.
The findings suggest that learning is achieved primarily by adjusting the strength of existing connections—known as synaptic weights—rather than by continually reorganizing or expanding the network itself.
“This finding is particularly intriguing in the context of biological brains, where the number of neurons remains approximately constant,” said Yanir Harel, an M.Sc. student at Bar-Ilan University and the study’s first author.
“Moreover, there is currently no evidence for a biological mechanism that would enable large-scale, continuous reconfiguration of neural network topology during learning.”
The research points to a possible common principle shared by biological and artificial intelligence: intelligence may emerge less from adding new components and more from improving how existing ones work together. If so, both the human brain and modern AI systems may owe much of their remarkable learning ability to the continual refinement of internal connections rather than to ever-expanding architectures.
Key Questions Answered:
A: The researchers trained artificial neural networks on language-learning tasks with expanding datasets and analyzed performance under synaptic removal. They observed that as models improved, they could still lose a large fraction of connections without performance dropping, proving that learning relied on refined connection strengths rather than adding architectural complexity.
A: In mature biological brains, the total number of neurons remains essentially fixed, and there is no evidence for large-scale, continuous structural rewiring of overall brain architecture during everyday learning. The study’s results support the biological reality that learning occurs through adjusting synaptic weights (synaptic plasticity) between established neurons.
A: Instead of endlessly scaling up network sizes and adding parameters, AI development can focus on optimizing how existing weights cooperate and prune unnecessary connections. This approach could lead to significantly more compact, energy-efficient, and computationally effective machine learning models.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- Journal paper reviewed in full.
- Additional context added by our staff.
About this learning and neuroscience research news
Author: Elana Oberlander
Source: Bar-Ilan University
Contact: Elana Oberlander – Bar-Ilan University
Image: The image is credited to Neuroscience News
Original Research: Open access.
“NLP models: Capacity per weight—capacity per label” by Yanir Harel, Tal Halevi, Ronit D. Gross, Shalom Rosner, Ido Kanter. Physica A: Statistical Mechanics and its Applications
DOI:10.1016/j.physa.2026.131855
Abstract
NLP models: Capacity per weight—capacity per label
The maximal capacity per weight (CpW) is finite for feedforward neural networks, suggesting that it may serve as a quantitative measure of the performance of natural language processing (NLP) tasks realized by enormous number of weights composing deep architectures.
Herein, simulations demonstrated that the CpW of a multilayer network approaches the one-step replica symmetry-breaking solution and remains independent of the number of hidden layers. However, CpW decreases as a power-law with the number of output units, calling into question its relevance to NLP tasks. Direct simulations of NLP pre-training tasks using a deep architecture revealed two main observations.
First, deep architectures can be heavily pruned without affecting the accuracy per token (APT). Second, the maximal pruning level that preserves APT is independent of the size of the training dataset containing a similar number of tokens, suggesting that CpW is not an appropriate measure of NLP task complexity. Similar observations are obtained for fine-tuning classification tasks.
To address this limitation, an alternative measure, capacity per label (CpL), is presented and its relevance is supported using a toy model based on archetype input vectors. A unique classification is assigned to all noisy patterns generated by each archetype.
Simulations showed that the maximal dilution preserving the accuracy of the undiluted model decreases with the number of output units, while remaining independent of the training dataset size and largely independent of the number of hidden layers.
Results are derived for BERT architectures pre-trained on tiny subsets of Wikipedia dataset and fine-tuned on the FewRel classification task.
