Piling on Tasks Made Artificial Networks Split Into Modules

A team at Zhejiang University and the University of Electronic Science and Technology of China reports in Nature Machine Intelligence that the demands of learning many tasks can push artificial neural networks into a modular structure, an organization usually explained by the physical cost of wiring. A modular network is one whose parts do distinguishable jobs.

Their paper states that earlier work holds that modular structure emerges from physical constraints, such as minimizing the metabolic cost of long wiring, and that such spatial models alone do not fully explain how brain networks are functionally organized. The team says one question remains open: how the demands of learning complex tasks shape a network's structure.

The authors report that training a recurrent network on several tasks produced more modularity than training it on one, and that the effect was strongest when the number of tasks strained what the network could hold. Networks given their tasks one at a time, which the paper calls incremental multitask learning, became the most modular of all and also performed best.

The team compared the trained networks with a map of the physical connections between 84 areas of the human cortex, drawn from the Human Connectome Project, and reports that the task-trained networks matched that biological pattern more closely than networks shaped only by wiring cost. These are artificial networks, not brains: the study is a simulation, and the authors describe it as a controlled computational account. Their own reading, that modular structure is an adaptive response to being given complex tasks one after another, is offered in the paper as a suggestion.

Yuhang Wu is the first author; Shi Gu and Gang Pan, both of Zhejiang University, are the corresponding authors. The data and analysis code are public, on GitHub and Zenodo.