Neuromodulation via Krotov-Hopfield improves accuracy and robustness of restricted Boltzmann machines

Abstract

In biological systems, neuromodulation tunes synaptic plasticity based on the internal state of the organism, complementing stimulus-driven Hebbian learning. The algorithm recently proposed by Krotov and Hopfield (KH) [D. Krotov and J. J. Hopfield, Proc. Natl. Acad. Sci. USA116, 7723 (2026) 0027-8424 10.1073/pnas.1820458116] can be utilized to mirror this process in artificial neural networks, where its built-in intralayer competition and selective inhibition of synaptic updates offer a cost-effective remedy for the lack of lateral connections through a simplified attention mechanism. We demonstrate that KH-modulated restricted Boltzmann machines (RBMs) outperform standard (shallow) RBMs in both reconstruction and classification tasks, offering a superior trade-off between generalization performance and model size, with the additional benefit of robustness to weight initialization as well as to overfitting during training.