Publication:
Correlative information maximization: a biologically plausible approach to supervised deep neural networks without weight symmetry

dc.contributor.coauthorPehlevan, Cengiz
dc.contributor.departmentDepartment of Electrical and Electronics Engineering
dc.contributor.departmentKUIS AI (Koç University & İş Bank Artificial Intelligence Center)
dc.contributor.departmentGraduate School of Sciences and Engineering
dc.contributor.kuauthorBozkurt, Barışcan
dc.contributor.kuauthorErdoğan, Alper Tunga
dc.contributor.schoolcollegeinstituteGRADUATE SCHOOL OF SCIENCES AND ENGINEERING
dc.contributor.schoolcollegeinstituteCollege of Engineering
dc.contributor.schoolcollegeinstituteResearch Center
dc.date.accessioned2024-12-29T09:36:09Z
dc.date.issued2023
dc.description.abstractThe backpropagation algorithm has experienced remarkable success in training large-scale artificial neural networks;however, its biological plausibility has been strongly criticized, and it remains an open question whether the brain employs supervised learning mechanisms akin to it. Here, we propose correlative information maximization between layer activations as an alternative normative approach to describe the signal propagation in biological neural networks in both forward and backward directions. This new framework addresses many concerns about the biological-plausibility of conventional artificial neural networks and the backpropagation algorithm. The coordinate descent-based optimization of the corresponding objective, combined with the mean square error loss function for fitting labeled supervision data, gives rise to a neural network structure that emulates a more biologically realistic network of multi-compartment pyramidal neurons with dendritic processing and lateral inhibitory neurons. Furthermore, our approach provides a natural resolution to the weight symmetry problem between forward and backward signal propagation paths, a significant critique against the plausibility of the conventional backpropagation algorithm. This is achieved by leveraging two alternative, yet equivalent forms of the correlative mutual information objective. These alternatives intrinsically lead to forward and backward prediction networks without weight symmetry issues, providing a compelling solution to this long-standing challenge.
dc.description.indexedbyWoS
dc.description.indexedbyScopus
dc.description.publisherscopeInternational
dc.description.sponsorsThis research was supported by KUIS AI Center Research Award. B. Bozkurt acknowledges the support by Gatsby PhD programme, which is supported by the Gatsby Charitable Foundation (GAT3850). C. Pehlevan is supported by NSF Award DMS-2134157, NSF CAREER Award IIS-2239780, and a Sloan Research Fellowship. This work has been made possible in part by a gift from the Chan Zuckerberg Initiative Foundation to establish the Kempner Institute for the Study of Natural and Artificial Intelligence.
dc.identifier.issn1049-5258
dc.identifier.quartileN/A
dc.identifier.scopus2-s2.0-85191197285
dc.identifier.urihttps://hdl.handle.net/20.500.14288/21968
dc.identifier.wos1220600006015
dc.keywordsBiological neural networks
dc.keywordsCoordinate descent
dc.keywordsForward-and-backward
dc.keywordsInformation maximization
dc.keywordsLarge-scales
dc.keywordsLearning mechanism
dc.keywordsMeans square errors
dc.keywordsOptimisations
dc.keywordsSignal propagation
dc.keywordsWeight symmetry
dc.languageen
dc.publisherNeural Information Processing Systems 36
dc.sourceAdvances in Neural Information Processing Systems 36 (NeurIPS 2023)
dc.subjectComputer science
dc.subjectArtificial intelligence
dc.subjectInformation systems
dc.titleCorrelative information maximization: a biologically plausible approach to supervised deep neural networks without weight symmetry
dc.typeConference proceeding
dspace.entity.typePublication
local.contributor.kuauthorBozkurt, Barışcan
local.contributor.kuauthorErdoğan, Alper Tunga
local.publication.orgunit1GRADUATE SCHOOL OF SCIENCES AND ENGINEERING
local.publication.orgunit1College of Engineering
local.publication.orgunit1Research Center
local.publication.orgunit2Department of Electrical and Electronics Engineering
local.publication.orgunit2KUIS AI (Koç University & İş Bank Artificial Intelligence Center)
local.publication.orgunit2Graduate School of Sciences and Engineering
relation.isOrgUnitOfPublication21598063-a7c5-420d-91ba-0cc9b2db0ea0
relation.isOrgUnitOfPublication77d67233-829b-4c3a-a28f-bd97ab5c12c7
relation.isOrgUnitOfPublication3fc31c89-e803-4eb1-af6b-6258bc42c3d8
relation.isOrgUnitOfPublication.latestForDiscovery21598063-a7c5-420d-91ba-0cc9b2db0ea0
relation.isParentOrgUnitOfPublication434c9663-2b11-4e66-9399-c863e2ebae43
relation.isParentOrgUnitOfPublication8e756b23-2d4a-4ce8-b1b3-62c794a8c164
relation.isParentOrgUnitOfPublicationd437580f-9309-4ecb-864a-4af58309d287
relation.isParentOrgUnitOfPublication.latestForDiscovery8e756b23-2d4a-4ce8-b1b3-62c794a8c164

Files

Original bundle

Now showing 1 - 1 of 1
Thumbnail Image
Name:
IR04538.pdf
Size:
569.39 KB
Format:
Adobe Portable Document Format