<link rel="stylesheet" href="styles.f3b1fba60ec7970c.css">

Publication:
Pade neurons for efficient neural models

Loading...
Thumbnail Image

Departments

School / College / Institute

Item type:Organizational Unit,

Program

Organization Authors

Co-Authors

Date

Language

eng

Embargo Status

No

Journal Title

Journal ISSN

Volume Title

Alternative Title

Abstract

Neural networks commonly employ the McCulloch-Pitts neuron model, which is a linear model followed bya point-wise non-linear activation. Various researchers havealready advanced inherently non-linear neuron models, such asquadratic neurons, generalized operational neurons, generativeneurons, and super neurons, which offer stronger non-linearitycompared to point-wise activation functions. In this paper, weintroduce a novel and better non-linear neuron model calledPad e neurons (Paons), inspired by Pad e approximants.Paonsoffer several advantages, such as diversity of non-linearity, sinceeachPaonlearns a different non-linear function of its inputs,and layer efficiency, sincePaons provide stronger non-linearity inmuch fewer layers compared to piecewise linear approximation.Furthermore,Paons include all previously proposed neuronmodels as special cases, thus any neuron model in any networkcan be replaced byPaons. We note that there has been a proposalto employ the Pad e approximation as a generalized point-wiseactivation function, which is fundamentally different from ourmodel. To validate the efficacy of Paons, in our experiments,we replace classic neurons in some well-known neural imagesuper-resolution, compression, and classification models basedon the ResNet architecture withPaons. Our comprehensiveexperimental results and analyses demonstrate that neural mod-els built byPaons provide better or equal performance thantheir classic counterparts with a smaller number of layers. The PyTorch implementation code forPaonis open-sourced at https://github.com/onur-keles/Paon

Source

Publisher

IEEE

Citation

item.page.haspartof

Source

IEEE Transactions on Image Processing

item.page.ispartofseries

item.page.edition

DOI

10.1109/TIP.2026.3653202

item.page.datauri

item.page.link

Rights

N/A

Copyrights Note

Rights and licensing

Endorsement

Review

Supplemented By

Referenced By

Related Patent

Related Goal

Google Scholar
Scholar'da Ara ↗
Altmetric
Dimensions
PlumX Metrikleri
BIP! Indicators