Positive Linear Framework for Neural Networks with Generalized Logistic Structure: Multivariate and Finite Domain Case
DOI:
https://doi.org/10.65135/atsf.2025.1Keywords:
Neural network operators, positive linear operators, multivariate-abstract quantitative approximation, convexity, generalized logistic activation functionAbstract
In this work, we examine a family of abstract neural network operators in a multivariate setting, defined on compact subsets of R^N and taking values in general Banach spaces. These operators are constructed using a symmetric density derived from a modified logistic-type activation function with adjustable parameters. Within the framework of positive linear operator theory, we provide quantitative convergence estimates towards the identity operator. Our results are formulated using tools such as the modulus of continuity and Frechet derivatives. Furthermore, we offer a detailed discussion on the behavior of the operators when applied to convex functions. This article reflects both the author's contributions to the field over the past 30 years and some recent developments in neural network approximation theory.
References
G. A. Anastassiou, A K-attainable inequality related to the convergence of positive linear operators. Journal of Approximation Theory 44 (1985), no. 4, 380-383. DOI: 10.1016/0021-9045(85)90087-5 DOI: https://doi.org/10.1016/0021-9045(85)90087-5
G. A. Anastassiou, Moments in Probability and Approximation Theory. Pitman Research Notes in Mathematics Series, 287, Longman Scientific & Technical, New York, 1993.
G. A. Anastassiou, Quantitative Approximations. Chapman & Hall/CRC, Boca Raton, FL, 2001.
G. A. Anastassiou, Multivariate and abstract approximation theory for Banach space valued functions. Demonstratio Mathematica 50 (2017), no. 1, 208-222. DOI: 10.1515/dema-2017-0020 DOI: https://doi.org/10.1515/dema-2017-0020
G. A. Anastassiou, Intelligent Computations: Abstract, Fractional Calculus, Inequalities, Approximations. Stud. Comput. Intell., 734, Springer, Cham, 2018. DOI: https://doi.org/10.1007/978-3-319-66936-6
G. A. Anastassiou, Parametrized, Deformed and General Neural Networks. Springer, Heidelberg, New York, 2023. DOI: https://doi.org/10.1007/978-3-031-43021-3
G. A. Anastassiou, Trigonometric and Hyperbolic Generated Approximation Theory. World Scientific, Singapore, New York, 2025. DOI: https://doi.org/10.1142/13857
H. Cartan, Differential Calculus. Herman, Paris, 1971.
Z. Chen and F. Cao, The approximation operators with sigmoidal functions. Computers & Mathematics with Applications, 58 (2009), no. 4, 758-765. DOI: 10.1016/j.camwa.2009.05.001 DOI: https://doi.org/10.1016/j.camwa.2009.05.001
D. Costarelli and R. Spigler, Approximation results for neural network operators activated by sigmoidal functions. Neural Networks 44 (2013), 101-106. DOI: 10.1016/j.neunet.2013.03.015 DOI: https://doi.org/10.1016/j.neunet.2013.03.015
D. Costarelli and R. Spigler, Multivariate neural network operators with sigmoidal activation functions. Neural Networks 48 (2013), 72-77. DOI: 10.1016/j.neunet.2013.07.009 DOI: https://doi.org/10.1016/j.neunet.2013.07.009
L. B. Rall, Computational Solution of Nonlinear Operator Equations. John Wiley & Sons, New York, 1969.
O. Shisha and B. Mond, The degree of convergence of sequences of linear positive operators. Proc. Nat. Acad. Sci. U.S.A. 60 (1968), 1196-1200. DOI: 10.1073/pnas.60.4.1196 DOI: https://doi.org/10.1073/pnas.60.4.1196
O. Shisha and B. Mond, The degree of approximation to periodic functions by linear positive operators. Journal of Approximation Theory 1 (1968), 335--339. DOI: 10.1016/0021-9045(68)90011-7 DOI: https://doi.org/10.1016/0021-9045(68)90011-7
S. Haykin, Neural Networks: A Comprehensive Foundation (2nd ed.), Prentice Hall, New York, 1998.
W. S. McCulloch and W. Pitts, A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5 (1943), 115-133. DOI: 10.1007/BF02478259 DOI: https://doi.org/10.1007/BF02478259
T. M. Mitchell, Machine Learning, WCB-McGraw-Hill, New York, 1997.
D. Yu and F. Cao, Construction and approximation rate for feedforward neural network operators with sigmoidal functions. Journal of Computational and Applied Mathematics 453 (2025), Paper No. 116150, 16 pp. DOI: 10.1016/j.cam.2024.116150 DOI: https://doi.org/10.1016/j.cam.2024.116150
S. Cen, B. Jin, Q. Quan, and Z. Zhou, Hybrid neural-network FEM approximation of diffusion coefficient in elliptic and parabolic problems. IMA Journal of Numerical Analysis 44 (2024), no. 5, 3059-3093. DOI: 10.1093/imanum/drad073 DOI: https://doi.org/10.1093/imanum/drad073
L. Coroianu, D. Costarelli, M. Natale and A. Pantiş, The approximation capabilities of Durrmeyer-type neural network operators. Journal of Applied Mathematics and Computing 70 (2024), no. 5, 4581-4599. DOI: 10.1007/s12190-024-02146-9 DOI: https://doi.org/10.1007/s12190-024-02146-9
X. Warin, The GroupMax neural network approximation of convex functions. IEEE Transactions on Neural Networks and Learning Systems 35 (2024), no. 8, 11608-11612. DOI: 10.1109/TNNLS.2023.3240183 DOI: https://doi.org/10.1109/TNNLS.2023.3240183
A. Fabra, O. Guasch, J. Baiges and R. Codina, Approximation of acoustic black holes with finite element mixed formulations and artificial neural network correction terms. Finite Elements in Analysis and Design 240 (2024), Paper No. 104236, 19 pp. DOI: 10.1016/j.finel.2024.104236 DOI: https://doi.org/10.1016/j.finel.2024.104236
P. Grohs and F. Voigtlaender, Proof of the theory-to-practice gap in deep learning via sampling complexity bounds for neural network approximation spaces. Foundations of Computational Mathematics 24 (2024), no. 4, 1085-1143. DOI: 10.1007/s10208-023-09607-w DOI: https://doi.org/10.1007/s10208-023-09607-w
A. Basteri and D. Trevisan, Quantitative Gaussian approximation of randomly initialized deep neural networks. Machine Learning 113 (2024), no. 9, 6373-6393. DOI: 10.1007/s10994-024-06578-z DOI: https://doi.org/10.1007/s10994-024-06578-z
T. De Ryck and S. Mishra, Error analysis for deep neural network approximations of parametric hyperbolic conservation laws. Mathematics of Computation 93 (2024), no. 350, 2643–2677. DOI: 10.1090/mcom/3934 DOI: https://doi.org/10.1090/mcom/3934
J. Liu, B. Zhang, Y. Lai and L. Fang, Hull form optimization research based on multi-precision back-propagation neural network approximation model. International Journal for Numerical Methods in Fluids 96 (2024), no. 8, 1445–1460. DOI: 10.1002/fld.5291 DOI: https://doi.org/10.1002/fld.5291
J. Yoo, J. Kim, M. Gim and H. Lee, Error estimates of physics-informed neural networks for initial value problems. Journal of the Korean Society for Industrial and Applied Mathematics 28 (2024), no. 1, 33–58. DOI: 10.12941/jksiam.2024.28.033
Downloads
Published
License
Copyright (c) 2025 Approximation Theory and Special Functions
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.