In this note we construct a family of recurrence generated parametric half hyperbolic tangent activation functions. We prove precise upper and lower estimates for the Hausdorff approximation of the sign function by means of this family. Numerical examples, illustrating our results are given.
| Published in | Biomedical Statistics and Informatics (Volume 2, Issue 2) |
| DOI | 10.11648/j.bsi.20170202.18 |
| Page(s) | 87-94 |
| Creative Commons |
This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited. |
| Copyright |
Copyright © The Author(s), 2017. Published by Science Publishing Group |
Parametric Hyperbolic Tangent Activation Function (PHTA), Parametric Half Hyperbolic Tangent Activation Function (PHHTA), Sign Function, Hausdorff Distance
| [1] | N. Guliyev, V. Ismailov, A single hidden layer feedforward network with only one neuron in the hidden layer san approximate any univariate function, Neural Computation, 28, 2016, 1289–1304. |
| [2] | D. Costarelli, R. Spigler, Approximation results for neural network operators activated by sigmoidal functions, Neural Networks, 44, 2013, 101–106. |
| [3] | D. Costarelli, G. Vinti, Pointwise and uniform approximation by multivariate neural network operators of the max-product type, Neural Networks, 2016, doi: 10.1016/j.neunet.2016.06.002, http://www.sciencedirect.com/science/article/pii/S0893608016300685. |
| [4] | D. Costarelli, R. Spigler, Solving numerically nonlinear systems of balance laws by multivariate sigmoidal functions approximation, Computational and Applied Mathematics 2016, doi: 10.1007/s40314-016-0334-8, http://link.springer.com/article/10.1007/s40314-016-0334-8 |
| [5] | D. Costarelli, G. Vinti, Convergence for a family of neural network operators in Orlicz spaces, Mathematische Nachrichten, 2016; doi: 10.1002/mana.20160006 |
| [6] | J. Dombi, Z. Gera, The Approximation of Piecewise Linear Membership Functions and Lukasiewicz Operators, Fuzzy Sets and Systems, 154 (2), 2005, 275–286. |
| [7] | R. Alt, S. Markov, Theoretical and Computational Studies of some Bioreactor Models, Computers and Mathematics with Applications, 64, 2012, 350–360, http://dx.doi.org/10.1016/j.camwa.2012.02.046 |
| [8] | R. Anguelov, S. Markov, Hausdorff Continuous Interval Functions and Approximations, In: M. Nehmeier et al. (Eds), Scientific Computing, Computer Arithmetic, and Validated Numerics, 16th International Symposium, Springer, SCAN 2014, LNCS 9553, 2016, 3–13, doi: 10.1007/978-3-319-31769-4 |
| [9] | G. Lente, Deterministic Kinetics in Chemistry and Systems Biology, Springer, New York, 2015. |
| [10] | N. Kyurkchiev, S. Markov, On the numerical solution of the general kinetic K-angle reaction system, J. Math. Chem., 54 (3), 2016, 792–805. |
| [11] | A. M. Bersani, G. Dell’Acqua, Is there anything left to say on enzyme kinetic constants and quasi-steady state approximation?, J. Math. Chem., 50, 2012, 335–344. |
| [12] | S. Markov, Cell Growth Models Using Reaction Schemes: Batch Cultivation, Biomath, 2, 2013, 1312301, http://dx.doi.org/10.11145/j.biomath.2013.12.301 |
| [13] | M. V. Putz, A. M. Putz, Logistic vs. W-Lambert information in quantum modeling of enzyme kinetics, Int. J. Chemoinf. Chem. Eng., 2011, 1, 42–60. |
| [14] | Z. Quan, Z. Zhang, The construction and approximation of the neural network with two weights, J. Appl. Math., 2014; http://dx.doi.org./10.1155/2014/892653 |
| [15] | I. A. Basheer, M. Hajmeer, Artificial Neural Networks: Fundamentals, Computing, Design, and Application, Journal of Microbiological Methods, 43, 2000, 3–31, http://dx.doi.org/10.1016/S0167-7012(00)00201-3 |
| [16] | Z. Chen, F. Cao, The Approximation Operators with Sigmoidal Functions, Computers & Mathematics with Applications, 58, 2009, 758–765, http://dx.doi.org/10.1016/j.camwa.2009.05.001 |
| [17] | Z. Chen, F. Cao, The Construction and Approximation of a Class of Neural Networks Operators with Ramp Functions, Journal of Computational Analysis and Applications, 14, 2012, 101–112. |
| [18] | Z. Chen, F. Cao, J. Hu, Approximation by Network Operators with Logistic Activation Functions, Applied Mathematics and Computation, 256, 2015, 565–571, http://dx.doi.org/10.1016/j.amc.2015.01.049 |
| [19] | D. Costarelli, R. Spigler, Constructive Approximation by Superposition of Sigmoidal Functions, Anal. Theory Appl., 29, 2013, 169–196, http://dx.doi.org/10.4208/ata.2013.v29.n2.8 |
| [20] | D. Elliott, A better activation function for artificial neural networks, the National Science Foundation, Institute for Systems Research, Washington, DC, ISR Technical Rep. TR–6, 1993. Available: http://ufnalski.edu.pl/zne/ci 2014/papers/Elliott TR 93-8.pdf |
| [21] | K. Babu, D. Edla, New algebraic activation function for multi-layered feed forward neural networks. IETE Journal of Research, 2016, http://dx.doi.org/0.1080/03772063.2016.1240633 |
| [22] | N. Kyurkchiev, Mathematical Concepts in Insurance and Reinsurance. Some Moduli in Programming Environment MATHEMATICA, LAP LAMBERT Academic Publishing, Saarbrucken, 2016, 136 pp. |
| [23] | S. Adam, G. Magoulas, D. Karras, M. Vrahatis, Bounding the search space for global optimization of neural networks learning error: an interval analysis approach, J. of Machine Learning Research, 17, 2016, 1–40. |
| [24] | S. Wang, T. Zhan, Y. Chen, Y. Zhang, M. Yang, H. Lu, H. Wang, B. Liu, P. Phillips, Multiple Sclerosis Detection Based on Biorthogonal Wavelet Transform, RBF Kernel Principal Component Analysis, and Logistic Regression, IEEE Access, Special section on advanced signal processing methods in medical imaging, 4, 2016, 7567–7576. |
| [25] | G. Cybenko, Approximation by superposition of a sigmoidal function, Math. of Control Signals and Systems, 2, 1989, 303–314. |
| [26] | K. Hornik, M. Stinchcombe, H. White, Multi-layer feed forward networks are universal approximations, Neural Networks, 2, 1989, 359–366. |
| [27] | V. Kreinovich, O. Sirisaengtaksin, 3–layer neural networks are universal approximations for functionals and for control strategies, Neural Parallel and Scientific Computations, 1, 1993, 325–346. |
| [28] | H. White, Connectionist nonparametric regression: multilayer feedforward networks can learn arbitrary mappings, Neural Networks, 3, 1990, 535–549. |
| [29] | W. Duch, N. Jankowski, Survey of neural transfer functions, Neural Computing Surveys, 2, 1999, 163–212. |
| [30] | F. Hausdorff, Set Theory (2 ed.) (Chelsea Publ., New York, (1962 [1957]) (Republished by AMS-Chelsea 2005), ISBN: 978–0–821–83835–8. |
| [31] | B. Sendov, Hausdorff Approximations, Kluwer, Boston, 1990, http://dx.doi.org/10.1007/978-94-009-0673-0 |
| [32] | N. Kyurkchiev, A. Andreev, Approximation and antenna and filter synthesis: Some moduli in programming environment Mathematica, LAP LAMBERT Academic Publishing, Saarbrucken, 2014, ISBN 978-3-659-53322-8. |
| [33] | N. Kyurkchiev, A family of recurrence generated sigmoidal functions based on the Verhulst logistic function. Some approximation and modeling aspects, Biomath Communications, 3 (2), 2016, 18 pp., http://www.biomathforum.org/biomath/index.php/conference/article/view/789/873 |
| [34] | N. Kyurkchiev, S. Markov, Hausdorff Approximation of the Sign Function by a Class of Parametric Activation Functions, Biomath Communications, 3 (2), 2016, 14 pp., http://dx.doi.org/10.11145/bmc.2016.12.217 |
| [35] | R. K. Cloues, W. A. Sather, Afterhyperpolarization Regulates Firing Rate in Neurons of the Suprachiasmatic Nucleus, The Journal of Neuroscience, 23 (5), 2003, 1593–1604. |
| [36] | N. Kyurkchiev, On the Approximation of the step function by some cumulative distribution functions, Compt. rend. Acad. bulg. Sci., 68 (12), 2015, 1475–1482. |
| [37] | N. Kyurkchiev, S. Markov, Sigmoid functions: Some Approximation and Modelling Aspects, LAP LAMBERT Academic Publishing, Saarbrucken, 2015, ISBN 978-3-659-76045-7. |
| [38] | N. Kyurkchiev, S. Markov, On the Hausdorff distance between the Heaviside step function and Verhulst logistic function. J. Math. Chem., 54 (1), 2016, 109–119, doi: 10.1007/S10910-015-0552-0 |
| [39] | N. Kyurkchiev, S. Markov, Sigmoidal functions: some computational and modelling aspects. Biomath Communications, 1 (2), 2014, 30–48, doi: 10.11145/j.bmc.2015.03.081. |
| [40] | N. Kyurkchiev, S. Markov, On the approximation of the generalized cut function of degree p+1 by smooth sigmoid functions. Serdica J. Computing, 9 (1), 2015, 101–112. |
| [41] | N. Kyurkchiev, A note on the new geometric representation for the parameters in the fibril elongation process, Compt. rend. Acad. bulg. Sci., 69 (8), 2016, 963–972. |
| [42] | N. Kyurkchiev, S. Markov, A. Iliev, A note on the Schnute growth model, Int. J. of Engineering Research and Development, 12 (6), 2016, 47-54, ISSN 2278-067X, http://www.ijerd.com/paper/vol12-issue6/Verison-1/G12614754.pdf |
| [43] | V. Kyurkchiev, N. Kyurkchiev, On the Approximation of the Step function by Raised-Cosine and Laplace Cumulative Distribution Functions. European International Journal of Science and Technology, 4 (9), 2016, 75–84. |
| [44] | A. Iliev, N. Kyurkchiev, S. Markov, On the Approximation of the Cut and Step Functions by Logistic and Gompertz Functions, BIOMATH, 4 (2), 2015, 1510101, http://dx.doi.org/10.11145/j.biomath.2015.10.101 |
| [45] | A. Iliev, N. Kyurkchiev, S. Markov, On the Approximation of the step function by some sigmoid functions, Mathematics and Computers in Simulation, 2015, http://dx.doi.org/10.1016/j.matcom.2015.11.005 |
| [46] | N. Kyurkchiev, A. Iliev, A note on some growth curves arising from Box-Cox transformation, Int. J. of Engineering Works, 3 (6), 2016, 47–51, ISSN: 2409-2770. |
| [47] | N. Kyurkchiev, A. Iliev, On some growth curve modeling: approximation theory and applications, Int. J. of Trends in Research and Development, 3 (3), 2016, 466–471, http://www.ijtrd.com/papers/IJTRD3869.pdf |
| [48] | A. Iliev, N. Kyurkchiev, S. Markov, Approximation of the cut function by Stannard and Richards sigmoid functions, IJPAM, 109 (1), 2016, 119-128, http://www.ijpam.eu/contents/2016-109-1/9/9.pdf |
| [49] | N. Kyurkchiev, A. Iliev, On the Hausdorff distance between the shifted Heaviside function and some generic growth functions, Int. J. of Engineering Works, 3 (10), 2016, 73-77. |
| [50] | N. Kyurkchiev, S. Markov, Approximation of the cut function by some generic logistic functions and applications, Advances in Applied Sciences, 1 (2), 2016, 24-29. |
| [51] | A. Iliev, N. Kyurkchiev, S. Markov, On the Hausdorff Distance Between the Shifted Heaviside Step Function and the Transmuted Stannard Growth Function, BIOMATH, 5 (2), 2016, 1-6. |
| [52] | N. Kyurkchiev, A. Iliev, On the Hausdorff distance between the Heaviside function and some transmuted activation functions, Mathematical Modelling and Applications, 2 (1), 2016, 1-5. |
| [53] | N. Kyurkchiev, Uniform Approximation of the Generalized Cut Function by Erlang Cumulative Distribution Function. Application in Applied Insurance Mathematics, International Journal of Theoretical and Applied Mathematics, 2 (2), 2016, 40–44. |
| [54] | N. Kyurkchiev, S. Markov, Hausdorff approximation of the sign function by a class of parametric activation functions, Biomath Communications, 3 (2), 2016; 14 pp. |
| [55] | A. Iliev, N. Kyurkchiev, S. Markov, A family of recurrence generated parametric activation functions with applications to neural networks, International Journal on Research Innovations in Engineering Science and Technology, 2 (1), 2017. |
| [56] | http://www.cse.scu.edu/tschwarz/coen266_09/PPT/Artificial |
| [57] | M. Turner, B. Blumenstein, J. Sebaugh, A Generalization of the Logistiv Law of Growth, Biometrics, 25 (3), 1969, 577–580. |
| [58] | J. A. Nelder, The fitting of a generalization of the logistic curve, Biometrics, 17, 1961, 89–110. |
| [59] | J. A. Nelder, An alternative form of a generalized logistic equations, Biometrics, 18 (4), 1962, 614–616. |
| [60] | A. Tsoularis, Analysis of logistic growth models, Res. Lett. Inf. Math. Sci., 2, 2001, 23–46. |
| [61] | O. Garcia, Unifying sigmoid univariate growth equations, Forest Biometry, Modelling and Information Sciences, 1, 2005, 63–68. |
| [62] | O. Garcia, Visualization of a general family of growth functions and probability distributions - The Growth–curve Explorer, Environmental Modelling and Software, 23, 2008, 1474–1475. |
| [63] | G. Box, D. Cox, An analysis of transformations, Journal of the Royal Statistical Society, B 26, 1964, 211–252. |
APA Style
Vesselin Kyurkchiev, Nikolay Kyurkchiev. (2017). A Family of Recurrence Generated Functions Based on “Half–Hyperbolic Tangent Activation Function”. Biomedical Statistics and Informatics, 2(2), 87-94. https://doi.org/10.11648/j.bsi.20170202.18
ACS Style
Vesselin Kyurkchiev; Nikolay Kyurkchiev. A Family of Recurrence Generated Functions Based on “Half–Hyperbolic Tangent Activation Function”. Biomed. Stat. Inform. 2017, 2(2), 87-94. doi: 10.11648/j.bsi.20170202.18
Share This Article
Twitter
Linked In
Facebook
Copy
Download@article{10.11648/j.bsi.20170202.18,
author = {Vesselin Kyurkchiev and Nikolay Kyurkchiev},
title = {A Family of Recurrence Generated Functions Based on “Half–Hyperbolic Tangent Activation Function”},
journal = {Biomedical Statistics and Informatics},
volume = {2},
number = {2},
pages = {87-94},
doi = {10.11648/j.bsi.20170202.18},
url = {https://doi.org/10.11648/j.bsi.20170202.18},
eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.bsi.20170202.18},
abstract = {In this note we construct a family of recurrence generated parametric half hyperbolic tangent activation functions. We prove precise upper and lower estimates for the Hausdorff approximation of the sign function by means of this family. Numerical examples, illustrating our results are given.},
year = {2017}
}
TY - JOUR T1 - A Family of Recurrence Generated Functions Based on “Half–Hyperbolic Tangent Activation Function” AU - Vesselin Kyurkchiev AU - Nikolay Kyurkchiev Y1 - 2017/03/22 PY - 2017 N1 - https://doi.org/10.11648/j.bsi.20170202.18 DO - 10.11648/j.bsi.20170202.18 T2 - Biomedical Statistics and Informatics JF - Biomedical Statistics and Informatics JO - Biomedical Statistics and Informatics SP - 87 EP - 94 PB - Science Publishing Group SN - 2578-8728 UR - https://doi.org/10.11648/j.bsi.20170202.18 AB - In this note we construct a family of recurrence generated parametric half hyperbolic tangent activation functions. We prove precise upper and lower estimates for the Hausdorff approximation of the sign function by means of this family. Numerical examples, illustrating our results are given. VL - 2 IS - 2 ER -
Information
Email: