Influence of Learning Rate Hyper-Parameter and Early Stopping Training Technique in Training Vanilla Neural Network Model for Effective Signal Power Loss Prediction
(*) Corresponding author
DOI: https://doi.org/10.15866/irecap.v13i5.23606
Abstract
In this research work, the authors employed a Vanilla Neural Network (VNN) model to examine the influence of the Learning Rate (LR) hyper-parameter, early stopping training technique, Levenberg-Marquardt (LM) and Bayesian Regularization (BR) training algorithms in the training and prediction of signal power loss using a measured dataset from a Long Term Evolution (LTE) micro-cell environment. First order statistical performance indices, including the Mean Squared Error (MSE), Root Mean Square Error (RMSE), and Regression (R), were adopted for interpreting and analyzing the results. The LR hyper-parameters were sequentially selected cyclically from 0.002 to 1.00, while the early stopping training technique was chosen at a ratio of 70%:15%:15% for training, testing, and validation of the neural network model during network training. The authors examined the neural network training and its predictive abilities of the measured signal power by training the model with LM and BR training algorithms and applying varied LR values and the early stopping training technique. Comparative analysis was also conducted by training the VNN model without the application of LR hyper-parameter and the early stopping training technique. The LR hyper-parameter is a distinct training parameter that, when efficiently applied, improves network convergence, while the application of training techniques such as early stopping minimizes over-fitting during network training. The training result outputs demonstrate the effectiveness of the VNN model when applying a very small LR of 0.002. The best prediction results of the VNN model were observed when using an LR of 0.002, with an R value of 0.9922, performance MSE of 1.48, and RMSE of 1.6790 while training the VNN model with the BR algorithm and applying the early stopping training technique. When training the VNN model with an LR of 0.002 using the LM algorithm, an R value of 0.9816, performance MSE of 5.060, and RMSE of 2.5619 were obtained. The worst prediction results were computed when training the VNN model without the application of LR and the early stopping training technique, resulting in an R value of 0.97480, performance MSE of 6.3, and RMSE of 3.0167. However, when training the same VNN model without the application of LR and the early stopping training technique using the BR training algorithm, an R value of 0.9910, performance MSE of 3.3, and RMSE of 1.8045 were obtained. This strongly demonstrates that the BR training algorithm is not only a good training algorithm but also an excellent training technique.
Copyright © 2023 The Authors - Published by Praise Worthy Prize under the CC BY-NC-ND license.
Keywords
Full Text:
PDFReferences
V. C. Ebhota, J. Isabona, and V. M. Srivastava, "Investigating signal power loss prediction in a metropolitan island using ADALINE and multi-layer perceptron back propagation networks," International Journal of Applied Engineering Research vol. 13, pp. 13409-13420, 2018.
M.M. Rathore, A. Paul, S. Rho, M. Khan,. S. Vimal,. S.A.Shah," Smart traffic control: Identifying driving-violations using fog devices with vehicular cameras in smart cities". Sustain Cities Soc., vol.71, pp.102986, 2021.
https://doi.org/10.1016/j.scs.2021.102986
J. Isabona, A.L. Imoize, S. Ojo, O. Karunwi, , Y. Kim, L. Cheng-Chi, L. Chun-Ta, "Development of a Multilayer Perceptron Neural Network for Optimal Predictive Modeling in Urban Microcellular Radio Environments" Appl.Sci. vol. 12, pp.1-27, 2022.
https://doi.org/10.3390/app12115713
Ebhota, V., Srivastava, V., Prediction of Signal Power Loss (at Micro-Cell Environments) Using Generalized Adaptive Regression and Radial Basis Function ANN Models Built on Filter for Error Correction, (2021) International Journal on Communications Antenna and Propagation (IRECAP), 11 (2), pp. 94-105.
https://doi.org/10.15866/irecap.v11i2.19964
Jiakai Wei, "Forget the learning rate, decay loss," International Journal of Machine Learning and Cybernetics, vol. 9, no. 3, pp. 267-272, June 2019.
https://doi.org/10.18178/ijmlc.2019.9.3.797
C. Nguyen, A.A. Cheema, "A deep neural network-based multi-frequency path loss prediction model from 0.8 GHz to 70 GHz". Sensors, vol. 21, pp5100, 2021.
https://doi.org/10.3390/s21155100
K.C Igwe, O.D. Oyedum, A.M. Aibinu, M.O. Ajewole, and A.S.Moses, "Application of artificial neural network modeling techniques to signal strength computation" Journal of Heliyon, vol. 7, pp. 1-9, 2021.
https://doi.org/10.1016/j.heliyon.2021.e06047
O. U. Anyama and C. P. Igiri, "An application of linear regression & artificial neural network model in the NFL result prediction," International Journal of Engineering Research & Technology, vol. 4, pp. 457-461, 2015.
C. P. Igiri and E. O. Nwachukwu, "An improved prediction system for football a match result," Journal of Engineering, vol. 4, pp. 12-20, 2014.
Virginia C. Ebhota and Viranjay M. Srivastava, "Enhanced Error Reduction of Signal Power loss during Electromagnetic propagation; An Architectural Composition and Learning Rate Selection", Journal of Communication (JCM), vol. 16, no 10, 2021.
https://doi.org/10.12720/jcm.16.10.450-456
Y. Bengio, "Practical recommendations for gradient-based training of deep architectures," Neural Networks:Tricks of the Trade, vol. 7700, pp. 437-478, 2012.
https://doi.org/10.1007/978-3-642-35289-8_26
H. White, "Learning in neural networks: A statistical perspective," Neural Computation, vol. 1, pp. 425- 464, 1989.
https://doi.org/10.1162/neco.1989.1.4.425
Virginia C. Ebhota and Viranjay M. Srivastava, "Performance Comparison of Non-Linear Median Filter Built on MLP-ANN and Conventional MLP-ANN: Using Improved Dataset Training in Micro-Cell Environment" Journal of Communication (JCM), vol.16, no 11, pp.508-515, 2021.
https://doi.org/10.12720/jcm.16.11.508-515
J. Isabona and V. M. Srivastava, "Hybrid neural network approach for predicting signal propagation loss in urban microcells," Proceedings of the 2016 IEEE Region 10 Humanitarian Technology Conference (R10-HTC), Agra, India, 2016.
https://doi.org/10.1109/R10-HTC.2016.7906853
V. C. Ebhota, J. Isabona, and V. M. Srivastava, "Environment-adaptation based hybrid neural network predictor for signal propagation loss prediction in cluttered and open urban microcells," Wireless Personal Communication, pp. 1-14, 2018.
https://doi.org/10.1007/s11277-018-6061-2
A. Neskovic and N. Neskovic, "Microcell electric field strength prediction model based upon artificial neural networks," International Journal Electronics and Communication, vol. 64, pp. 733-738, 2010.
https://doi.org/10.1016/j.aeue.2009.05.005
P. S. Sotirios and K. Siakavara, "Mobile radio propagation path loss prediction using artificial neural networks with optimal input information for urban environments," International Journal of Electronics and Communications pp. 1453-1463, 2015.
https://doi.org/10.1016/j.aeue.2015.06.014
A. B. Tammam, A. Rabie, and K. S. Mustafa, "Neural network approach to model the propagation path loss for great tripoli area at 900, 1800, and 2100 MHz bands," 16th International Conference on Sciences and Techniques of Automatic Control & Computer Engineering pp. 793-798, 2015.
V. C. Ebhota, J. Isabona, and V. M. Srivastava, "Signal power loss prediction based on artificial neural networks in microcell environment,," 3rd IEEE International Conference on Electro-Technology for National Development (IEEE NIGERCON), Owerri, Nigeria, 7-10 Nov. 2017, pp. 250-257.
https://doi.org/10.1109/NIGERCON.2017.8281897
Virginia C. Ebhota, Joseph Isabona, and Viranjay M. Srivastava, "Improved adaptive signal power loss prediction using combined vector statistics based smoothing and neural network approach," Int. J. on Progress in Electromagnetics Research C (PIER C), vol. 82, pp. 155-169, March 2018.
https://doi.org/10.2528/PIERC18011203
Joseph Isabona and Viranjay M. Srivastava, "Coverage and link quality trends in suburban mobile broadband HSPA network environments," Wireless personal Communication (WPC), vol. 95, no. 4, pp. 3955-3968, Aug. 2017.
https://doi.org/10.1007/s11277-017-4034-5
Zhang, S., Sakulyeva, T., Pitukhin, E., Doguchaeva, S., Neuro-Fuzzy and Soft Computing - A Computational Approach to Learning and Artificial Intelligence, (2020) International Review of Automatic Control (IREACO), 13 (4), pp. 191-199.
https://doi.org/10.15866/ireaco.v13i4.19179
Anggriawan, D., Amsyar, A., Firdaus, A., Wahjono, E., Sudiharto, I., Prasetyono, E., Tjahjono, A., Real Time Harmonic Load Identification Based on Fast Fourier Transform and Artificial Neural Network, (2021) International Review of Electrical Engineering (IREE), 16 (3), pp. 220-228.
https://doi.org/10.15866/iree.v16i3.18131
Refbacks
- There are currently no refbacks.
Please send any question about this web site to info@praiseworthyprize.com
Copyright © 2005-2024 Praise Worthy Prize