Open Access Open Access  Restricted Access Subscription or Fee Access

Load Balancing in Big Data Processing Systems


(*) Corresponding author


Authors' affiliations


DOI: https://doi.org/10.15866/ireaco.v12i1.16808

Abstract


The article examines the load balancing methods used in the construction of big data processing systems. In the light of the widespread transition to Industry 4.0, the authors analyze the major trends in the development of science and technology, describe the existing balancing methods and suggest their recommendations to improve their effective use. The analysis has revealed the key features of load balancing algorithms. The study examines load balancers in terms of the four layers of the TCP/IP network model – data link, network, transport and application. Furthermore, the authors refer to the features of load balancing algorithms and formalize the relevant requirements. The study primarily focuses on the load balancing at the Internet layer. The authors suggest formulas for calculating the response time for various load balancing algorithms. A new method of load balancing is proposed for big data processing systems. The article presents a typical network topology. The solution involves the integration of large data methods into a load balancing system. To implement the load distribution in the server cluster, the authors use a processing cluster analyzing the server machines and managing the distribution of the load in the network, based on the received data. In the end, the authors suggest the load balancing algorithm.
Copyright © 2019 Praise Worthy Prize - All rights reserved.

Keywords


Network Model; Load Balancing Algorithm; Network Building; Round Robin; Least Connections; Source Hash Scheduling; Sticky Sessions; Big Data

Full Text:

PDF


References


G. Prause, S. Atari, On sustainable production networks for Industry 4.0, Entrepreneurship and Sustainability (Issue 4(4)): 421-431, June 2017.
https://doi.org/10.9770/jesi.2017.4.4(2)

S. M. Bychkova, N. N. Makarova, E. A. Zhidkova, Measurement of information in the subsystem of internal control of the controlling system of organizations of the agro-industrial complex, Entrepreneurship and Sustainability (Issue 6(1)): 35-43, September 2018.
https://doi.org/10.9770/jesi.2018.6.1(3)

A. I. Vlasov, M. N. Yuldashev, Analysis of methods and means for processing information of the sensor cluster, Sensors and systems (Issue 221): 24-30, January 2018.

T. Cai, Ph.D. thesis, Load balancing algorithms for local area network. University of York, 1996.

S. Phillips, J. Westbrook On-line load balancing and network flow, Algorithmica, Vol. 1(Issue 3): 245-261, 1998.
https://doi.org/10.1007/pl00009214

Load Balancing FAQ. Accessed date: 17 April 2018. https://www.haproxy.com/blog/loadbalancing-faq/

B. Awerbuch, A. Brinkmann, Ch. Scheideler, Anycasting in adversarial systems: routing and admission control, Lecture Notes in Computer Science, Vol. 4: 1153-1168, 2003.
https://doi.org/10.1007/3-540-45061-0_88

Z. Hu, L. Xiao, Y. Lin, Research and application of e-commerce platform for enterprise based on NLB, 2nd International Conference on Pervasive Computing and Applications, ICPCA'07, pp. 360-364, Birmingham, 2007.
https://doi.org/10.1109/icpca.2007.4365469

H. Tahilramani, Ph.D. thesis, Traffic-sensitive routing and traffic engineering. Rensselaer Polytechnic Institute, 2002.

J. G. Carlsson, R. Devulapalli, E. Carlsson, Balancing workloads of service vehicles over a geographic territory, IEEE International Conference on Intelligent Robots and Systems ‘IROS 2013: New Horizon, Conference Digest’, pp. 209-2016, 2013.
https://doi.org/10.1109/iros.2013.6696355

P. Di Marco, P. Park, C. Fischione, K. H. Johansson, Trend: a timely, reliable, energy-efficient and dynamic WSN protocol for control applications, IEEE International Conference on Communications – ICC 2010, pp. 1-6, Cape Town, 2010.
https://doi.org/10.1109/icc.2010.5501971

V. Cardellini, M. Colajanni, Ph.S. Yu, DNS dispatching algorithms with state estimators for scalable web-server clusters, World Wide Web, Vol. 2(Issue 3): 101-113, 1999.
https://doi.org/10.1023/a:1019296605640

C. Ji, J. Yuan, X. Shan, X. Yuchi, X. Li, Analysis of domain name queries based on the k-means algorithm, Qinghua Daxue Xuebao (Ziran Kexue Ban), Vol. 50(Issue 4): 601-608, 2010.

A. N. Jordan, E. V. Sukhorukov, S. Pilgram, Fluctuation statistics in networks: a stochastic path integral approach, Journal of Mathematical Physics, Vol. 45(Issue 11): 4386-4417, 2004.
https://doi.org/10.1063/1.1803927

A. I. Vlasov, A. V. Yudin, V. A. Shakhnov, K. A. Usov, M. A. Salmina, Design methods of teaching the development of internet of things components with considering predictive maintenance on the basis of mechatronic devices, International Journal of Applied Engineering Research, Vol. 12(Issue 20): 9390-9396, December 2013.

A. Balachandran, Ph.D. thesis, Incorporating location-awareness in public-area wireless networks. University of California, 2004.

S. G. Korchagin, O. G. Yaskevich, NGINX as a tool for creating complex optimized web-applications, Proceedings of an international youth scientific school ‘Optimization and modeling in automated systems’, pp. 179-181, 2018.

H. Brunst, E. Gabriel, M. Lange, M. S. Muller, W. E. Nagel, M. M. Resch, Performance analysis of a parallel application in the GRID, Lecture Notes in Computer Science, Vol. 2658: 285-294, 2003.
https://doi.org/10.1007/3-540-44862-4_31

D. Ageyev, L. Kirichenko, T. Radivilova, M. Tawalbeh, O. Baranovskyi, Method of self-similar load balancing in network intrusion detection system, 28th International Conference ‘RADIOELEKTRONIKA’, pp. 1-4, 2018.
https://doi.org/10.1109/radioelek.2018.8376406

K. A. Muravyev, V. V. Terekhov, Methods of managing network traffic of heterogeneous distributed telecommunication systems, Designing and technology of electronic means (Issue 2): 15-21, February 2018.

A. I. Vlasov, M. N. Yuldashev, Gaussian processes in regression analysis of states of a wireless sensor network with allowance for electromagnetic interference, Electromagnetic compatibility technologies (Issue 62): 35-43, March 2017.

A. M. Marusik, Web-server dynamic load balancing system, Information system, mechanics and Kerwan (Issue 12): 25-32, 2015.

A. Yemelianov. Load Balancing: Basic Algorithms and Methods. Accessed date: 22 April 2018. https://blog.selectel.com/load-balancing-basic-algorithms-and-methods/

O. N. Soboleva, Master's thesis, Developing an application for load balancing on a server complex of virtual machines. Tomsk State University, 2016.

E. O. Vikulov, E. A. Leonov, L. A. Denisova, Automated distribution of large data volumes of high-loaded systems, Dynamics of systems, mechanisms and machines (Issue 3): 146-149, 2014.

M. N. Yuldashev, A. I. Vlasov, A. N. Novikov, Energy-efficient algorithm for classification of states of wireless sensor network using machine learning methods, Journal of Physics: Conference Series (Issue 1015), 2018.
https://doi.org/10.1088/1742-6596/1015/3/032153

K. A. Muravev, A. E. Averyanikhin, A. V. Kotelnitskiy, Method for calculating the optimal number of nodes in a cluster of virtualization of a private cloud of virtual desktops by the efficiency criterion, International Scientific and Research Journal (Issue 5-3 (47)): 6-13, 2016.

K. A. Muraviev, V. V. Terekhov, Software and Hardware Complex for Monitoring Distributed Telecommunication Systems, International Symposium Reliability and Quality Conference, Vol. 1. pp. 324-329, Penza State University, 2017.


Refbacks

  • There are currently no refbacks.



Please send any question about this web site to info@praiseworthyprize.com
Copyright © 2005-2024 Praise Worthy Prize