Routing in Elastic Optical Networks Based on Deep Reinforcement Learning for Multi-Agent Systems

Diego Fernando Aguirre; Julio Barón Velandia; Octavio Salcedo Parra

doi:10.15866/iremos.v15i5.22768

Routing in Elastic Optical Networks Based on Deep Reinforcement Learning for Multi-Agent Systems

Diego Fernando Aguirre^(1*), Julio Barón Velandia⁽²⁾, Octavio Salcedo Parra⁽³⁾

^(*) Corresponding author

Authors' affiliations

DOI: https://doi.org/10.15866/iremos.v15i5.22768

Abstract

Reinforcement Learning (RL) has become a valuable strategy in artificial intelligence and it has showed some success in real-world scenarios. Nonetheless, most of the progress achieved in research is often hard to harness in real-world systems given the theoretical assumptions, which are rarely aligned with practical settings. This work focuses on the assignment of resources within elastic optical networks, as a solution to their ever-increasing traffic. This document performs an assessment of two multi-agent reinforcement learning algorithms to solve Routing, Modulation, Spectrum, and Core Assignment (RMSCA), seeking to optimize availability for resource assignment over a network topology and increase the overall capacity, while considering the variability of traffic-related demands. Simulations are carried out in a 14-node topology. The results evidence a 50% increase in spectral efficiency and a blocking probability below 10%. After the system training process, low latency, high speed, and high availability are ensured, thus improving quality for the end user.
Copyright © 2022 Praise Worthy Prize - All rights reserved.

Keywords

Blocking Probability; Deep Q-Network; Elastic Optical Network; Frequency Slot Unit; Multi-Agent Reinforcement Learning; Reinforcement Learning

Full Text:

PDF

References

V. Perasso and B. Mundo, News Latin America International Economy Technology Science What is the fourth industrial revolution (and why should we care), 2016. Accessed: May 07, 2020. [Online]. Available:
http://www.bbc.com/mundo/noticias-37631834?ocid=socialflow_facebook

A. S. Lucía, E. Cavalo, and G. Benza, Impact of the COVID-19 pandemic on the employment situation of early childhood care workers, Accessed: Jul. 28, 2022. [Online].
Available: www.ilo.org/buenosaires

New trends in teleworking: this is how COVID-19 has changed our daily lives - news center. (accessed Jul. 28, 2022).
https://news.microsoft.com/es-es/2020/04/14/nuevas-tendencias-en-el-teletrabajo-asi-nos-ha-cambiado-covid-19/

Y. Zhou et al., Service-aware 6G: An intelligent and open network based on the convergence of communication, computing and caching, Digit. Commun. Networks, vol. 6, no. 3, pp. 253-260, Aug. 2020.
https://doi.org/10.1016/j.dcan.2020.05.003

Idowu-Bismark, O., Okokpujie, K., Husbands, R., Adedokun, M., 5G Wireless Communication Network Architecture and Its Key Enabling Technologies, (2019) International Review of Aerospace Engineering (IREASE), 12 (2), pp. 70-82.
https://doi.org/10.15866/irease.v12i2.15461

Cisco, Cisco Annual Internet Report (2018-2023) White Paper, Cisco, 2020.

Huawei, Intelligent Word 2030, Shenzhen, 2018. Accessed: Jul. 28, 2022. [Online]. Available: https://www-file.huawei.com/-/media/CORP2020/pdf/giv/Intelligent_World_2030_en.pdf.

Ciena Corporation - AnnualReports.com. (accessed Sep. 09, 2022).
https://www.annualreports.com/Company/ciena-corporation

H. Steck, L. Baltrunas, E. Elahi, D. Liang, Y. Raimond, and J. Basilico, Deep Learning for Recommender Systems: A Netflix Case Study, AI Mag., vol. 42, no. 3, pp. 7-18, Nov. 2021.
https://doi.org/10.1609/aimag.v42i3.18140

T. J. O'Shea and J. Hoydis, An Introduction to Machine Learning Communications Systems, Mach. Learn. Methods Ecol. Appl., pp. 1-37, 2017.
https://doi.org/10.1007/978-3-319-20010-1

N. M. Bernhardt Levin, V. Oliveira Pintro, M. Boff de Avila, B. Boldrini de Mattos, and W. Filgueira De Azevedo Jr., Understanding the Structural Basis for Inhibition of Cyclin-Dependent Kinases. New Pieces in the Molecular Puzzle, Curr. Drug Targets, vol. 18, no. 9, pp. 1104-1111, Jun. 2017.
https://doi.org/10.2174/1389450118666161116130155

OpenAI et al., Dota 2 with Large Scale Deep Reinforcement Learning, Dec. 2019.
https://doi.org/10.48550/arxiv.1912.06680

G. Dulac-Arnold, D. Mankowitz, and T. Hester, Challenges of Real-World Reinforcement Learning, Apr. 2019.
https://doi.org/10.48550/arxiv.1904.12901

G. Papoudakis, F. Christianos, and S. V. Albrecht, Agent Modelling under Partial Observability for Deep Reinforcement Learning, Adv. Neural Inf. Process. Syst., vol. 23, pp. 19210-19222, Jun. 2020.
https://doi.org/10.48550/arxiv.2006.09447

F. Christianos, L. Schäfer, and S. V. Albrecht, Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning, Adv. Neural Inf. Process. Syst., vol. 2020-December, Jun. 2020.
https://doi.org/10.48550/arxiv.2006.07169

T. Rashid, M. Samvelyan, C. S. de Witt, G. Farquhar, J. Foerster, and S. Whiteson, QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning, Mar. 2018.
https://doi.org/10.48550/arxiv.1803.11485

P. Sunehag et al., Value-Decomposition Networks For Cooperative Multi-Agent Learning, Jun. 2017.
https://doi.org/10.48550/arxiv.1706.05296

L. Zintgraf, S. Devlin, K. Ciosek, S. Whiteson, and K. Hofmann, Deep interactive Bayesian reinforcement learning via meta-learning, Proc. Int. Jt. Conf. Auton. Agents Multiagent Syst. AAMAS, vol. 3, pp. 1700-1702, 2021.

L. Velasco, A. P. Vela, F. Morales, and M. Ruiz, Designing, operating, and reoptimizing elastic optical networks, J. Light. Technol., vol. 35, no. 3, pp. 513-526, Feb. 2017.
https://doi.org/10.1109/JLT.2016.2593986

C. S. R. Murthy and M. Gurusamy, WDM optical networks : concepts, design, and algorithms, Prentice Hall PTR, vol. 1, p. 430, 2002, Accessed: Feb. 27, 2019. [Online]. Available:
https://books.google.com/books/about/WDM_Optical_Networks.html?hl=fr&id=du1SAAAAMAAJ

Aguirre, D., Velandia, J., Parra, O., Routing in Elastic Optical Networks Based on Deep Reinforcement Learning, (2022) International Review on Modelling and Simulations (IREMOS), 15 (1), pp. 47-52.
https://doi.org/10.15866/iremos.v15i1.21569

D. F. A. Moreno, O. J. S. Parra, and D. A. L. Sarmiento, Heuristic algorithm for flexible optical networks OTN, in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2018, vol. 10699 LNCS, pp. 163-172.
https://doi.org/10.1007/978-3-319-73830-7_17

A. Valadarsky, M. Schapira, D. Shahaf, and A. Tamar, Learning to route, HotNets 2017 - Proc. 16th ACM Work. Hot Top. Networks, pp. 185-191, Nov. 2017.
https://doi.org/10.1145/3152434.3152441

S. C. Lin, I. F. Akyildiz, P. Wang, and M. Luo, QoS-aware adaptive routing in multi-layer hierarchical software defined networks: A reinforcement learning approach, Proc. - 2016 IEEE Int. Conf. Serv. Comput. SCC 2016, pp. 25-33, Aug. 2016
https://doi.org/10.1109/SCC.2016.12

Danilo R. B. Araújo, NsfNet -Topologia com 14 nós e 21 enlaces. | Download Scientific Diagram. (accessed Mar. 12, 2019).
https://www.researchgate.net/figure/Figura-2-NsfNet-Topologia-com-14-nos-e-21-enlaces_fig2_307578174

V. Mnih et al., Human-level control through deep reinforcement learning, Nat. 2015 5187540, vol. 518, no. 7540, pp. 529-533, Feb. 2015.
https://doi.org/10.1038/nature14236

Z. Wang, T. Schaul, M. Hessel, H. Van Hasselt, M. Lanctot, and N. De Frcitas, Dueling Network Architectures for Deep Reinforcement Learning, 33rd Int. Conf. Mach. Learn. ICML 2016, vol. 4, pp. 2939-2947, Nov. 2015.
https://doi.org/10.48550/arxiv.1511.06581

T. Schaul, J. Quan, I. Antonoglou, and D. Silver, Prioritized Experience Replay, 4th Int. Conf. Learn. Represent. ICLR 2016 - Conf. Track Proc., Nov. 2015.
https://doi.org/10.48550/arxiv.1511.05952

F. I. Calderon et al., BER-Adaptive RMLSA Algorithm for Wide-Area Flexible Optical Networks, IEEE Access, vol. 8, pp. 128018-128031, 2020. doi: 10.1109/ACCESS.2020.3008883
https://doi.org/10.1109/ACCESS.2020.3008883

Gorshkova, K., Zueva, V., Kuznetsova, M., Tugashova, L., Optimizing Deep Learning Methods in Neural Network Architectures, (2021) International Review of Automatic Control (IREACO), 14 (2), pp. 93-101.
https://doi.org/10.15866/ireaco.v14i2.20591

Kaissari, S., El Attaoui, A., Benba, A., Jilbab, A., Bourouhou, A., Kaissari, A., PlanTech: Early Detection of Plant Disease Based on HWSN Using Deep Learning, (2021) International Journal on Engineering Applications (IREA), 9 (3), pp. 162-172.
https://doi.org/10.15866/irea.v9i3.20720

Pinzon-Arenas, J., Jimenez-Moreno, R., Pachon-Suescun, C., Place Recognition with DAG-CNN, (2020) International Review of Automatic Control (IREACO), 13 (2), pp. 58-66.
https://doi.org/10.15866/ireaco.v13i2.17053

Refbacks

There are currently no refbacks.

Username
Password
Remember me