A carregar...
Publicação
Encaminhamento por DRL com críticos centrais e locais: impacto das GNNs em cenários dinâmicos
| Nome: | Descrição: | Tamanho: | Formato: | |
|---|---|---|---|---|
| 25.54 MB | Adobe PDF |
Autores
Orientador(es)
Resumo(s)
As redes modernas apresentam um dinamismo crescente e uma complexidade que
desafiam as abordagens tradicionais de encaminhamento. Os algoritmos convencionais,
como o shortest path, mostram-se limitados na otimização global do tráfego e na gestão
eficiente dos recursos da rede. Neste contexto, o Deep Reinforcement Learning (DRL) surge
como uma abordagem promissora, capaz de aprender políticas de encaminhamento
adaptativas a partir da experiência direta com o ambiente.
Este trabalho propõe um sistema baseado no algoritmo Multi Agent Deep Deterministic
Policy Gradient (MADDPG), com arquiteturas de críticos centralizados e locais, bem como
a integração de Graph Neural Networks (GNNs) para reforçar a generalização topológica.
A avaliação foi realizada numa topologia realista de um Service Provider, em cenários que
incluíram falhas críticas e variações de tráfego.
Os resultados demonstraram que o DRL supera consistentemente o Shortest Path, o
que resulta numa maior eficiência na utilização da rede e menores perdas de pacotes. A
arquitetura Simple Q Network destacou-se pela sua adaptabilidade em cenários dinâmicos
quando não foi utilizada Graph Neural Network (GNN); em contraste, com a sua integração,
as arquiteturas Duelling beneficiaram substancialmente, com o Local Critic a aproximar-se
do desempenho centralizado. Contudo, observou-se, que a reaprendizagem contínua com
GNN pode introduzir alguma rigidez, limitando a adaptação em certos cenários.
Em síntese, o estudo confirma o potencial do DRL, em combinação com GNNs e abor-
dagens multi-agent, como solução escalável, adaptativa e resiliente para o encaminhamento
em redes modernas.
Modern networks are increasingly dynamic and complex, challenging traditional routing approaches. Conventional algorithms, such as shortest path, are limited in their ability to optimize traffic globally and manage network resources efficiently. In this context, Deep Reinforcement Learning (DRL) emerges as a promising approach, capable of learning adaptive routing policies from direct experience with the environment. This work proposes a system based on the Multi Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, with centralized and local critic architectures, as well as the integration of Graph Neural Networks (GNNs) to reinforce topological generalization. The evaluation was performed on a realistic topology of a service provider, in scenarios that included critical failures and traffic variations. The results demonstrated that DRL consistently outperforms shortest path, resulting in greater network utilization efficiency and lower packet loss. The Simple Q Network architecture stood out for its adaptability in dynamic scenarios when GNN was not used; in contrast, with its integration, the Duelling architectures benefited substantially, with Local Critic approaching centralized performance. However, it was observed that continuous relearning with GNN can introduce some rigidity, limiting adaptation in certain scenarios. In summary, the study confirms the potential of DRL, in combination with GNNs and multi-agent approaches, as a scalable, adaptive, and resilient solution for routing in modern networks.
Modern networks are increasingly dynamic and complex, challenging traditional routing approaches. Conventional algorithms, such as shortest path, are limited in their ability to optimize traffic globally and manage network resources efficiently. In this context, Deep Reinforcement Learning (DRL) emerges as a promising approach, capable of learning adaptive routing policies from direct experience with the environment. This work proposes a system based on the Multi Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, with centralized and local critic architectures, as well as the integration of Graph Neural Networks (GNNs) to reinforce topological generalization. The evaluation was performed on a realistic topology of a service provider, in scenarios that included critical failures and traffic variations. The results demonstrated that DRL consistently outperforms shortest path, resulting in greater network utilization efficiency and lower packet loss. The Simple Q Network architecture stood out for its adaptability in dynamic scenarios when GNN was not used; in contrast, with its integration, the Duelling architectures benefited substantially, with Local Critic approaching centralized performance. However, it was observed that continuous relearning with GNN can introduce some rigidity, limiting adaptation in certain scenarios. In summary, the study confirms the potential of DRL, in combination with GNNs and multi-agent approaches, as a scalable, adaptive, and resilient solution for routing in modern networks.
Descrição
Palavras-chave
DRL Multi-Agent Resiliência a Falhas Críticos Centrais e Locais GNNs