A carregar...
Projeto de investigação
New Generation of data STORage And manaGement systEms
Financiador
Autores
Publicações
Overlay Networks for Edge Management
Publication . Costa, Pedro Akos; Fouto, Pedro; Leitão, João; DI - Departamento de Informática; NOVALincs
Edge computing has emerged as a solution to address existing limitations of cloud computing for bandwidth-heavy and time-sensitive applications, by moving (some) computations from bandwidth saturated Cloud infrastructures closer to client devices, where data is effectively produced and consumed. However, existing materializations of the edge computing paradigm take limited advantage of computational and storage power that exists in the edge and between client devices and the cloud. Most of these leverage static hierarchical topologies (e.g., Fog Computing) to pre-process data before sending it to the Cloud, which limits the advantages that can be extracted from the edge computing paradigm. In the past, peer-to-peer systems have sought to tackle the challenges of increasing scalability and availability for very large systems, with a large number of solutions being proposed namely, distributed overlay networks for resource management. In this paper, we argue that the clever adaptation of peer-to-peer solutions can enable novel applications to fully exploit the potential of the edge. In particular, we study the viability of taking advantage of specialized overlay networks in edge environments to enable the management of a large number of computational resources. Contrary to previous proposals, that assume the environment to be composed of mostly homogeneous devices, our proposal embraces existing heterogeneity and exploits the location of computational resources to devise a (partially) self-organizing overlay network that can be exploited both to provide membership information to applications, but also do efficiently disseminate management information across edge devices. We have conducted an experimental evaluation using container-based emulation in an heterogeneous network composed by 100 devices, with results showing that our protocol is able to maximize the bandwidth usage of the system, allowing more data to flow throughout the network, while retaining high robustness to failures.
Enriching Kademlia by Partitioning
Publication . Monteiro, João; Costa, Pedro Akos; Leitão, João; De La Rocha, Alfonso; Psaras, Yiannis; DI - Departamento de Informática; NOVALincs
Decentralizing the Web is becoming an increasingly interesting endeavor that aims at improving user security and privacy as well as providing guaranteed ownership of content. One such endeavor that pushes towards this reality, is Protocol Labs' Inter-Planetary File System (IPFS) network, that provides a decentralized large scale file system to support the decentralized Web. To achieve this, the IPFS network leverages the Kademlia DHT to route and store pointers to content stored by network members (i.e., peers). However, due to the large number of network peers, content, and accesses, the DHT routing needs to be efficient and quick to enable a decentralized web that is competitive. In this paper, we present work in progress that aims at improving the Kademlia DHT performance through the manipulation of DHT identifiers by adding prefixes to identifiers. With this, we are able to bias the DHT topological organization towards locality (which can be either geographical or applicational), which creates partitions in the DHT and enables faster and more efficient query resolution on local content. We designed prototypes that implement our proposal, and performed a first evaluation of our work in an emulated network testbed composed of 5000 nodes. Our results show that our proposal can benefit the DHT look up on data with locality with minimal overhead.
Rethinking Distributed Caching Systems Design and Implementation
Publication . Cordeiro, Miguel da Silva de Brito; Leitão, João; Duarte, Vitor
Distributed caching systems based on in-memory key-value stores have become a
crucial aspect of fast and efficient content delivery in modern web-applications. However,
due to the dynamic and skewed execution environments and workloads, under which
such systems typically operate, several problems arise in the form of load imbalance.
This thesis addresses the sources of load imbalance in caching systems, mainly: i) data
placement, which relates to distribution of data items across servers and ii) data item
access frequency, which describes amount of requests each server has to process, and how
each server is able to cope with it. Thus, providing several strategies to overcome the
sources of imbalance in isolation.
As a use case, we analyse Memcached, its variants, and propose a novel solution for
distributed caching systems. Our solution revolves around increasing parallelism through
load segregation, and solutions to overcome the load discrepancies when reaching high
saturation scenarios, mostly through access re-arrangement, and internal replication.
Unidades organizacionais
Descrição
Palavras-chave
Contribuidores
Financiadores
Entidade financiadora
Fundação para a Ciência e a Tecnologia
Programa de financiamento
3599-PPCDT
Número da atribuição
PTDC/CCI-INF/32038/2017