Publicação
Lakehouse Data Architecture: Data as a first-class citizen within an organization
| datacite.subject.fos | Ciências Naturais::Ciências da Computação e da Informação | pt_PT |
| dc.contributor.advisor | Pinheiro, Flávio Luís Portas | |
| dc.contributor.author | Lopes, Fábio Rafael Santos | |
| dc.date.accessioned | 2024-03-19T11:24:33Z | |
| dc.date.available | 2024-03-19T11:24:33Z | |
| dc.date.issued | 2024-02-05 | |
| dc.description | Internship Report presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics, specialization in Data Science | pt_PT |
| dc.description.abstract | This thesis presents an in-depth exploration of the Lakehouse Data Architecture. This paradigm merges the strengths of data lakes and data warehouses, enabling organizations to harness the full potential of their data. The research investigates the architectural components, operational mechanisms, and strategic implications of implementing a Lakehouse within an organization using advanced technologies like Microsoft Azure, Google Cloud Platform, Databricks, Apache Spark, Delta Lake, and Dremio. The study also scrutinizes the Lakehouse's ability to facilitate a data-centric culture by integrating advanced analytics into business processes. The thesis further delves into the FAIR data principles, advocating for data to be Findable, Accessible, Interoperable, and Reusable, and the Data Mesh concept, a decentralized data management approach. The research concludes that the Lakehouse architecture provides a comprehensive and robust framework for managing vast and diverse data sets, optimizing data pipeline performance, reducing redundancy, and enhancing data security. It underscores the pivotal role of the Lakehouse in driving strategic innovation and positions it as a flexible and adaptable model for future technological advancements in AI and machine learning. The insights offered in this thesis serve as a guide for organizations aiming to navigate the complexities of becoming data-centric and underscore the transformative power of modern data platforms. | pt_PT |
| dc.identifier.tid | 203553047 | pt_PT |
| dc.identifier.uri | http://hdl.handle.net/10362/165104 | |
| dc.language.iso | eng | pt_PT |
| dc.rights.uri | http://creativecommons.org/licenses/by/4.0/ | pt_PT |
| dc.subject | Data Mesh | pt_PT |
| dc.subject | Databricks | pt_PT |
| dc.subject | Data Lakehouse | pt_PT |
| dc.subject | Delta Lake | pt_PT |
| dc.subject | Microsoft Azure | pt_PT |
| dc.subject | Apache Spark | pt_PT |
| dc.subject | SDG 9 - Industry, innovation and infrastructure | pt_PT |
| dc.title | Lakehouse Data Architecture: Data as a first-class citizen within an organization | pt_PT |
| dc.type | master thesis | |
| dspace.entity.type | Publication | |
| rcaap.rights | openAccess | pt_PT |
| rcaap.type | masterThesis | pt_PT |
| thesis.degree.name | Mestrado em Ciência de Dados e Métodos Analíticos Avançados, especialização em Ciência de Dados | pt_PT |