Open Data as a Catalyst for the Evolution of Smart Cities

Abstract

Cities today face a range of common challenges, including environmental degradation, climate change, population growth, and resource depletion. These challenges have adverse social, economic, and environmental consequences, requiring urgent attention and action. Smart cities initiatives are crucial in addressing these challenges through their transformative role in integrating technological, organizational, and political innovations to create intelligent solutions for governance, economy, mobility, environment, living, and people. Smart cities are data-driven ecosystems, and their open data initiatives are portrayed as a means to enhance governance, citizen engagement, and innovation, as a tool to address socioeconomic and environmental problems. These initiatives emphasize the importance of data in this transformation, with governments urged to promote openness and transparency by making data more accessible. However, despite the recognized importance of open data for urban development, there is a gap in systematic research on the impact evaluation of open data initiatives within the smart city context. The first study of this dissertation aims to address this gap by proposing a theoretical framework designed to serve as a comprehensive tool for evaluating and monitoring the impacts of open data initiatives within the context of smart cities. The framework is structured to provide a systematic approach to assess how open data can influence various aspects of smart city development, including economic opportunities, governance, citizen empowerment, and the resolution of complex public problems. The framework comprises a conceptual model and an experiment that employs Randomized Controlled Trials (RCTs) to offer a detailed view of the context and characteristics of open data impacts. It includes components such as problem and demand definition, capacity and culture, governance, partnerships, and risks, which are essential for understanding the multifaceted nature of open data initiatives while encompassing smart city dimensions to ensure a holistic evaluation. Moreover, the framework's design reflects a commitment to continual improvement, advocating for a cyclical testing process, learning, and adapting open data policies and interventions based on empirical evidence. The second study empirically tests the framework, exploring the impact of open data in the context of urban development and smart cities, specifically through real estate price prediction, with a specific case study on Lisbon's housing market from 2018 to 2021. The methodology employed in this study involves integrating proprietary data and open data sources into an XGBoost machine learning (ML) model, which is optimized using the Optuna hyperparameter framework. The study also compared the baseline and open data-enhanced models to assess the added value of open data. The models’ performance was evaluated using a range of metrics, including the mean absolute error (MAE), which was significantly reduced by 8.24% after incorporating open data features. For interpretability, SHapley Additive exPlanations (SHAP) were employed to analyze the predictions and understand the importance of features and their interactions. By integrating explainable artificial intelligence (XAI) into urban data analysis, this model is proposed to enhance real estate predictions' accuracy and ensure transparency and accountability in urban development processes. This dual focus bridges the gap between data science and urban management, offering novel insights into optimizing city planning and policymaking through the strategic leverage of open data. This dissertation advances the field through several contributions: it provides a systematic literature review on the impact of open data initiatives on smart cities, which is a timely effort given the lack of existing robust frameworks for such evaluations. The review delineates the current research landscape, identifies existing gaps, and promotes the alignment of open data supply and demand to meet citizens' expectations better. Additionally, this research proposes a theoretical framework that includes a conceptual model and an experiment employing Randomized Controlled Trials (RCTs), offering a structured approach to evaluate the effectiveness of open data policies and their impact on smart cities. It bridges a critical research gap and suggests a method for assessing impacts, thereby enriching the theoretical and practical understanding of leveraging open data in smart city. environments. Furthermore, this dissertation explores the use of advanced predictive tools in the real estate market, measuring and quantifying how open data enhances the accuracy of predictive models. It highlights the emerging role of XAI in improving model transparency and interpretability, contributing to more informed policy decisions for the development of sustainable smart cities.