FCT: DI - Dissertações de Mestrado
URI permanente para esta coleção:
Navegar
Entradas recentes
- Gesture classification for assembly tasks with event camerasPublication . Estróia, Pedro Afonso Fulgêncio; Soares, Cláudia; Valdeira, Filipa; Mendes, NunoIn recent years, the use of collaborative robots in manufacturing environments has in- creased significantly, enabling humans and robots to work together in shared environments. To ensure efficient cooperation, robots must be capable of detecting and predicting the actions performed by human operators. Traditionally, this detection is performed us- ing frame-based cameras (e.g., RGB cameras). These cameras capture a vast number of frames, requiring substantial bandwidth and processing time, which is a limitation in real-time detection of human gestures. Event cameras have emerged as a promising alternative, as they only capture changes in luminosity, significantly reducing storage and processing requirements. However, they remain an understudied field in the context of manufacturing. We propose a gesture classification model for assembly tasks using data captured with an event camera. This event data is from the Event-based Dataset of Assembly Tasks (EDAT24), which contains recordings of primitive tasks (such as picking, placing, and screwing) in a simulated industrial environment. First, with the goal of comparing between frames and events, we built a frame-based model (CNN–LSTM) trained on conventional frames and event frames created by aggregating events over short intervals—we use identical architectures and training to isolate the effect of the input representation. Then, we implemented an event-based PointNet++ pipeline that treats each video as a sequence of 3D point clouds of events, extracts the features hierarchically from each cloud, and then aggregates the classifications. We evaluated each of these models using several metrics. Frame-based and aggregated- event CNN–LSTM variants achieved very similar performance, with both reaching an accuracy of 84%, indicating that aggregation of events does not bring improvements over standard frames. In contrast, the PointNet++ pipeline substantially outperformed both, reaching an accuracy of 97%, while avoiding event-to-frame conversion and better exploiting the temporal structure of event data. Therefore, this research suggests the use of event data for classification of assembly tasks in manufacturing environments.
- Desenvolvimento de um módulo de visualização para simulador da florestaPublication . Amorim, João Miguel Lobão Santos Rodrigues; Birra, Fernando; Pires, João; Tomé, MargaridaAs florestas desempenham um papel central na biodiversidade, na regulação do clima e no desenvolvimento económico e social. Contudo, enfrentam crescente pressão das alterações climáticas, da desflorestação e da intensificação das atividades humanas. Em Portugal, são um recurso económico relevante, com impacto em setores como a produção de madeira e cortiça, cujo potencial depende de uma gestão florestal eficiente e sustentável. Neste contexto, as ferramentas de apoio à gestão florestal assumem uma importância cada vez maior, sobretudo quando disponibilizadas através de interfaces com elevados níveis de usabilidade e acessibilidade. Em Portugal, existem ferramentas dedicadas à gestão florestal, como o simulador de povoamentos standsSIM 1, que permite a simulação da evolução dos povoamentos sob diferentes regimes de gestão florestal. No entanto, estas ferramentas carecem, em geral, de componentes de visualização que facilitem a interpretação dos dados de simulação. A ausência de representações gráficas adequadas limita a compreensão dos resultados por parte dos utilizadores, comprometendo a sua eficácia na tomada de decisão. Este trabalho apresenta uma solução para este problema, através do desenvolvimento de um módulo de visualização para o simulador standsSIM 1. Este módulo permite a visualização tridimensional da evolução dos povoamentos ao longo do tempo, utilizando modelos 3D de árvores, bem como a apresentação analítica dos principais indicadores de gestão florestal. Com a abordagem efetuada, pretende-se melhorar a usabilidade e acessibilidade das ferramentas de apoio à gestão florestal, enriquecendo o processo de análise das opções de gestão. Espera-se, assim, contribuir para uma melhor compreensão dos dados de simulação e, consequentemente, para uma tomada de decisão mais informada e eficaz no domínio da gestão florestal.
- A Flexible Tool for Monitoring System CallsPublication . Fidalgo, Tiago Jardim; Resende, João; Martins, InêsIn recent years, cybersecurity threats have been evolving, making real-time threat detection and response critical for securing modern computing environments. Host-based security monitoring is vital in capturing detailed telemetry data, such as system calls and process activities, providing insight into potential security incidents. However, existing telemetry collection solutions often lack the depth and integration required for valuable real-world security analysis. This research proposes a comprehensive security telemetry collection framework that integrates host-based data collection tools with a monitoring system designed to track system calls and processes as key structural and functional units of a system. Additionally, our approach incorporates a security environmental test, simulating real-world threats and vulnerabilities to generate a complete, practical, and real-time host- based security dataset. To enhance the usability of collected data, we also integrate a data visualization dashboard, facilitating more effective analysis and interpre- tation of security metrics. To evaluate the effectiveness of our proposed solution, we compare our security telemetry collection approach, using Miningbeat, with Metricbeat. This comparison is conducted through simulated attacks based on the Red Team Atomic Testing framework, replicating various adversarial tactics. By an- alyzing the distinct sets of metrics collected by each tool and their behavior during different attack scenarios, we assess the capabilities of our approach in providing enhanced security telemetry for host-based threat detection. This thesis aims to contribute to security telemetry collection by demonstrating the advantages of a more integrated approach. Our findings offer insights into how host-based mon- itoring can be optimized for real-time threat detection and response, ultimately improving the security posture of modern computing systems.
- Dynamic Substitutional Reality In Adaptive Virtual ExperiencesPublication . Figueira, Guilherme Ribeiro; Nóbrega, RuiThis dissertation aims to enable the creation of Substitutional Reality (SR) experiences that dynamically adapt to unknown spaces. A key limitation of Substitutional Reality (SR) today is its static deployment: most experiences are designed for a specific physical environment, preventing portability and restricting broader adoption. Researching the practical applications of SR and its advantages in certain fields is also important to encourage further research. To address these challenges, this work explores techniques for developing interactive SR experiences that adapt to arbitrary rooms while supporting the integration of narrative and user goals. A dynamic SR system was developed, which extends existing scene-understanding capabilities of modern Head-Mounted Displays (HMDs) with a runtime substitution pipeline. This pipeline maps physical objects to predefined virtual counterparts, allowing experiences to run across heterogeneous environments without manual calibration. Ad- ditionally, the system introduces versatile screen-space portals that serve two purposes: enabling seamless transitions between multiple substitutional environments and inte- grating video see-through Augmented Reality (AR) for a stronger connection to the real world. Designers can therefore construct SR experiences that combine portability, tangible interaction, and hybrid AR/SR configurations. To evaluate the system, a conceptual museum experience was implemented as a case study. This application demonstrated how Dynamic SR can support cultural heritage use cases, combining interpretive content and interactive artifacts with adaptability to different physical spaces. Evaluations included user studies on how substitution and passthrough integration affect locomotion and engagement, an investigation of portal interactions within the museum scenario, and adaptability tests assessing performance across diverse environments. The results validate the system’s effectiveness in creating adaptive SR experiences and demonstrate its applicability to cultural heritage contexts.
- Causal Inference Explanations for Graph Neural NetworksPublication . Kumar, Sahil Satish; Soares, Cláudia; Caldas, FranciscoThe increasing popularity of Artificial Intelligence, notably Deep Learning, has prompted its application across diverse real-world scenarios. Graph Neural Networks, a type of Deep Learning model developed for graph-structured data, have gained significant popu- larity owing to their versatility and efficacy. Despite their high accuracy and performance, contemporary Deep Learning models often function as black box systems, lacking trans- parency and trustworthiness crucial for deployment in critical decision-making domains. In response, Explainable Artificial Intelligence has emerged, aiming to enhance the trustworthiness of black box models by devising explanation methods that clarify their inner workings. However, prevalent explanation techniques predominantly leverage correlation and association rather than employing causality, a significant aspect of human comprehension. In this thesis, we propose a novel explanation method using causal inference developed specifically for Graph Neural Networks. Our approach seeks to illuminate the decision- making process of Graph Neural Networks, thereby augmenting their transparency and trustworthiness. To show the generality of our approach we do a benchmark evaluation of our method against state-of-the-art explanation methods on a variety of datasets, tasks and Graph Neural Networks architectures. Furthermore, we apply and evaluate our explanation method in two real-world use cases: medical referral recommendation and space object conjunction prediction. In both cases, we demonstrate the applicability of our method and discuss its limitations.
- Large Language Model for Querying Databases in PortuguesePublication . Figueiredo, Lourenço Maria Morais da Silva Pinto de; Marques, Nuno; Cavique, LuísThis dissertation presents a system designed to generate SQL queries from the Portuguese natural language, using Cedis’ sports facility management application ESPORT.IA as a case study. The system’s performance was validated with real-world client queries, focusing on its ability to interpret synonyms and handle complex query requirements. Experiments highlighted challenges such as model hallucination and time interval pro- cessing, emphasizing the need for precise context in improving query accuracy. A custom benchmark, ESPORTNL2SQL, based on the Spider SQL Hardness Criteria, was developed to evaluate the system across varying query difficulties. For the benchmark’s first version, the GPT-4 Turbo and GPT-4o scored 88.79% and 80.17%. For the second version, they scored 75.00% and 80.17% respectively. Results gave evidence that adding contextual information improves accuracy. While the system shows potential in simplifying database interactions through natural language, limitations exist, particularly in managing larger contexts without degrading performance. This research offers insights into the practical application of NL2SQL systems in enterprise environments and their potential for broader deployment.
- Automated Data Extraction from Documents Using AI: Response Automation and Integration in Document Management SystemsPublication . Silva, Afonso Pimenta de Oliveira e; Silva, Joaquim; Campos, Adriano; Roque, MiguelOrganizations are increasingly challenged by the need to process large volumes of un- structured documents such as invoices, receipts, and contracts. Traditional rule-based approaches often fail to capture the variability and complexity of such data, leading to inefficiencies and high error rates. This thesis investigates two AI-driven strategies for automated information extraction from unstructured documents. The first combines Optical Character Recognition (OCR) through Tesseract with a Large Language Model (LLaMA) to identify and extract key fields from recognized text. The second employs Donut, a transformer-based vision-language model designed for end-to-end document understanding, which directly maps document images to structured outputs without an intermediate OCR step. A comparative evaluation is conducted across multiple mea- sures like character and word error rates, extraction accuracy, latency, and scalability to assess the trade-offs between modular and end-to-end pipelines. The results highlight the strengths and limitations of each method, providing insights into the practical deployment of intelligent document processing systems. This work contributes to advancing the field by offering a systematic comparison of OCR-LLM pipelines versus transformer-based end-to-end models for real-world unstructured document processing.
- Ferramenta para conversão de linguagem natural em açõesPublication . Lopes, Rita Gomes; Carvalho, Alberto; Medeiros, PedroA rápida digitalização de serviços aumentou a necessidade de ferramentas capazes de compreender instruções em linguagem natural e de realizar ações em conformidade. Con- tudo, soluções tradicionais, baseadas em regras rígidas, revelam-se insuficientes perante interações complexas, comprometendo a eficácia do apoio ao cliente e a experiência do utilizador. Neste contexto, esta dissertação apresenta um sistema conversacional avançado, focado na gestão de marcações em serviços consulares digitais e integrável em aplicações web, nomeadamente nas soluções da Opensoft. A solução é composta por três componentes principais: um agente conversacional, um servidor e uma interface de utilizador. O agente foi desenvolvido com recurso a um Large Language Model (LLM) e à framework LangChain. O servidor, utilizando FastAPI, assegura a comunicação com a interface de utilizador, sendo também responsável pela invocação do LLM. A interface, desenvolvida em React, disponibiliza funcionalidades como internacionalização, gestão dinâmica de tema e mecanismos de feedback, proporcionando uma experiência de utilização moderna. A solução foi ainda alvo de avaliações da sua robustez e fiabilidade, através de testes manuais, unitários, de integração e end-to-end, realizados com recurso a ferramentas como Pytest e RAGAs. Estes testes asseguraram a utilização correta das ferramentas pelo agente, a consistência das respostas produzidas e a conformidade com os requisitos de segurança definidos. Os resultados demonstram que a integração de LLMs em agentes conversacionais com acesso a ferramentas ultrapassa as limitações de chatbots tradicionais, permitindo melhores interpretações de linguagem natural, execuções de ações fiáveis e respostas ajustadas ao contexto. Além disso, a arquitetura modular e escalável desenvolvida constitui uma base sólida para adaptação a diferentes domínios, contribuindo para modernizar o atendimento digital e melhorar a experiência do utilizador.
- Autoformalização: Inteligência artificial generativa na formalização de teoremasPublication . Atalaia, Bernardo João Ferreira; Marques, Nuno; Janota, Mikoláš; Araujo, JoãoA autoformalização é a tarefa de traduzir automaticamente conteúdos matemáticos escritos em linguagem natural para expressões em linguagem formal. Este processo tem ganho força com os avanços recentes nos Grandes Modelos de Linguagem Large Language Models (LLMs), que demonstram crescente proficiência tanto em linguagens naturais como formais. No entanto, os LLMs, por si só, ainda não são capazes de fornecer autoformalização de forma consistente e fiável, sobretudo em domínios matemáticos especializados. Este trabalho propõe uma framework integrada que combina LLMs e a ferramenta ProverX para automatizar o processo de geração automática de datasets para autoforma- lização em Prover9/Mace4. Partindo de dados formais, esta ferramenta abrange as etapas de informalização, formalização e validação, afim de gerar as respetivas representações em linguagem natural dos dados iniciais. Em testes realizados sobre um dataset de 438 classes matemáticas, observou-se que ajustes nos hiperparâmetros e no input da LLM melhoraram significativamente os resultados, alcançando 346 informalizações válidas e até 108 classes válidas em uma única execução. A principal contribuição consiste em demonstrar a viabilidade prática da geração automática de datasets assistida por LLMs, disponibilizando uma framework funcional e extensível para a criação de pares de teoremas e classes matemáticas em representações formais e informais. Esta abordagem estabelece uma base para pesquisas futuras em autoformalização em larga escala, tirando partido da integração com a ferramenta ProverX.
- A Textbook of Verified OCaml Programs. A deductive study on algorithms and data structuresPublication . Gasparinho, Pedro Ramos; Pereira, MárioWe live in a society that has an ever-increasing dependency on critical software. Even minor programming oversights can have disastrous consequences, ranging from endangering human and animal lives to economic loss. Thus, it is crucial to verify the correctness of such programs, which can be achieved with deductive verification techniques. However, the teaching of such techniques is hampered by the small number of bibliographical works in this field. Our goal with this work is ambitious: to write a textbook on verified algorithms and data structures, fully automated or as close to it as possible. We believe that OCaml is the perfect programming language choice for this endeavour, due to its multi-paradigm nature, and its status as a general-purpose language with use cases in both academia and the industry. And, most importantly, it has good support for automated deductive verification with Cameleer and GOSPEL. The current version of the textbook includes more than 1000 lines of verified OCaml code spread across 6 chapters and more than 40 programs. Moreover, we offer a blend of functional and imperative case studies, and, overall, a substantial subset of constructs found in OCaml, including its module system. In this document, we discuss our thought process and design decisions behind the creation of a textbook on verified OCaml algorithms. Namely, identifying which classes of algorithms to tackle, which algorithms were chosen within those classes, and how to orga- nize the contents of the textbook, to name a few topics. Additionally, this is accompanied by an overview of theoretical and practical concepts concerning programming languages, algorithm analysis, deductive verification, and the selected tools.