Bação, Fernando José Ferreira LucasFigueiredo, Fernando Niglio de2024-11-262024-11-262024-10-24http://hdl.handle.net/10362/175829Dissertation presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Information Systems and Technologies ManagementPre-Trained Models (PTMs) or Large Language Models (LLMs) are deep neural networks trained on vast amounts of text data, enabling them to make predictions based on learned knowledge. Google has played a significant role in this field, particularly through popularizing the Transformers Architecture. However, the landscape evolved dramatically with the release of ChatGPT by OpenAI in November 2022, marking the advent of the universal artificial intelligence era. This event sparked significant interest and efforts in studying LLMs, prompting industries to adapt their operations, software providers to refine their skills, and society to contemplate ethical implications. This research delves into the use of Open-Source LLMs, focusing particularly on text classification — a critical task in Natural Language Processing (NLP). The study employs techniques such as fine-tuning and model quantization, which are essential for leveraging LLMs effectively in practical applications. Key questions addressed include evaluating the comparability of open-source models with established benchmarks across different text classification approaches. The research aims to identify primary challenges and limitations associated with running modern open-source LLMs in lowresource environments. By exploring these topics, the research aims to contribute insights into optimizing the deployment of open-source LLMs, enhancing their accessibility, and addressing practical constraints that affect their widespread adoption across various sectors and applications in NLP.engLarge Language Models (LLM)Pre-trained models (PTM)Text classificationNews classificationFine-tuningOpen SourceLoRAQLoRAlow-resource environmentSDG 8 - Decent work and economic growthmaster thesis203785142