A carregar...
Publicação
Open-Source Language Models for News Classification: Implementing Small Models in Low-Resource Environments
| Nome: | Descrição: | Tamanho: | Formato: | |
|---|---|---|---|---|
| 1.37 MB | Adobe PDF |
Autores
Orientador(es)
Resumo(s)
Pre-Trained Models (PTMs) or Large Language Models (LLMs) are deep neural networks trained on vast amounts of text data, enabling them to make predictions based on learned knowledge. Google has played a significant role in this field, particularly through popularizing the Transformers Architecture. However, the landscape evolved dramatically with the release of ChatGPT by OpenAI in November 2022, marking the advent of the universal artificial intelligence era. This event sparked significant interest and efforts in studying LLMs, prompting industries to adapt their operations, software providers to refine their skills, and society to contemplate ethical implications. This research delves into the use of Open-Source LLMs, focusing particularly on text classification — a critical task in Natural Language Processing (NLP). The study employs techniques such as fine-tuning and model quantization, which are essential for leveraging LLMs effectively in practical applications. Key questions addressed include evaluating the comparability of open-source models with established benchmarks across different text classification approaches. The research aims to identify primary challenges and limitations associated with running modern open-source LLMs in lowresource environments. By exploring these topics, the research aims to contribute insights into optimizing the deployment of open-source LLMs, enhancing their accessibility, and addressing practical constraints that affect their widespread adoption across various sectors and applications in NLP.
Descrição
Dissertation presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Information Systems and Technologies Management
Palavras-chave
Large Language Models (LLM) Pre-trained models (PTM) Text classification News classification Fine-tuning Open Source LoRA QLoRA low-resource environment SDG 8 - Decent work and economic growth