Balancing trust and utility in Large Language Models: a comprehensive trade-off analysis of key performance metrics

Abstract

This thesis investigates biases in Large Language Models (LLMs) by analyzing their responses to knowledge- and reasoning-based prompts, evaluating bias evolution across selected models. Persistent biases in knowledge-based prompts are linked to skewed data and hallucinations, while reasoning-based prompts reveal context-dependent systemic inequities. Larger text-to-text models often enhance accuracy but may amplify biases, whereas targeted interventions in text-to-image models show modest bias reductions, reflecting industry efforts to improve representation. The trade-off analysis emphasizes domain-specific LLM deployment, balancing fairness, reliability, and utility for equitable and effective AI applications. Focusing on trust-utility trade-offs, this study examines LLM performance across Truthfulness, Safety, Fairness, Robustness, Privacy, and Machine Ethics. The research uncovers synergies and conflicts among these metrics. Results identify Truthfulness as key to utility, revealing significant trade-offs in fairness, safety, and privacy dimensions. The study highlights the need for transparent trade-off management, offering insights to develop ethical, reliable, and high-performing LLMs for diverse applications.