Effectiveness in Retrieving Legal Precedents

Abstract

This study examines the interplay between text summarization techniques and embeddings from Language Models (LMs) in constructing expert systems dedicated to the retrieval of legal precedents, with an emphasis on achieving cost-efficiency. Grounded in the growing domain of Artificial Intelligence (AI) in law, our research confronts the perennial challenges of computational resource optimization and the reliability of precedent identification. Through Named Entity Recognition (NER) and part-of-speech (POS) tagging, we juxtapose various summarization methods to distill legal documents into a convenient form that retains their essence. We investigate the effectiveness of these methods in conjunction with state-of-the-art embeddings based on Large Language Models (LLMs), particularly ADA from OpenAI, which is trained on a wide range of general-purpose texts. Utilizing a dataset from one of Brazil’s administrative courts, we explore the efficacy of embeddings derived from a Transformer model tailored to legal corpora against those from ADA, gauging the impact of parameter size, training corpora, and context window on retrieving legal precedents. Our findings suggest that while the full text embedded with ADA’s extensive context window leads in retrieval performance, a balanced combination of POS-derived summaries and ADA embeddings presents a compelling trade-off between performance and resource expenditure, advocating for an efficient, scalable, intelligent system suitable for broad legal applications. This study contributes to the literature by delineating an optimal approach that harmonizes the dual imperatives of computational frugality and retrieval accuracy, propelling the legal field toward more strategic AI utilization.