Avaliação de Grandes Modelos de Linguagem para Raciocínio em Direito Tributário

• Ano de defesa: 2024
• Autor(a) principal: Presa, João Paulo Cavalcante
• Orientador(a): Camilo Junior, Celso Gonçalves
• Banca de defesa: Camilo Júnior, Celso Gonçalves, Oliveira, Sávio Salvarino Teles de, Silva , Nádia Felix Felipe da, Leite, Karla Tereza Figueiredo
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• dARK ID: ark:/38995/001300000gchh
• Idioma: por
• Instituição de defesa: Universidade Federal de Goiás
• Programa de Pós-Graduação: Programa de Pós-graduação em Ciência da Computação (INF)
• Departamento: Instituto de Informática - INF (RMG)
• País: Brasil
• Palavras-chave em Português: Direito Tributário; Raciocínio Jurídico; Grandes Modelos de Linguagem (LLM); Processamento de Linguagem Natural
• Palavras-chave em Inglês: Tax Law; Legal Reasoning; Large Language Models (LLMs); Natural Language Processing
• Área do conhecimento CNPq: CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO::METODOLOGIA E TECNICAS DA COMPUTACAO
• Link de acesso: http://repositorio.bc.ufg.br/tede/handle/tede/13871

Tax law is essential for regulating relationships between the State and taxpayers, being crucial for tax collection and maintaining public functions. The complexity and constant evolution of tax laws make their interpretation an ongoing challenge for legal professionals. Although Natural Language Processing (NLP) has become a promising technology in the legal field, its application in brazilian tax law, especially for legal entities, remains a relatively unexplored area. This work evaluates the use of Large Language Models (LLMs) in Brazilian tax law covering federal tax aspects, analyzing their ability to process questions and generate answers in Portuguese for legal entities’ queries. For this purpose, we built an original dataset composed of real questions and answers provided by experts, allowing us to evaluate the ability of both proprietary and open-source LLMs to generate legally valid answers. The research uses quantitative and qualitative metrics to measure the accuracy and relevance of generated answers, capturing aspects of legal reasoning and semantic coherence. As contributions, this work presents a dataset specific to the tax law domain, a detailed evaluation of different LLMs’ performance in legal reasoning tasks, and an evaluation approach that combines quantitative and qualitative metrics, thus advancing the application of artificial intelligence in the analysis of tax laws and regulations.