Using large language models for legal decision-making in Austrian value-added tax law: a comparative study

Background: Tax consulting clients often describe cases in natural language, making large language models (LLMs) attractive for supporting legal decision-making in Austrian and EU value-added tax (VAT) law. However, the requirement for legally grounded, well-justified analyses is challenged by LLMs’ propensity to hallucinate.

Methods: This study experimentally evaluates two common approaches for enhancing LLM performance—fine-tuning and retrieval-augmented generation (RAG)—applied to both text-book VAT cases and real-world cases from a tax consulting firm. The aim is to identify optimal configurations of LLM-based systems and assess their legal-reasoning capabilities.

Results: Properly configured LLMs can effectively support tax professionals in VAT-related tasks, automating routine work, providing initial analyses, and generating legally grounded justifications for decisions. However, current prototypes are not yet ready for full automation given the sensitivity of the legal domain, and limitations persist in handling implicit client knowledge and context-specific documentation.

Conclusion: LLMs show strong potential to assist tax consultants by reducing workload and supporting VAT decision-making, though future work should focus on integrating structured background information to address remaining gaps in contextual understanding.