Classificação de consultas fiscais na SEFAZ-PB: uma análise comparativa entre método estatístico e modelos baseados em transformers

Abstract

Automatic classification of tax queries submitted to public digital systems, such as SefazVirtual from the State Department of Finance of Para´ıba (SEFAZ-PB), involves natural language messages sent by taxpayers through institutional service channels. These queries express doubts, questions, or requests related to taxes, legislation, and administrative procedures. The main challenge lies in the thematic overlap between categories, linguistic ambiguity, and the scarcity of annotated data for supervised training. Therefore, this research investigates and compares different modeling paradigms to mitigate these difficulties, including a classical statistical baseline based on TF-IDF combined with Logistic Regression, supervised fine-tuning of BERT (Bidirectional Encoder Representations from Transformers), and in-context learning strategies using the LLaMA 3.3 70B model. The dataset, composed of messages distributed across five categories related to the Tax on the Circulation of Goods and Services (ICMS) and the Motor Vehicle Property Tax (IPVA), underwent preprocessing steps including anonymization in compliance with the Brazilian General Data Protection Law (LGPD), text normalization, tokenization, and stopword removal. Two experimental training configurations were conducted for BERT (base and large), each evaluated using two versions of the dataset — the original version and an enriched version obtained through controlled rewriting of the queries with the support of ChatGPT and the expansion of label definitions. The TF-IDF model combined with Logistic Regression was used as a baseline to contextualize the performance gains of neural architectures. The evaluation, based on accuracy, precision, recall, and F1-score, showed that data augmentation increased the accuracy of BERT-large from 0.62 to 0.77 and BERT-base from 0.62 to 0.66, while the TF-IDF baseline combined with Logistic Regression achieved an accuracy of 0.80 with enriched data, and LLaMA 70B reached up to 0.87 accuracy in a few-shot scenario with 20 examples. These results confirm the viability of both statistical approaches and transformer-based models adapted to the tax domain as effective alternatives for optimizing the routing of queries and reducing delays in public service responses.