Um estudo sobre ferramentas de código aberto para carga de dados de documentos fiscais eletrônicos

Abstract

This work raised the state of the art in the area of data extraction, loading and transformation and analysis of two open-source code alternatives: LoaderNFE and Airbyte. At first, it composts various tools and currently performs the ETL process of the SEFAZ-PB Data Integration Platform project in a non-automated way. Secondly, it proposes an alternative based on ELT processes and has a Web interface for the operation and automation of some tasks. Additionally, the two tools are endorsed in the environment of SEFAZ-PB for extraction, transformation and loading of 33 batches of consumer electronic tax notes, each with 125,000 documents. The objective was to assess the main differences in configuration and operation between the approaches and carry out a comparative study of performance to verify the feasibility of using Airbyte to perform daily synchronization between the operational base of SEFAZ-PB and the Data Integration Platform. The results obtained indicate that Airbyte can be used in scenarios that require an automatic and more frequent synchronization between data sources and destinations, even being slower than LoaderNFE.