Analýza SQL příkazů v dbt projektu

Abstract

This master thesis deals with the analysis of SQL commands within data projects built on the dbt tool. The main goal of the thesis is the integration of the dbt ecosystem into an already existing analytical project and its use for the automated detection of structural flaws in the design of data models. In the practical part, a command-line tool was first designed and implemented to ensure the extraction of schemas and dependencies directly from the metadata of a dbt project and their storage in a state database. The tool includes an algorithm for analyzing data flows and identifying redundant logic in table joins. The output of this analysis, which is a recommendation to extract these duplicates into separate shared models, is automatically written into the project's text documentation and the aforementioned state database. The implemented solution thus actively helps to reduce code duplication, simplifies the maintenance of large-scale database systems, and contributes to adherence to best practices in data engineering.