Normalizace SQL příkazů pro zpracování knihovnou DataFusion

Abstract

The quality of SQL queries directly affects the performance and reliability of database applications. Tools for static analysis of SQL queries allow you to detect errors and performance anti-patterns automatically, without running the query against a real database. However, in an enterprise environment, each database system has its own variant of the SQL language, which deviates from the standard and from other systems. The DBShepherd project, developed at the Department of Informatics at VŠB-TUO, performs such analysis using the Apache DataFusion library. In order to process queries from different systems, dialect-specific syntax must be normalized into a form that DataFusion accepts before processing. The aim of this thesis is to extend the DBShepherd project to support three cloud data warehouses: Snowflake, Databricks, and Google BigQuery. The extension includes the implementation of components for reading metadata, normalizing queries, and registering dialect-specific functions. The thesis also includes a test collection of more than 100 SQL queries for each of the three systems.