Development and Implementation of a Crawler for Collecting and Analyzing Data from Public Databases

Abstract

Open Source Intelligence (OSINT) is a critical methodology for gathering data from publicly accessible sources to improve intelligence analysis and decision-making processes. This thesis explores its conception and application. The main goal of this project is to create a tool specifically for the compilation of information on wind energy resources. The thesis carefully proceeds through each stage of the intelligence cycle, divided into three separate sections. First, it outlines a comprehensive structure for displaying the crawled data. Next, it conducts a comprehensive analysis of the collected data. Last but not least, it provides a methodical way to use the data analysis to support well-informed decision-making. With this thorough investigation, the thesis not only clarifies the nuances of OSINT, but also shows how useful it is in the field of evaluating energy resources.