Analýza metod senzorové fúze LiDARu a kamerového obrazu pro interpretaci scén

Abstract

This bachelor’s thesis examines the analysis and experimental comparison of LiDAR and camera image sensor fusion methods for scene interpretation in autonomous driving applications. The theoretical section describes the physical principles of both sensors—the distance measurement mechanism using Time-of-Flight and continuous phase modulation methods in LiDAR, the pinhole camera model, and intrinsic and extrinsic calibration parameters—and systematically classifies fusionstrategies according to the level of abstraction of the processed data into early fusion, feature-level fusion, and decision-level fusion, including an analysis of their advantages, limitations, and typical examples in the literature. The practical section describes the design and implementation of a modular system in Python, working with 1500 images from the KITTI dataset training set. LiDAR data processing involves road segmentation via height thresholding, adaptive voxelization, and clustering using the HDBSCAN algorithm with parameters adapted to the average distance of points from the sensor, supplemented by geometric filtering and the NMS algorithm. Camera detection is performed using the YOLOv8n model pre-trained on the COCO dataset with output classes mapped to KITTI categories. The system implements two fusion methods: Feature-level Fusion uses LiDAR to define the region of interest (ROI) in the image, with object classification exclusively within this area performed by the YOLOv8n model with a confidence threshold. Decision-level Fusion processes data from both sensors independently and pairs the results using a Greedy algorithm based on a matching score combining the IoU of the bounding boxes and the Euclidean distance between their centers in image space, with only detections confirmed by both sensors simultaneously entering the final evaluation. The accuracy of both methods was evaluated using the Precision, Recall, and F1-score metrics.