D5.2 : New AI Models to Reduce Manual Annotation

Abstract

This deliverable presents two new AI models developed in Task 5.3 to reduce the amount of required manual annotations in the context of annotating large-scale multi-modal data in autonomous driving in WP5. The developed models build on recent breakthrough advancements in (multi-modal) self-supervised machine learning. We address two key perception tasks for the automatic annotation of data from autonomous driving. We start with models for object instance detection and segmentation. Then we proceed to models for semantic segmentation but address the semantic segmentation problem in a novel 2D-3D setting. For both problems, we address the open vocabulary setting, which enables reasoning about any object class that can be specified in natural language. This is an important set-up for autonomous driving as it enables describing also unusual situations and corner cases. The developed models have been deployed on use-case-specific data provided by VALEO and are ready to be integrated into the tool for testing and validation of advanced driver assistance systems developed in Task 5.2.