Toward an optimal and structured feature subset selection for multi-target regression using genetic algorithm

Abstract

Multi Target Regression (MTR) is a machine learning method that simultaneously predicts multiple real-valued outputs using a set of input variables. A lot of emerging applications that can be mapped to this class of problem. In MTR method one of the critical aspect is to handle structural information like instance and target correlation. MTR algorithms attempt to exploit these interdependences when building a model. This results in increased model complexities, which in turn, reduce the interpretability of the model through manual analysis of the result. However, data driven real-world applications often require models that can be used to analyze and improve real-world workflows. Leveraging dimensionality reduction techniques can reduce model complexity while retaining the performance and boost interpretability. This research proposes multiple feature subset alternatives for MTR using genetic algorithm, and provides a comparison of the different feature subset selection alternatives in conjunction with MTR algorithms. We proposed a genetic algorithm based feature subset selection with all targets and with individual target keeping the structural information intact in the selection process. Experiments are performed on real world benchmarked MTR data sets and the results indicate that a significant improvement in performance can be obtained with comparatively simple MTR models by utilizing optimal and structured feature selection.