Neural Networks in High Performance Computing Environment

Abstract

With increasing opportunities for analysing large data sources, we have noticed a lack of effective processing in data mining tasks working with large sparse datasets of high dimensions. This work focuses on this issue and on effective clustering using models of artificial intelligence.

We propose an effective clustering algorithm to exploit the features of neural networks, especially Self Organizing Maps (SOM) and Growing Neural Gas (GNG), for the reduction of data dimensionality. The issue of computational complexity is resolved by using a parallelization of the standard SOM and GNG algorithm. We have focused on the acceleration of the presented algorithm using a version suitable for data collections with a certain level of sparsity. Effective acceleration is achieved by improving the winning neuron finding phase and the weight actualization phase. The output presented here demonstrates sufficient acceleration of the standard SOM and GNG algorithm while preserving the appropriate accuracy.

Next, we again proposed a parallelization of the SOM and GNG algorithms, but now with out restrictions to identical results as in the case of the standard algorithms. In the SOM algorithm there is a combination of Euclidean distance and cosine similarity in the winning neuron finding phase and the weight actualization phase. Another option is a combination of standard SOM and batch SOM in the weight actualization phase. In the GNG algorithm there is a pre-processing by Self Organizing Maps with Spanning tree algorithm.

Finally, we focused on the possibility of migration of parallel applications written in C# language between Windows HPC and Linux HPC, including a study of possible time losses.