Přenos velkého množství dat v prostředí HPC

Abstract

This master thesis deals with the transfer of large amounts of data in an HPC environment, specifically in the HPC DataStore application. The aim was to investigate the possibilities of increasing the data throughput and to test the implemented solutions. The proposed solutions were switching from HTTP/1.1 to HTTP/2, implementing server-side caching and switching from classic blocked access to REST API to reactive access. Comparisons were also made with other technologies, namely SCP and SFTP. After implementing the proposed modifications, application testing was performed for all the proposed solutions and also for the final solution combining all the proposals. The evaluation was done by testing the transfer times and data throughput both downloading and uploading files to the HPC DataStore. The results of the testing confirmed that the modifications made led to an improvement in the application throughput.