Improving Quadratic Programming Algorithms

Abstract

The main objective of this thesis is to present improvements in quadratic programming algorithms. These improvements speed up the solution of quadratic programming problems, with or without constraints, which are key in various fields, including, but not limited to, economics, engineering, and machine learning.

The main improvements are for solving box-constrained quadratic programming problems. The MPRGP (Modified Proportioning with Reduced Gradient Projections) algorithm is analyzed and, based on this analysis, improved. The analysis reveals that the expansion of the active set through the reduced gradient projections is the most expensive part of the algorithm. Our modification of the expansion step, using the projected conjugate gradient, proves significantly superior to the original algorithm in most cases. The presented fallback steps and criteria can be used to guarantee convergence or even ensure that the convergence rate is at least as good as that of the standard MPRGP algorithm. Another presented modification is to use the Spectral Projected Gradient (SPG) method as the expansion step. This proves to be extremely effective in certain cases, but a little less so in others. Numerical experiments showcasing the effectiveness of the proposed methods, as well as a comparison with the SPG method, are presented on a number of benchmarks.

Another improvement of MPRGP is in preconditioning, which is not straightforward to implement when the problem is constrained. Our improvement is to cheaply approximate the preconditioning in face, which must be recomputed every time the active set changes, with a preconditioner that is set up only once. The numerical experiments show speedups between $5.1$ and $13.4$ compared to the unpreconditioned algorithm. The previous expansion step modification is a key ingredient for an effective preconditioned algorithm. An error analysis comparing the standard and the approximate variant of the preconditioning in face is provided to complement the numerical experiments.

Further improvements include the scalability of FETI (Finite Element Tearing and Interconnecting) domain decomposition methods, which allow us to solve problems with more than one billion unknowns using tens of thousands of computational cores on large supercomputers.

Most of the presented algorithms are implemented in the PERMON library, of which the author of this thesis was the maintainer and the main contributor throughout their doctoral studies. The main aim of PERMON is to provide a scalable framework for the solution of large-scale quadratic problems. Another software contribution was the multilevel deflation preconditioner, PCDEFLATION, in the PETSc library for scientific computing.