SVD Augmented Gradient Optimization

Solution time of nonlinear constrained optimization problem depends on the number of constraints, decision variables and conditioning of decision variables space. While the numbers of constraints and decision variables are external to the optimization procedure itself, one may try to affect the conditioning of the decision variables space within the self contained optimization module. This will directly affect the ratio of convergence of an iterative, gradient based optimization routine. Another opportunity for speedup of the solution process in case of quadratic objective function lies in the chance to eliminate the decision variables least affecting the objective function, and thus decrease the optimization problem size. Elimination of decision variables is based on the singular value decomposition of the objective function. Singular values showing up as a result of such procedure indicate that certain linear combinations of original decision variables do not affect the objective function, and thus may be eliminated from further deliberations. Also if near singular values are encountered as well, even deeper reduction of the optimization problem size is still possible, but at a cost in terms of final solution quality. An idea how to improve the conditioning of decision variables space, and limit the number of decision variables in case of quadratic objective function using singular value decomposition is presented in this paper. Results of computer tests performed during minimization of quadratic objective function and subject to quadratic constraints are enclosed and discussed.