Steepest descent and conjugate gradient methods with variable preconditioning

Ilya Lashuk

Email: ilashuk@math.cudenver.edu
Postal address: Department of Mathematical Sciences, University of Colorado at Denver and Health Sciences Center, P.O. Box 173364, Campus Box 170, Denver, CO 80217-3364

We show that the conjugate gradient method with variable preconditioning in certain situations cannot give any improvement, compared to the steepest descent method for solving a linear system with a real symmetric positive definite (SPD) matrix of coefficients. We assume that the preconditioner is SPD on each step, and that the condition number of the preconditioned system matrix is bounded from above by a constant independent of the step number. Our proof is geometric and is based on the simple fact that a nonzero vector multiplied by all SPD matrices with a condition number bounded by a constant generates a pointed circular cone. We discuss the peculiarities of the complex case and prove that our main results also hold for complex Hermitian positive definite matrices. Numerical experiments support our theoretical findings.