Nonexpansive Matrices with Applications to Solutions of Linear Systems by Fixed Point Iterations
© Teck-Cheong Lim. 2010
Received: 28 August 2009
Accepted: 19 October 2009
Published: 25 October 2009
We characterize (i) matrices which are nonexpansive with respect to some matrix norms, and (ii) matrices whose average iterates approach zero or are bounded. Then we apply these results to iterative solutions of a system of linear equations.
Throughout this paper, will denote the set of real numbers, the set of complex numbers, and the complex vector space of complex matrices. A function is a matrix norm if for all , it satisfies the following five axioms:
This norm on is a matrix norm, called the matrix norm induced by . A matrix norm on is called an induced matrix norm if it is induced by some norm on . If is a matrix norm on , there exists an induced matrix norm on such that for all (cf. [1, page 297]). Indeed one can take to be the matrix norm induced by the norm on defined by
Evidently this means that for the matrix norm induced by , . The following theorem is well known (cf. [1, Sections 5.6.9–5.6.12]).
That (b) follows from (c) is a consequence of the previous remark about an induced matrix norm being less than a matrix norm. Since all norms on are equivalent, the limit in (d) can be relative to any norm on , so that (d) is equivalent to all the entries of converge to zero as , which in turn is equivalent to for all .
proving that is bounded in norm for every . Taking , we see that the set of all columns of is bounded. This proves that is bounded in maximum column sum matrix norm ([1, page 294]), and hence in any norm in . Note that the last part of the proof also follows from the Uniform Boundedness Principle (see, e.g., [2, Corollary , page 66])
as , where is any vector norm of . This contradicts (d). Hence . Now suppose that is an eigenvalue with and the Jordan block corresponding to is not diagonal. Then there exist nonzero vectors such that . Let . Then
Lastly we prove that (e) implies (c). Assume that (e) holds. is similar to its Jordan canonical form whose nonzero off-diagonal entries can be made arbitrarily small by similarity ([1, page 128]). Since the Jordan block for each eigenvalue with modulus 1 is diagonal, we see that there is an invertible matrix such that the -sum of each row of is less than or equal to 1, that is, , where is the maximum row sum matrix norm ([1, page 295]). Define a matrix norm by . Then we have .
Let be an eigenvalue of a matrix . The index of , denoted by index( ) is the smallest value of for which ([1, pages 148 and 131]). Thus condition (e) above can be restated as , and for any eigenvalue of with , .
Now we prove the necessity part of (a). If 1 is an eigenvalue of and is a corresponding eigenvector, then for every and of course fails to converge to 0. If is an eigenvalue of with and is a corresponding eigenvector, then
Suppose that satisfies the conditions in (b) and that is the Jordan canonical form of . Let be an eigenvalue of and let be a column vector of corresponding to . If , then the restriction of to the subspace spanned by is a contraction, and we have . If , and , then by conditions in (b) either , or there exist with such that . In the former case, we have and in the latter case, we see from (16) that is bounded. Finally if then since , we have and hence . In all cases, we proved that is bounded. Since column vectors of form a basis for , the sufficiency part of (b) follows.
Now we prove the necessity part of (b). If has an eigenvalue with and eigenvector , then as shown above as . If has 1 as an eigenvalue and , then there exist nonzero vectors such that and . Then which is unbounded. If is an eigenvalue of with and , then there exist nonzero vectors and such that and . By expanding and using the identity
We now consider applications of preceding theorems to approximation of solution of a linear system , where and a given vector in . Let be a given invertible matrix in . is a solution of if and only if is a fixed point of the mapping defined by
is a contraction if and only if is. In this case, by the well known Contraction Mapping Theorem, given any initial vector , the sequence of iterates converges to the unique solution of . In practice, given , each successive is obtained from by solving the equation
The classical methods of Richardson, Jacobi, and Gauss-Seidel (see, e.g., ) have and respectively, where is the identity matrix, the diagonal matrix containing the diagonal of , and the lower triangular matrix containing the lower triangular portion of . Thus by Theorem 1 we have the following known theorem.
Let be a singular matrix in such that the geometric multiplicity and the algebraic multiplicity of the eigenvalue 0 are equal, that is, . Then there is a unique projection whose range is the range of and whose null space is the null space of , or equivalently, . Moreover, restricted to is an invertible transformation from onto .
Let be a matrix in and a vector in . Let be an invertible matrix in and let . Assume that and that for every eigenvalue of with modulus , that is, is nonexpansive relative to a matrix norm. Starting with an initial vector in define recursively by
Next we consider the case when is not invertible. Since is invertible, we have and . The index of the eigenvalue 0 of is the index of eigenvalue 1 of . Thus by Lemma 5, . For every vector , let and denote the component of in the subspace and , respectively.
Next we consider another kind of iteration in which the nonlinear case was considered in Ishikawa . Note that the type of mappings in this case is slightly weaker than nonexpansivity (see condition (c) in the next lemma).
As in the proof of Theorem 2, (a) and (b) are equivalent. For , denote by . Suppose now that (a) holds. Let be an eigenvalue of . Then is an eigenvalue of . By Theorem 2 for every and hence . If 1 is an eigenvalue of , then it is also an eigenvalue of . By Theorem 2, the index of 1, as an eigenvalue of , is 1. Since obviously and have the same eigenvectors corresponding to the eigenvalue 1, the index of 1, as an eigenvalue of , is also 1. This proves (c).
Now assume (c) holds. Since for , every eigenvalue of , except possibly for 1, has modulus less than 1. Reasoning as above, if 1 is an eigenvalue of , then its index is 1. Therefore by Theorem 2, (a) holds. This completes the proof.
By a well known theorem (see, e.g. ), for every .
Assume now that is not invertible and . Then is in the range of . Since satisfies the condition in Lemma 8, satisfies the condition in Lemma 5. Thus the restriction of on its range is invertible and there exists in such that , or equivalently, . For any vector , we have
If in the previous corollary, , and in part (b), the sequence converges to a solution. This is the Richardson method, see for example, . Even in this case, our method in part (b) may yield a better approximation. For example if
for all . If is diagonally dominant with for every and if or , where is the diagonal matrix containing the diagonal of , and the lower triangular matrix containing the lower triangular entries of , then it is easy to prove that where denotes the maximum row sum matrix norm; see, for example, [1, 3]. The following follows from Theorems 7 and 9.
Let be a diagonally dominant matrix with for all . Let or , where is the diagonal matrix containing the diagonal of , and the lower triangular matrix containing the lower triangular entries of . Let be a vector in . Then:
- Horn RA, Johnson CR: Matrix Analysis. Cambridge University Press, Cambridge, UK; 1985.View ArticleMATHGoogle Scholar
- Dunford N, Schwartz JT: Linear Operators, Part I. Interscience Publishers, New York, NY, USA; 1957.MATHGoogle Scholar
- Kincaid D, Cheney W: Numerical Analysis. Brooks/Cole, Pacific Grove, Calif, USA; 1991.MATHGoogle Scholar
- Ishikawa S: Fixed points and iteration of a nonexpansive mapping in a Banach space. Proceedings of the American Mathematical Society 1976,59(1):65–71. 10.1090/S0002-9939-1976-0412909-XMathSciNetView ArticleMATHGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.