- Research Article
- Open access
- Published:
On Properties of Solutions for Two Functional Equations Arising in Dynamic Programming
Fixed Point Theory and Applications volume 2010, Article number: 905858 (2010)
Abstract
We introduce and study two new functional equations, which contain a lot of known functional equations as special cases, arising in dynamic programming of multistage decision processes. By applying a new fixed point theorem, we obtain the existence, uniqueness, iterative approximation, and error estimate of solutions for these functional equations. Under certain conditions, we also study properties of solutions for one of the functional equations. The results presented in this paper extend, improve, and unify the results according to Bellman, Bellman and Roosta, Bhakta and Choudhury, Bhakta and Mitra, Liu, Liu and Ume, and others. Two examples are given to demonstrate the advantage of our results over existing results in the literature.
1. Introduction and Preliminaries
The existence, uniqueness, and successive approximations of solutions for the following functional equations arising in dynamic programming:
were first introduced and discussed by Bellman [1, 2]. Afterwards, further analyses on the properties of solutions for the functional equations (1.1) and (1.2) and others have been studied by several authors in [3–7] and [8–11] by using various fixed point theorems and monotone iterative technique, where (1.2) are as follows:
The aim of this paper is to investigate properties of solutions for the following more general functional equations arising in dynamic programming of multistage decision processes:
where and are real Banach spaces, is the state space, is the decision space, opt denotes the sup or inf, and stand for the state and decision vectors, respectively, , represent the transformations of the processes, and denotes the optimal return function with initial state . The rest of the paper is organized as follows. In Section 2, we state the definitions, notions, and a lemma and establish a new fixed point theorem, which will be used in the rest of the paper. The main results are presented in Section 3. By applying the new fixed point theorem, we establish the existence, uniqueness, iterative approximation, and error estimate of solutions for the functional equation (1.3) and (1.4). Under certain conditions, we also study other properties of solutions for the functional equations (1.4). The results present in this paper extend, improve, and unify the corresponding results according to Bellman [1], Bellman and Roosta [5], Bhakta and Choudhury [6], Bhakta and Mitra [7], Liu [8], Liu and Ume [11], and others. Two examples are given to demonstrate the advantage of our results over existing results in the literature.
Throughout this paper, we assume that , , and . For any , denotes the largest integer not exceeding . Define
2. A Fixed Point Theorem
Let be a countable family of pseudometrics on a nonvoid set such that for any two different points , for some . For any , let
then is a metric on . A sequence in is said to converge to a point if as for any and to be a Cauchy sequence if as for any .
Theorem 2.1.
Let be a complete metric space, and let be defined by (2.1). If satisfies the following inequality:
where is some element in , then
(i) has a unique fixed point and for any ,
(ii)if, in addition, , then
Proof.
Given and , define for each . In view of (2.2), we know that
Since , by (2.4) we easily conclude that is nonincreasing. It follows that has a limit . We claim that . Otherwise, . On account of (2.4) and , we deduce that
which is impossible. That is, . We now show that is a Cauchy sequence. Suppose that is not a Cauchy sequence, then there exist , , and two sequences of positive integers and with and
which yields that
As in (2.7), we derive that . Note that (2.2) and (2.7) mean that
for any . Letting in (2.8), we see that
This is a contradiction. By completeness of , there exists a point , such that . Using (2.1), (2.2), and , we obtain that for each
which yields that
that is, is a fixed point of . If has a fixed point different from , then there exists such that . By (2.2), we have
which is a contradiction. Consequently, is a unique fixed point of .
Suppose that . By (2.2), we get that for any , , and
This completes the proof.
Remark 2.2.
Theorem 2.1 extends Theorem 2.1 of Bhakta and Choudhury [6] and Theorem 1 of Boyd and Wong [12].
Lemma 2.3 (see [11]).
Let , ,, and be in , then
3. Properties of Solutions
In this section, we assume that and are real Banach spaces, is the state space, and is the decision space. Define
For any positive integer and , let
where and , then is a countable family of pseudometrics on . It is clear that is a complete metric space.
Theorem 3.1.
Let and be mappings, and let be in , such that
(C1)for any and ,
(C2)for any and , there exists satisfying
then the functional equation (1.3) possesses a unique solution , and converges to for each , where is defined by
In addition, if is in , then
Proof.
It follows from (C2) and (3.4) that maps into itself. Given , , , and , suppose that , then there exist such that
In view of (3.3), (3.5), and (3.7), we deduce that
which implies that
Similarly, we can show that (3.9) holds for . As in (3.9), we get that
Notice that the functional equation (1.3) possesses a unique solution if and only if the mapping has a unique fixed point . Thus, Theorem 3.1 follows from Theorem 2.1. This completes the proof.
Remark 3.2.
The conditions of Theorem 3.1 are weaker than the conditions of Theorem 3.1 of Bhakta and Choudhury [6].
Theorem 3.3.
Let and be mappings for . Assume that the following conditions are satisfied:
(C3)for each , there exists such that
(C4), for all ,
(C5)there exists a constant such that
then the functional equation (1.4) possesses a unique solution , and converges to for each , where is defined by
Moreover,
Proof.
Set
It follows from (C3)–(C5) and (3.15) that
for any and . Consequently, is a self mapping on . By Lemma 2.3, (C4), and (C5), we obtain that for any and ,
where for . Thus, Theorem 3.3 follows from Theorem 3.1. This completes the proof.
Remark 3.4.
Theorem 2 of Bellman [1, page 121], the result of Bellman and Roosta [5, page 545], Theorem 3.3 of Bhakta and Choudhury [6], and Theorems  3.3 and  3.4 of Liu [8] are special cases of Theorem 3.3. The example below shows that Theorem 3.3 extends properly the results in [1, 5, 6, 8].
Example 3.5.
Let and . Put , , and for any . It follows from Theorem 3.3 that the functional equation
possesses a unique solution . However, the results in [1, 5, 6, 8] are not applicable.
Theorem 3.6.
Let and be mappings for , and, be in satisfying
(C6), for all ,
(C7), for all ,
(C8),
then the functional equation (1.4) possesses a solution that satisfies the following conditions:
(C9)the sequence defined by
converges to ,
(C10) for any and ,
(C11) is unique with respect to condition (C10).
Proof.
Let and be defined by (3.15) and (3.16), respectively. We now claim that
If not, then there exists some such that . On account of , we know that for any ,
whence
which is a contradiction since .
Next, we assert that the mapping is nonexpansive on . Let and . It is easy to see that
by (C7) and (3.21). Consequently, there exists a constant satisfying
In view of (C6), (3.16), and (3.25), we derive that for any ,
which yields that maps into itself. Given , , , and , suppose that , then there exist such that
Using (C6)–(C8), (3.15) and (3.27), and Lemma 2.3, we deduce that
which means that
Similarly, we can conclude that the above inequality holds for . Letting , we get that
which implies that
That is, is nonexpansive.
We show that for each ,
In terms of (C6) and (C9), we obtain that
which means that (3.32) holds for . Suppose that (3.32) holds for some . It follows from (C6)–(C8) and (3.25) that
Therefore, (3.32) holds for any .
Next, we prove that is a Cauchy sequence in . Given , , , , and , suppose that . We select that with
According to (C6)–(C8) and (3.35), we have
for some and . In a similar way, we can conclude that (3.36) holds for . Proceeding in this way, we select and for such that
In terms of (C7), (3.21), (3.32), (3.36), and (3.37), we know that
which implies that
Letting in the above inequality, we have
which means that is a Cauchy sequence in because for each . Let . By the nonexpansivity of , we get that
which implies that . That is, is a solution of the functional equation (1.4).
Now, we show that (C10) holds. Given , , , and , for , set . It is easy to verify that there exists a positive integer satisfying
Notice that
for any . Consequently, we infer immediately that, for ,
which yields that .
At last, we show that (C11) holds. Suppose that the functional equation (1.4) possesses another solution , which satisfies (C10). Given and , suppose that , then there are satisfying
Whence there exists and such that
by (C8). Proceeding in this way, we select and for satisfying
It follows that
which yields that
by letting . Similarly, (3.49) also holds for . As , we know that . This completes the proof.
Remark 3.7.
Theorem 3.6 generalizes Theorem 1 of Bellman [1, page 119], Theorem 3.5 of Bhakta and Choudhury [6], Theorem 2.4 of Bhakta and Mitra [7], Theorem 3.5 of Liu [8] and Theorem 3.1 of Liu and Ume [11]. The following example reveals that Theorem 3.6 is indeed a generalization of the results in [1, 6–8, 11].
Example 3.8.
Let ,. Define by
It is easy to verify that the following functional equation:
satisfies conditions (C6)–(C8). Consequently, Theorem 3.6 ensures that it has a solution that satisfies conditions (C9)–(C11). However, Theorem 1 of Bellman [1, page 119], Theorem 3.5 of Bhakta and Choudhury [6], Theorem 2.4 of Bhakta and Mitra [7], Theorem 3.5 of Liu [8], and Theorem 3.1 of Liu and Ume [11] are not applicable.
References
Bellman R: Dynamic Programming. Princeton University Press, Princeton, NJ, USA; 1957:xxv+342.
Bellman R: Methods of Nonlinear Analysis. Volume 2. Academic Press, New York, NY, USA; 1973:xvii+261.
Belbas SA: Dynamic programming and maximum principle for discrete Goursat systems. Journal of Mathematical Analysis and Applications 1991,161(1):57–77. 10.1016/0022-247X(91)90362-4
Bellman R, Lee ES: Functional equations in dynamic programming. Aequationes Mathematicae 1978,17(1):1–18. 10.1007/BF01818535
Bellman R, Roosta M: A technique for the reduction of dimensionality in dynamic programming. Journal of Mathematical Analysis and Applications 1982,88(2):543–546. 10.1016/0022-247X(82)90212-8
Bhakta PC, Choudhury SR: Some existence theorems for functional equations arising in dynamic programming. II. Journal of Mathematical Analysis and Applications 1988,131(1):217–231. 10.1016/0022-247X(88)90201-6
Bhakta PC, Mitra S: Some existence theorems for functional equations arising in dynamic programming. Journal of Mathematical Analysis and Applications 1984,98(2):348–362. 10.1016/0022-247X(84)90254-3
Liu Z: Existence theorems of solutions for certain classes of functional equations arising in dynamic programming. Journal of Mathematical Analysis and Applications 2001,262(2):529–553. 10.1006/jmaa.2001.7551
Liu Z: Coincidence theorems for expansion mappings with applications to the solutions of functional equations arising in dynamic programming. Acta Scientiarum Mathematicarum 1999,65(1–2):359–369.
Liu Z: Compatible mappings and fixed points. Acta Scientiarum Mathematicarum 1999,65(1–2):371–383.
Liu Z, Ume JS: On properties of solutions for a class of functional equations arising in dynamic programming. Journal of Optimization Theory and Applications 2003,117(3):533–551. 10.1023/A:1023945621360
Boyd DW, Wong JSW: On nonlinear contractions. Proceedings of the American Mathematical Society 1969, 20: 458–464. 10.1090/S0002-9939-1969-0239559-9
Acknowledgment
This research is financially supported by Changwon National University in 2009-2010.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Liu, Z., Ume, J. & Kang, S. On Properties of Solutions for Two Functional Equations Arising in Dynamic Programming. Fixed Point Theory Appl 2010, 905858 (2010). https://doi.org/10.1155/2010/905858
Received:
Accepted:
Published:
DOI: https://doi.org/10.1155/2010/905858