Skip to main content
  • Research Article
  • Open access
  • Published:

On Properties of Solutions for Two Functional Equations Arising in Dynamic Programming

Abstract

We introduce and study two new functional equations, which contain a lot of known functional equations as special cases, arising in dynamic programming of multistage decision processes. By applying a new fixed point theorem, we obtain the existence, uniqueness, iterative approximation, and error estimate of solutions for these functional equations. Under certain conditions, we also study properties of solutions for one of the functional equations. The results presented in this paper extend, improve, and unify the results according to Bellman, Bellman and Roosta, Bhakta and Choudhury, Bhakta and Mitra, Liu, Liu and Ume, and others. Two examples are given to demonstrate the advantage of our results over existing results in the literature.

1. Introduction and Preliminaries

The existence, uniqueness, and successive approximations of solutions for the following functional equations arising in dynamic programming:

(1.1)

were first introduced and discussed by Bellman [1, 2]. Afterwards, further analyses on the properties of solutions for the functional equations (1.1) and (1.2) and others have been studied by several authors in [3–7] and [8–11] by using various fixed point theorems and monotone iterative technique, where (1.2) are as follows:

(1.2)

The aim of this paper is to investigate properties of solutions for the following more general functional equations arising in dynamic programming of multistage decision processes:

(1.3)
(1.4)

where and are real Banach spaces, is the state space, is the decision space, opt denotes the sup or inf, and stand for the state and decision vectors, respectively, , represent the transformations of the processes, and denotes the optimal return function with initial state . The rest of the paper is organized as follows. In Section 2, we state the definitions, notions, and a lemma and establish a new fixed point theorem, which will be used in the rest of the paper. The main results are presented in Section 3. By applying the new fixed point theorem, we establish the existence, uniqueness, iterative approximation, and error estimate of solutions for the functional equation (1.3) and (1.4). Under certain conditions, we also study other properties of solutions for the functional equations (1.4). The results present in this paper extend, improve, and unify the corresponding results according to Bellman [1], Bellman and Roosta [5], Bhakta and Choudhury [6], Bhakta and Mitra [7], Liu [8], Liu and Ume [11], and others. Two examples are given to demonstrate the advantage of our results over existing results in the literature.

Throughout this paper, we assume that , , and . For any , denotes the largest integer not exceeding . Define

(1.5)

2. A Fixed Point Theorem

Let be a countable family of pseudometrics on a nonvoid set such that for any two different points , for some . For any , let

(2.1)

then is a metric on . A sequence in is said to converge to a point if as for any and to be a Cauchy sequence if as for any .

Theorem 2.1.

Let be a complete metric space, and let be defined by (2.1). If satisfies the following inequality:

(2.2)

where is some element in , then

(i) has a unique fixed point and for any ,

(ii)if, in addition, , then

(2.3)

Proof.

Given and , define for each . In view of (2.2), we know that

(2.4)

Since , by (2.4) we easily conclude that is nonincreasing. It follows that has a limit . We claim that . Otherwise, . On account of (2.4) and , we deduce that

(2.5)

which is impossible. That is, . We now show that is a Cauchy sequence. Suppose that is not a Cauchy sequence, then there exist , , and two sequences of positive integers and with and

(2.6)

which yields that

(2.7)

As in (2.7), we derive that . Note that (2.2) and (2.7) mean that

(2.8)

for any . Letting in (2.8), we see that

(2.9)

This is a contradiction. By completeness of , there exists a point , such that . Using (2.1), (2.2), and , we obtain that for each

(2.10)

which yields that

(2.11)

that is, is a fixed point of . If has a fixed point different from , then there exists such that . By (2.2), we have

(2.12)

which is a contradiction. Consequently, is a unique fixed point of .

Suppose that . By (2.2), we get that for any , , and

(2.13)

This completes the proof.

Remark 2.2.

Theorem 2.1 extends Theorem 2.1 of Bhakta and Choudhury [6] and Theorem 1 of Boyd and Wong [12].

Lemma 2.3 (see [11]).

Let , ,, and be in , then

(2.14)

3. Properties of Solutions

In this section, we assume that and are real Banach spaces, is the state space, and is the decision space. Define

(3.1)

For any positive integer and , let

(3.2)

where and , then is a countable family of pseudometrics on . It is clear that is a complete metric space.

Theorem 3.1.

Let and be mappings, and let be in , such that

(C1)for any and ,

(3.3)

(C2)for any and , there exists satisfying

(3.4)

then the functional equation (1.3) possesses a unique solution , and converges to for each , where is defined by

(3.5)

In addition, if is in , then

(3.6)

Proof.

It follows from (C2) and (3.4) that maps into itself. Given , , , and , suppose that , then there exist such that

(3.7)

In view of (3.3), (3.5), and (3.7), we deduce that

(3.8)

which implies that

(3.9)

Similarly, we can show that (3.9) holds for . As in (3.9), we get that

(3.10)

Notice that the functional equation (1.3) possesses a unique solution if and only if the mapping has a unique fixed point . Thus, Theorem 3.1 follows from Theorem 2.1. This completes the proof.

Remark 3.2.

The conditions of Theorem 3.1 are weaker than the conditions of Theorem 3.1 of Bhakta and Choudhury [6].

Theorem 3.3.

Let and be mappings for . Assume that the following conditions are satisfied:

(C3)for each , there exists such that

(3.11)

(C4), for all ,

(C5)there exists a constant such that

(3.12)

then the functional equation (1.4) possesses a unique solution , and converges to for each , where is defined by

(3.13)

Moreover,

(3.14)

Proof.

Set

(3.15)
(3.16)

It follows from (C3)–(C5) and (3.15) that

(3.17)

for any and . Consequently, is a self mapping on . By Lemma 2.3, (C4), and (C5), we obtain that for any and ,

(3.18)

where for . Thus, Theorem 3.3 follows from Theorem 3.1. This completes the proof.

Remark 3.4.

Theorem 2 of Bellman [1, page 121], the result of Bellman and Roosta [5, page 545], Theorem 3.3 of Bhakta and Choudhury [6], and Theorems  3.3 and  3.4 of Liu [8] are special cases of Theorem 3.3. The example below shows that Theorem 3.3 extends properly the results in [1, 5, 6, 8].

Example 3.5.

Let and . Put , , and for any . It follows from Theorem 3.3 that the functional equation

(3.19)

possesses a unique solution . However, the results in [1, 5, 6, 8] are not applicable.

Theorem 3.6.

Let and be mappings for , and, be in satisfying

(C6), for all ,

(C7), for all ,

(C8),

then the functional equation (1.4) possesses a solution that satisfies the following conditions:

(C9)the sequence defined by

(3.20)

converges to ,

(C10) for any and ,

(C11) is unique with respect to condition (C10).

Proof.

Let and be defined by (3.15) and (3.16), respectively. We now claim that

(3.21)

If not, then there exists some such that . On account of , we know that for any ,

(3.22)

whence

(3.23)

which is a contradiction since .

Next, we assert that the mapping is nonexpansive on . Let and . It is easy to see that

(3.24)

by (C7) and (3.21). Consequently, there exists a constant satisfying

(3.25)

In view of (C6), (3.16), and (3.25), we derive that for any ,

(3.26)

which yields that maps into itself. Given , , , and , suppose that , then there exist such that

(3.27)

Using (C6)–(C8), (3.15) and (3.27), and Lemma 2.3, we deduce that

(3.28)

which means that

(3.29)

Similarly, we can conclude that the above inequality holds for . Letting , we get that

(3.30)

which implies that

(3.31)

That is, is nonexpansive.

We show that for each ,

(3.32)

In terms of (C6) and (C9), we obtain that

(3.33)

which means that (3.32) holds for . Suppose that (3.32) holds for some . It follows from (C6)–(C8) and (3.25) that

(3.34)

Therefore, (3.32) holds for any .

Next, we prove that is a Cauchy sequence in . Given , , , , and , suppose that . We select that with

(3.35)

According to (C6)–(C8) and (3.35), we have

(3.36)

for some and . In a similar way, we can conclude that (3.36) holds for . Proceeding in this way, we select and for such that

(3.37)

In terms of (C7), (3.21), (3.32), (3.36), and (3.37), we know that

(3.38)

which implies that

(3.39)

Letting in the above inequality, we have

(3.40)

which means that is a Cauchy sequence in because for each . Let . By the nonexpansivity of , we get that

(3.41)

which implies that . That is, is a solution of the functional equation (1.4).

Now, we show that (C10) holds. Given , , , and , for , set . It is easy to verify that there exists a positive integer satisfying

(3.42)

Notice that

(3.43)

for any . Consequently, we infer immediately that, for ,

(3.44)

which yields that .

At last, we show that (C11) holds. Suppose that the functional equation (1.4) possesses another solution , which satisfies (C10). Given and , suppose that , then there are satisfying

(3.45)

Whence there exists and such that

(3.46)

by (C8). Proceeding in this way, we select and for satisfying

(3.47)

It follows that

(3.48)

which yields that

(3.49)

by letting . Similarly, (3.49) also holds for . As , we know that . This completes the proof.

Remark 3.7.

Theorem 3.6 generalizes Theorem 1 of Bellman [1, page 119], Theorem 3.5 of Bhakta and Choudhury [6], Theorem 2.4 of Bhakta and Mitra [7], Theorem 3.5 of Liu [8] and Theorem 3.1 of Liu and Ume [11]. The following example reveals that Theorem 3.6 is indeed a generalization of the results in [1, 6–8, 11].

Example 3.8.

Let ,. Define by

(3.50)

It is easy to verify that the following functional equation:

(3.51)

satisfies conditions (C6)–(C8). Consequently, Theorem 3.6 ensures that it has a solution that satisfies conditions (C9)–(C11). However, Theorem 1 of Bellman [1, page 119], Theorem 3.5 of Bhakta and Choudhury [6], Theorem 2.4 of Bhakta and Mitra [7], Theorem 3.5 of Liu [8], and Theorem 3.1 of Liu and Ume [11] are not applicable.

References

  1. Bellman R: Dynamic Programming. Princeton University Press, Princeton, NJ, USA; 1957:xxv+342.

    MATH  Google Scholar 

  2. Bellman R: Methods of Nonlinear Analysis. Volume 2. Academic Press, New York, NY, USA; 1973:xvii+261.

    Google Scholar 

  3. Belbas SA: Dynamic programming and maximum principle for discrete Goursat systems. Journal of Mathematical Analysis and Applications 1991,161(1):57–77. 10.1016/0022-247X(91)90362-4

    Article  MathSciNet  MATH  Google Scholar 

  4. Bellman R, Lee ES: Functional equations in dynamic programming. Aequationes Mathematicae 1978,17(1):1–18. 10.1007/BF01818535

    Article  MathSciNet  MATH  Google Scholar 

  5. Bellman R, Roosta M: A technique for the reduction of dimensionality in dynamic programming. Journal of Mathematical Analysis and Applications 1982,88(2):543–546. 10.1016/0022-247X(82)90212-8

    Article  MathSciNet  MATH  Google Scholar 

  6. Bhakta PC, Choudhury SR: Some existence theorems for functional equations arising in dynamic programming. II. Journal of Mathematical Analysis and Applications 1988,131(1):217–231. 10.1016/0022-247X(88)90201-6

    Article  MathSciNet  MATH  Google Scholar 

  7. Bhakta PC, Mitra S: Some existence theorems for functional equations arising in dynamic programming. Journal of Mathematical Analysis and Applications 1984,98(2):348–362. 10.1016/0022-247X(84)90254-3

    Article  MathSciNet  MATH  Google Scholar 

  8. Liu Z: Existence theorems of solutions for certain classes of functional equations arising in dynamic programming. Journal of Mathematical Analysis and Applications 2001,262(2):529–553. 10.1006/jmaa.2001.7551

    Article  MathSciNet  MATH  Google Scholar 

  9. Liu Z: Coincidence theorems for expansion mappings with applications to the solutions of functional equations arising in dynamic programming. Acta Scientiarum Mathematicarum 1999,65(1–2):359–369.

    MathSciNet  MATH  Google Scholar 

  10. Liu Z: Compatible mappings and fixed points. Acta Scientiarum Mathematicarum 1999,65(1–2):371–383.

    MathSciNet  MATH  Google Scholar 

  11. Liu Z, Ume JS: On properties of solutions for a class of functional equations arising in dynamic programming. Journal of Optimization Theory and Applications 2003,117(3):533–551. 10.1023/A:1023945621360

    Article  MathSciNet  MATH  Google Scholar 

  12. Boyd DW, Wong JSW: On nonlinear contractions. Proceedings of the American Mathematical Society 1969, 20: 458–464. 10.1090/S0002-9939-1969-0239559-9

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgment

This research is financially supported by Changwon National University in 2009-2010.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to JeongSheok Ume.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Liu, Z., Ume, J. & Kang, S. On Properties of Solutions for Two Functional Equations Arising in Dynamic Programming. Fixed Point Theory Appl 2010, 905858 (2010). https://doi.org/10.1155/2010/905858

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/2010/905858

Keywords