- Research Article
- Open Access

# Solvability and Algorithms for Functional Equations Originating from Dynamic Programming

- Guojing Jiang
^{1}, - ShinMin Kang
^{2}and - YoungChel Kwun
^{3}Email author

**2011**:701519

https://doi.org/10.1155/2011/701519

© Guojing Jiang et al. 2011

**Received:**5 January 2011**Accepted:**11 February 2011**Published:**10 March 2011

## Abstract

The main purpose of this paper is to study the functional equation arising in dynamic programming of multistage decision processes , . A few iterative algorithms for solving the functional equation are suggested. The existence, uniqueness and iterative approximations of solutions for the functional equation are discussed in the Banach spaces and and the complete metric space , respectively. The properties of solutions, nonnegative solutions, and nonpositive solutions and the convergence of iterative algorithms for the functional equation and other functional equations, which are special cases of the above functional equation, are investigated in the complete metric space , respectively. Eight nontrivial examples which dwell upon the importance of the results in this paper are also given.

## Keywords

- Banach Space
- Unique Solution
- Functional Equation
- Dynamic Programming
- Iterative Algorithm

## 1. Introduction

in .

where denotes or , and stand for the state and decision vectors, respectively, , and represent the transformations of the processes, and represents the optimal return function with initial .

This paper is divided into four sections. In Section 2, we recall a few basic concepts and notations, establish several lemmas that will be needed further on, and suggest ten iterative algorithms for solving the functional equations (1.3)–(1.9). In Section 3, by applying the Banach fixed-point theorem, we establish the existence, uniqueness, and iterative approximations of solutions for the functional equation (1.3) in the Banach spaces and , respectively. By means of two iterative algorithms defined in Section 2, we obtain the existence, uniqueness, and iterative approximations of solutions for the functional equation (1.3) in the complete metric space . Under certain conditions, we investigate the behaviors of solutions, nonpositive solutions, and nonnegative solutions and the convergence of iterative algorithms for the functional equations (1.3)–(1.7), respectively, in . In Section 4, we construct eight nontrivial examples to explain our results, which extend and improve substantially the results due to Bellman [3], Bhakta and Choudhury [7], Liu [12], Liu and Kang [14, 15], Liu and Ume [17], Liu et al. [18], and others.

## 2. Lemmas and Algorithms

Clearly and are Banach spaces with norm .

where
and
. It is easy to see that
is a countable family of pseudometrics on
. A sequence
in
is said to be *converge* to a point
if, for any
as
and to be a *Cauchy sequence* if, for any
,
as
. It is clear that
is a complete metric space.

Lemma 2.1.

Let . Then

(a) ,

(b) for , ,

(c) for ,

(d) for ,

(e) .

Proof.

Hence (e) holds for any . This completes the proof.

Lemma 2.2.

Let and . Then

(a) ,

(b) .

Proof.

That is, (a) is true for . Therefore (a) holds for any . Similarly we can prove (b). This completes the proof.

Lemma 2.3.

Proof.

This completes the proof.

Proof.

This completes the proof.

Algorithm 1.

Algorithm 2.

For any , compute by (2.17) and (2.18).

Algorithm 3.

For any , compute by (2.17) and (2.18).

Algorithm 4.

Algorithm 5.

Algorithm 6.

Algorithm 7.

Algorithm 8.

Algorithm 9.

Algorithm 10.

## 3. The Properties of Solutions and Convergence of Algorithms

Now we prove the existence, uniqueness, and iterative approximations of solutions for the functional equation (1.3) in and , respectively, by using the Banach fixed-point theorem.

Theorem 3.1.

Let be compact. Let and satisfy the following conditions:

(C1) is bounded in ;

(C2) for some constant ;

uniformly for , respectively.

Proof.

Thus (3.10), (3.11), and (2.17) ensure that the mapping and Algorithm 1 are well defined.

and the sequence converges to by (2.18). This completes the proof.

Dropping the compactness of and (C3) in Theorem 3.1, we conclude immediately the following result.

Theorem 3.2.

Let and satisfy conditions (C1) and (C2). Then the functional equation (1.3) possesses a unique solution and the sequence generated by Algorithm 2 converges to and satisfies (3.2).

Next we prove the existence, uniqueness, and iterative approximations of solution for the functional equation (1.3) in .

Theorem 3.3.

Let and satisfy condition (C2) and the following two conditions:

(C4) is bounded on for each ;

(C5) for all .

Proof.

which means that is a self-mapping in . Consequently, Algorithms 3 and 4 are well defined.

Similarly we conclude that (3.24) holds for . As in (3.24), we get that (3.22) holds.

which gives that as . This completes the proof.

Next we investigate the behaviors of solutions for the functional equations (1.3)–(1.5) and discuss the convergence of Algorithms 4–6 in , respectively.

Theorem 3.4.

Let , and satisfy the following conditions:

(C6) for all ;

(C7) for all ;

(C8) .

Then the functional equation (1.3) possesses a solution satisfying conditions (C9)–(C12) below:

(C9)the sequence generated by Algorithm 4 converges to , where with for all ;

(C10) for all ;

(C11) for any , and , , for all ;

(C12) is unique relative to condition (C11).

Proof.

That is, the mapping is nonexpansive.

That is, (3.37) is true for . Hence (3.37) holds for each .

which yields that . That is, the functional equation (1.3) possesses a solution .

that is, (C10) holds.

which means that .

Since is arbitrary, we conclude immediately that . This completes the proof.

Theorem 3.5.

Let , and satisfy conditions (C6)–(C8). Then the functional equation (1.4) possesses a solution satisfying conditions (C10)–(C12) and the following two conditions:

(C13) the sequence generated by Algorithm 5 converges to , where with for all ;

then is nonnegative.

Proof.

In terms of (C8), (C11), and (3.55), we see that as . Letting in (3.59), we get that . Since is arbitrary, we infer immediately that . This completes the proof.

Theorem 3.6.

Let , and satisfy conditions (C6), (C7), and the following condition:

(C15) , and are nonnegative and .

Then the functional equation (1.6) possesses a solution satisfying for any , where the sequence is generated by Algorithm 7 with , and for all .

Proof.

and hence (3.60) holds for . That is, (3.60) holds for any .

It follows from (3.71) and (3.74) that is a solution of the functional equation (1.6). This completes the proof.

Following similar arguments as in the proof of Theorems 3.5 and 3.6, we have the following results.

Theorem 3.7.

Let , and satisfy conditions (C6)–(C8). Then the functional equation (1.5) possesses a solution satisfying conditions (C10)–(C12) and the two following conditions:

(C16)the sequence generated by Algorithm 6 converges to , where with for all ;

then is nonpositive.

Theorem 3.8.

Let , and satisfy conditions (C6), (C7), and (C15). Then the functional equation (1.7) possesses a solution satisfying for any , where the sequence is generated by Algorithm 8 with , and for all .

## 4. Applications

In this section we use these results in Section 3 to establish the existence of solutions, nonnegative solutions, and nonpositive solutions and iterative approximations for several functional equations, respectively.

Example 4.1.

possesses a unique solution and the sequence generated by Algorithm 1 converges to and satisfies (3.2).

Example 4.2.

possesses a unique solution and the sequence generated by Algorithm 2 converges to and satisfies (3.2).

Remark 4.3.

If , for all , then Theorem 3.3 reduces to a result which generalizes the result in [3, page 149] and Theorem 3.4 in [7]. The following example demonstrates that Theorem 3.3 generalizes properly the corresponding results in [3, 7].

Example 4.4.

has a unique solution in . However, the results in [3, page 149] and Theorem 3.4 in [7] are valid for the functional equation (4.3).

- (1)
If , for all , then Theorems 3.4, 3.5, and 3.7 reduce to three results which generalize and unify the result in [3, page 149], Theorem 3.5 in [7], Theorem 3.5 in [12], Corollaries 2.2 and 2.3 in [14], Corollaries 3.3 and 3.4 in [17], and Theorems 2.3 and 2.4 in [18], respectively.

- (2)
The results in [3, page 149], Theorem 3.5 in [7], Theorem 3.5 in [12], and Theorem 3.4 in [15] are special cases of Theorem 3.5 with , for all .

The examples below show that Theorems 3.4, 3.5, and 3.7 are indeed generalizations of the corresponding results in [3, 7, 12, 14, 15, 17, 18].

Example 4.6.

possesses a solution that satisfies (C9)–(C12). However, the corresponding results in [3, 7, 12, 14, 17, 18] are not applicable for the functional equation (4.4).

Example 4.7.

has a solution satisfying (C10)–(C14). But the corresponding results in [3, 7, 12, 14, 15, 17, 18] are not valid for the functional equation (4.5).

Example 4.8.

Example 4.9.

has a solution satisfying (C10)–(C12), (C16), and (C17). But the corresponding results in [3, 7, 12, 14, 17, 18] are not valid for the functional equation (4.8).

Example 4.10.

## Declarations

### Acknowledgments

The authors wish to thank the referees for pointing out some printing errors. This study was supported by research funds from Dong-A University.

## Authors’ Affiliations

## References

- Belbas SA:
**Dynamic programming and maximum principle for discrete Goursat systems.***Journal of Mathematical Analysis and Applications*1991,**161**(1):57–77. 10.1016/0022-247X(91)90362-4MathSciNetView ArticleMATHGoogle Scholar - Bellman R:
**Some functional equations in the theory of dynamic programming. I. Functions of points and point transformations.***Transactions of the American Mathematical Society*1955,**80:**51–71. 10.1090/S0002-9947-1955-0074692-0MathSciNetView ArticleMATHGoogle Scholar - Bellman R:
*Dynamic Programming*. Princeton University Press, Princeton, NJ, USA; 1957:xxv+342.MATHGoogle Scholar - Bellman R:
*Methods of Nonlinear Analysis, Vol. II*. Academic Press, New York, NY, USA; 1973:xvii+261.Google Scholar - Bellman R, Lee ES:
**Functional equations in dynamic programming.***Aequationes Mathematicae*1978,**17**(1):1–18. 10.1007/BF01818535MathSciNetView ArticleMATHGoogle Scholar - Bellman R, Roosta M:
**A technique for the reduction of dimensionality in dynamic programming.***Journal of Mathematical Analysis and Applications*1982,**88**(2):543–546. 10.1016/0022-247X(82)90212-8MathSciNetView ArticleMATHGoogle Scholar - Bhakta PC, Choudhury SR:
**Some existence theorems for functional equations arising in dynamic programming. II.***Journal of Mathematical Analysis and Applications*1988,**131**(1):217–231. 10.1016/0022-247X(88)90201-6MathSciNetView ArticleMATHGoogle Scholar - Bhakta PC, Mitra S:
**Some existence theorems for functional equations arising in dynamic programming.***Journal of Mathematical Analysis and Applications*1984,**98**(2):348–346. 10.1016/0022-247X(84)90254-3MathSciNetView ArticleMATHGoogle Scholar - Chang S, Ma YH:
**Coupled fixed points for mixed monotone condensing operators and an existence theorem of the solutions for a class of functional equations arising in dynamic programming.***Journal of Mathematical Analysis and Applications*1991,**160**(2):468–479. 10.1016/0022-247X(91)90319-UMathSciNetView ArticleMATHGoogle Scholar - Liu Z:
**Coincidence theorems for expansion mappings with applications to the solutions of functional equations arising in dynamic programming.***Acta Scientiarum Mathematicarum*1999,**65**(1–2):359–369.MathSciNetMATHGoogle Scholar - Liu Z:
**Compatible mappings and fixed points.***Acta Scientiarum Mathematicarum*1999,**65**(1–2):371–383.MathSciNetMATHGoogle Scholar - Liu Z:
**Existence theorems of solutions for certain classes of functional equations arising in dynamic programming.***Journal of Mathematical Analysis and Applications*2001,**262**(2):529–553. 10.1006/jmaa.2001.7551MathSciNetView ArticleMATHGoogle Scholar - Liu Z, Agarwal RP, Kang SM:
**On solvability of functional equations and system of functional equations arising in dynamic programming.***Journal of Mathematical Analysis and Applications*2004,**297**(1):111–130. 10.1016/j.jmaa.2004.04.049MathSciNetView ArticleMATHGoogle Scholar - Liu Z, Kang SM:
**Properties of solutions for certain functional equations arising in dynamic programming.***Journal of Global Optimization*2006,**34**(2):273–292. 10.1007/s10898-005-2605-6MathSciNetView ArticleMATHGoogle Scholar - Liu Z, Kang SM:
**Existence and uniqueness of solutions for two classes of functional equations arising in dynamic programming.***Acta Mathematicae Applicatae Sinica. English Series*2007,**23**(2):195–208. 10.1007/s10255-007-0363-6MathSciNetView ArticleMATHGoogle Scholar - Liu Z, Kim JK:
**A common fixed point theorem with applications in dynamic programming.***Nonlinear Functional Analysis and Applications*2006,**11**(1):11–19.MathSciNetMATHGoogle Scholar - Liu Z, Ume JS:
**On properties of solutions for a class of functional equations arising in dynamic programming.***Journal of Optimization Theory and Applications*2003,**117**(3):533–551. 10.1023/A:1023945621360MathSciNetView ArticleMATHGoogle Scholar - Liu Z, Ume JS, Kang SM:
**Some existence theorems for functional equations arising in dynamic programming.***Journal of the Korean Mathematical Society*2006,**43**(1):11–28.MathSciNetView ArticleMATHGoogle Scholar - Liu Z, Xu Y, Ume JS, Kang SM:
**Solutions to two functional equations arising in dynamic programming.***Journal of Computational and Applied Mathematics*2006,**192**(2):251–269. 10.1016/j.cam.2005.04.033MathSciNetView ArticleMATHGoogle Scholar - Zhang SS:
**Some existence theorems of common and coincidence solutions for a class of systems of functional equations arising in dynamic programming.***Applied Mathematics and Mechanics*1991,**12**(1):31–37.MathSciNetGoogle Scholar - Wang C-L:
**The principle and models of dynamic programming. II.***Journal of Mathematical Analysis and Applications*1988,**135**(1):268–283. 10.1016/0022-247X(88)90153-9MathSciNetView ArticleMATHGoogle Scholar - Wang C-L:
**The principle and models of dynamic programming. III.***Journal of Mathematical Analysis and Applications*1988,**135**(1):284–296. 10.1016/0022-247X(88)90154-0View ArticleMathSciNetMATHGoogle Scholar - Wang C-L:
**The principle and models of dynamic programming. V.***Journal of Mathematical Analysis and Applications*1989,**137**(1):161–167. 10.1016/0022-247X(89)90279-5View ArticleMathSciNetMATHGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.