Solvability and Algorithms for Functional Equations Originating from Dynamic Programming

Abstract

The main purpose of this paper is to study the functional equation arising in dynamic programming of multistage decision processes ,. A few iterative algorithms for solving the functional equation are suggested. The existence, uniqueness and iterative approximations of solutions for the functional equation are discussed in the Banach spaces and and the complete metric space , respectively. The properties of solutions, nonnegative solutions, and nonpositive solutions and the convergence of iterative algorithms for the functional equation and other functional equations, which are special cases of the above functional equation, are investigated in the complete metric space , respectively. Eight nontrivial examples which dwell upon the importance of the results in this paper are also given.

1. Introduction

The existence, uniqueness, and iterative approximations of solutions for several classes of functional equations arising in dynamic programming were studied by a lot of researchers; see [1–23] and the references therein. Bellman [3], Bhakta and Choudhury [7], Liu [12], Liu and Kang [15], and Liu et al. [18] investigated the following functional equations:

(11)

and gave some existence and uniqueness results and iterative approximations of solutions for the functional equations in . Liu and Kang [14] and Liu and Ume [17] generalized the results in [3, 7, 12, 15, 18] and studied the properties of solutions, nonpositive solutions and nonnegative solutions and the convergence of iterative approximations for the following functional equations, respectively

(12)

in .

The purpose of this paper is to introduce and study the following functional equations arising in dynamic programming of multistage decision processes:

(13)

(14)

(15)

(16)

(17)

(18)

(19)

where denotes or , and stand for the state and decision vectors, respectively, , and represent the transformations of the processes, and represents the optimal return function with initial .

This paper is divided into four sections. In Section 2, we recall a few basic concepts and notations, establish several lemmas that will be needed further on, and suggest ten iterative algorithms for solving the functional equations (1.3)–(1.9). In Section 3, by applying the Banach fixed-point theorem, we establish the existence, uniqueness, and iterative approximations of solutions for the functional equation (1.3) in the Banach spaces and , respectively. By means of two iterative algorithms defined in Section 2, we obtain the existence, uniqueness, and iterative approximations of solutions for the functional equation (1.3) in the complete metric space . Under certain conditions, we investigate the behaviors of solutions, nonpositive solutions, and nonnegative solutions and the convergence of iterative algorithms for the functional equations (1.3)–(1.7), respectively, in . In Section 4, we construct eight nontrivial examples to explain our results, which extend and improve substantially the results due to Bellman [3], Bhakta and Choudhury [7], Liu [12], Liu and Kang [14, 15], Liu and Ume [17], Liu et al. [18], and others.

2. Lemmas and Algorithms

Throughout this paper, we assume that , , , denotes the set of positive integers, and, for each , denotes the largest integer not exceeding . Let and be real Banach spaces, the state space, and the decision space. Define

(21)

Clearly and are Banach spaces with norm .

For any and , let

(22)

where and . It is easy to see that is a countable family of pseudometrics on . A sequence in is said to be converge to a point if, for any as and to be a Cauchy sequence if, for any , as . It is clear that is a complete metric space.

Lemma 2.1.

Let . Then

(a),

(b) for , ,

(c) for ,

(d) for ,

(e).

Proof.

Clearly (a)–(d) are true. Now we show (e). Note that (e) holds for . Suppose that (e) is true for some . It follows from (a) and Lemma  2.1 in [17] that

(23)

Hence (e) holds for any . This completes the proof.

Lemma 2.2.

Let and . Then

(a),

(b).

Proof.

It is clear that (a) is true for . Suppose that (a) is also true for some . Using Lemma  2.3 in [19] and Lemma 2.1, we infer that

(24)

That is, (a) is true for . Therefore (a) holds for any . Similarly we can prove (b). This completes the proof.

Lemma 2.3.

Let , be convergent sequences in . Then

(25)

Proof.

Put for . In view of Lemma 2.1 we deduce that

(26)

which yields that

(27)

This completes the proof.

Lemma 2.4.

• (a) Assume that is a mapping such that is bounded for some . Then

  

  (28)

• (b) Assume that are mappings such that and are bounded for some . Then

  

  (29)

Proof.

Now we show (a). If , (a) holds clearly. Suppose that . Note that

(210)

It follows that

(211)

which implies that

(212)

Next we show (b). If , (b) is true. Suppose that . Note that

(213)

which yields that

(214)

It follows that

(215)

which gives that

(216)

This completes the proof.

Algorithm 1.

For any , compute by

(217)

where

(218)

Algorithm 2.

For any , compute by (2.17) and (2.18).

Algorithm 3.

For any , compute by (2.17) and (2.18).

Algorithm 4.

For any , compute by

(219)

Algorithm 5.

For any , compute by

(220)

Algorithm 6.

For any , compute by

(221)

Algorithm 7.

For any , compute by

(222)

Algorithm 8.

For any , compute by

(223)

Algorithm 9.

For any , compute by

(224)

Algorithm 10.

For any , compute by

(225)

3. The Properties of Solutions and Convergence of Algorithms

Now we prove the existence, uniqueness, and iterative approximations of solutions for the functional equation (1.3) in and , respectively, by using the Banach fixed-point theorem.

Theorem 3.1.

Let be compact. Let and satisfy the following conditions:

(C1) is bounded in ;

(C2) for some constant ;

(C3)for each fixed ,

(31)

uniformly for , respectively.

Then the functional equation (1.3) possesses a unique solution , and the sequence generated by Algorithm 1 converges to and has the error estimate

(32)

Proof.

Define a mapping by

(33)

Let and and . It follows from (C1), (C3), and the compactness of that there exist constants , , and satisfying

(34)

(35)

(36)

(37)

(38)

(39)

Using (3.3)–(3.5), (C2), and Lemmas 2.1 and 2.4, we get that

(310)

In light of (C2), (3.3), (3.5)–(3.9), and Lemmas 2.1 and 2.4, we deduce that for all with

(311)

Thus (3.10), (3.11), and (2.17) ensure that the mapping and Algorithm 1 are well defined.

Next we assert that the mapping is a contraction. Let , , and . Suppose that . Choose such that

(312)

Lemma 2.1 and (3.12) lead to

(313)

which implies that

(314)

Letting in the above inequality, we know that

(315)

Similarly we conclude that (3.15) holds for . The Banach fixed-point theorem yields that the contraction mapping has a unique fixed point . It is easy to verify that is also a unique solution of the functional equation (1.3) in . By means of (2.17), (2.18), (3.15), and

(316)

we infer that

(317)

which yields that

(318)

and the sequence converges to by (2.18). This completes the proof.

Dropping the compactness of and (C3) in Theorem 3.1, we conclude immediately the following result.

Theorem 3.2.

Let and satisfy conditions (C1) and (C2). Then the functional equation (1.3) possesses a unique solution and the sequence generated by Algorithm 2 converges to and satisfies (3.2).

Next we prove the existence, uniqueness, and iterative approximations of solution for the functional equation (1.3) in .

Theorem 3.3.

Let and satisfy condition (C2) and the following two conditions:

(C4) is bounded on for each ;

(C5) for all .

Then the functional equation (1.3) possesses a unique solution , and the sequences and generated by Algorithms 3 and 4, respectively, converge to and have the error estimates

(319)

Proof.

Define a mapping by (3.3). Let and . Thus (C4) and (C5) guarantee that there exist and satisfying

(320)

Using (3.3), (3.20), (C2), (C5), and Lemmas 2.1 and 2.4, we infer that

(321)

which means that is a self-mapping in . Consequently, Algorithms 3 and 4 are well defined.

Now we claim that

(322)

Let , , , and . Suppose that . Select such that (3.12) holds. Thus (3.3), (3.12), (C2), (C5), and Lemma 2.1 ensure that

(323)

which implies that

(324)

Similarly we conclude that (3.24) holds for . As in (3.24), we get that (3.22) holds.

Let . It follows from Algorithm 4 that

(325)

and (3.22) leads to

(326)

which yields that is a Cauchy sequence in the complete metric space , and hence converges to some . In light of (3.22) and (C2), we know that

(327)

That is, the mapping is nonexpansive. It follows from (3.27) and Algorithm 4 that

(328)

that is, . Suppose that there exists with . Consequently there exists some satisfying . It follows from (3.22) and (C2) that

(329)

which is a contradiction. Hence the mapping has a unique fixed point , which is a unique solution of the functional equation (1.3) in . Letting in (3.26), we infer that

(330)

It follows from Algorithm 3, (2.18), and (3.22) that

(331)

which gives that as . This completes the proof.

Next we investigate the behaviors of solutions for the functional equations (1.3)–(1.5) and discuss the convergence of Algorithms 4–6 in , respectively.

Theorem 3.4.

Let , and satisfy the following conditions:

(C6) for all ;

(C7) for all ;

(C8).

Then the functional equation (1.3) possesses a solution satisfying conditions (C9)–(C12) below:

(C9)the sequence generated by Algorithm 4 converges to , where with for all ;

(C10) for all ;

(C11) for any , and , , for all ;

(C12) is unique relative to condition (C11).

Proof.

First of all we assert that

(332)

Suppose that there exists some with . It follows from that

(333)

That is,

(334)

which is impossible. That is, (3.32) holds. Let the mapping be defined by (3.3) in . Note that (C6) and (C7) imply (C4) and (C5) by (3.32) and , respectively. As in the proof of Theorem 3.3, by (C8) we conclude that the mapping maps into and satisfies

(335)

(336)

That is, the mapping is nonexpansive.

Let the sequence be generated by Algorithm 4 and with for all . We now claim that for each

(337)

Clearly (3.37) holds for . Assume that (3.37) is true for some . It follows from (C6)–(C8), (3.32), Algorithm 4, and Lemmas 2.1 and 2.4 that

(338)

That is, (3.37) is true for . Hence (3.37) holds for each .

Next we claim that is a Cauchy sequence in . Let , , and . Suppose that . Choose with

(339)

It follows from (3.39), (C8), and Lemmas 2.2 and 2.3 that

(340)

Therefore there exist and satisfying

(341)

In a similar method, we can derive that (3.41) holds also for . Proceeding in this way, we choose and for such that

(342)

On account of , (C7), (3.37), (3.41), and (3.42), we gain that

(343)

which yields that

(344)

Letting in the above inequality, we infer that

(345)

It follows from and (3.45) that is a Cauchy sequence in and it converges to some . Algorithm 4 and (3.36) lead to

(346)

which yields that . That is, the functional equation (1.3) possesses a solution .

Now we show (C10). Let . Put . It follows from (3.37), (C7), and that

(347)

that is, (C10) holds.

Next we prove (C11). Given , , and , , for . Put . Note that (C7) implies that

(348)

In view of (3.32), (3.37), (3.48), and , we know that

(349)

which means that .

Finally we prove (C12). Assume that the functional equation (1.3) has another solution that satisfies (C11). Let and . Suppose that . Select with

(350)

On account of (3.50), (C8), and Lemma 2.1, we conclude that there exist and , satisfying

(351)

that is,

(352)

Similarly we can prove that (3.52) holds for . Proceeding in this way, we select and for and such that

(353)

It follows from (3.52) and (3.53) that

(354)

Since is arbitrary, we conclude immediately that . This completes the proof.

Theorem 3.5.

Let , and satisfy conditions (C6)–(C8). Then the functional equation (1.4) possesses a solution satisfying conditions (C10)–(C12) and the following two conditions:

(C13) the sequence generated by Algorithm 5 converges to , where with for all ;

(C14) if , and are nonnegative and there exists a constant such that

(355)

then is nonnegative.

Proof.

It follows from Theorem 3.4 that the functional equation (1.4) has a solution that satisfies (C10)–(C13). Now we show (C14). Given , and . It follows from Lemma 2.2, (3.55), and (1.4) that there exist and such that

(356)

That is,

(357)

Proceeding in this way, we choose and for and such that

(358)

It follows from (3.57) and (3.58) that

(359)

In terms of (C8), (C11), and (3.55), we see that as . Letting in (3.59), we get that . Since is arbitrary, we infer immediately that . This completes the proof.

Theorem 3.6.

Let , and satisfy conditions (C6), (C7), and the following condition:

(C15), and are nonnegative and .

Then the functional equation (1.6) possesses a solution satisfying for any , where the sequence is generated by Algorithm 7 with , and   for all .

Proof.

We are going to prove that, for any ,

(360)

Using and Algorithm 7, we gain that

(361)

that is, (3.60) holds for . Assume that (3.60) holds for some . Lemma 2.1 and (C15) lead to

(362)

which implies that

(363)

and hence (3.60) holds for . That is, (3.60) holds for any .

Now we claim that, for any ,

(364)

In fact, (C6) ensures that

(365)

that is, (3.64) is true for . Assume that (3.64) is true for some . In view of Lemmas 2.1 and 2.4, Algorithm 7, (C6), (C7), and C(15), we gain that

(366)

which yields that (3.64) is true for . Therefore (3.64) holds for each . Given , note that exists. It follows that there exist constants and satisfying for any . Thus (3.64) leads to

(367)

On account of (3.60), (3.67), and Algorithm 7, we deduce that is convergent for each and . Put

(368)

Obviously (3.67) ensures that . Notice that

(369)

Letting in the above inequality, by Lemmas 2.1 and 2.3 and the convergence of we infer that

(370)

which yields that

(371)

It follows from (3.60), (C15), and Lemma 2.1 that

(372)

which implies that

(373)

Letting , we gain that

(374)

It follows from (3.71) and (3.74) that is a solution of the functional equation (1.6). This completes the proof.

Following similar arguments as in the proof of Theorems 3.5 and 3.6, we have the following results.

Theorem 3.7.

Let , and satisfy conditions (C6)–(C8). Then the functional equation (1.5) possesses a solution satisfying conditions (C10)–(C12) and the two following conditions:

(C16)the sequence generated by Algorithm 6 converges to , where with for all ;

(C17)if , and are nonnegative and there exists a constant such that

(375)

then is nonpositive.

Theorem 3.8.

Let , and satisfy conditions (C6), (C7), and (C15). Then the functional equation (1.7) possesses a solution satisfying for any , where the sequence is generated by Algorithm 8 with , and   for all .

4. Applications

In this section we use these results in Section 3 to establish the existence of solutions, nonnegative solutions, and nonpositive solutions and iterative approximations for several functional equations, respectively.

Example 4.1.

Let , , , and . It follows from Theorem 3.1 that the functional equation

(41)

possesses a unique solution and the sequence generated by Algorithm 1 converges to and satisfies (3.2).

Example 4.2.

Let , , , and . It is clear that Theorem 3.2 ensures that the functional equation

(42)

possesses a unique solution and the sequence generated by Algorithm 2 converges to and satisfies (3.2).

Remark 4.3.

If , for all , then Theorem 3.3 reduces to a result which generalizes the result in [3, page 149] and Theorem  3.4 in [7]. The following example demonstrates that Theorem 3.3 generalizes properly the corresponding results in [3, 7].

Example 4.4.

Let , , and . It is easy to verify that Theorem 3.3 guarantees that the functional equation

(43)

has a unique solution in . However, the results in [3, page 149] and Theorem  3.4 in [7] are valid for the functional equation (4.3).

Remark 4.5.

• (1) If , for all , then Theorems 3.4, 3.5, and 3.7 reduce to three results which generalize and unify the result in [3, page 149], Theorem  3.5 in [7], Theorem  3.5 in [12], Corollaries  2.2 and  2.3 in [14], Corollaries 3.3  and  3.4 in [17], and Theorems  2.3 and  2.4 in [18], respectively.

• (2) The results in [3, page 149], Theorem  3.5 in [7], Theorem  3.5 in [12], and Theorem  3.4 in [15] are special cases of Theorem 3.5 with , for all .

The examples below show that Theorems 3.4, 3.5, and 3.7 are indeed generalizations of the corresponding results in [3, 7, 12, 14, 15, 17, 18].

Example 4.6.

Let , . Define two functions by , for all . It is easy to see that Theorem 3.4 guarantees that the functional equation

(44)

possesses a solution that satisfies (C9)–(C12). However, the corresponding results in [3, 7, 12, 14, 17, 18] are not applicable for the functional equation (4.4).

Example 4.7.

Let . Put , , and for all . It is easy to verify that Theorem 3.5 guarantees that the functional equation

(45)

has a solution satisfying (C10)–(C14). But the corresponding results in [3, 7, 12, 14, 15, 17, 18] are not valid for the functional equation (4.5).

Example 4.8.

Let , . Put and for all . It is easy to verify that Theorem 3.6 guarantees that the functional equation

(46)

has a solution and the sequence generated by Algorithm 7 satisfies that for each , where with

(47)

Example 4.9.

Let . Put , and for all . It is easy to verify that Theorem 3.7 guarantees that the functional equation

(48)

has a solution satisfying (C10)–(C12), (C16), and (C17). But the corresponding results in [3, 7, 12, 14, 17, 18] are not valid for the functional equation (4.8).

Example 4.10.

Let , . Put and for all . It is easy to verify that Theorem 3.8 guarantees that the functional equation

(49)

possesses a solution and the sequence generated by Algorithm 8 satisfies that for each , where with

(410)