Fixed point theorems and explicit estimates for convergence rates of continuous time Markov chains

In this paper we give Banach fixed point theorems and explicit estimates on the rates of convergence of the transition function to the stationary distribution for a class of exponential ergodic Markov chains. Our results are different from earlier estimates using coupling theory, and from estimates using stochastically monotone one. Our estimates show a noticeable improvement on existing results if Markov chains contain instantaneous states or nonconservative states. The proof uses existing results of discrete time Markov chains together with h-skeleton. At last, we apply this result, Ray-Knight compactification and Itô excursion theory to two examples: a class of singular Markov chains and Kolmogorov matrix.


Introduction
Throughout this paper, unless otherwise specified, let {X t ; t ∈ [, ∞)} be a time homogeneous, continuous time Markov chain with an honest and standard transition function p ij (t) on a state space E = {, , , . . .}, and its density matrix is Q = (q ij ), q i = -q ii . Let P x and E x denote the probability law and expectation of the Markov chain respectively under the initial condition of X  = x, where x ∈ E. Let X = (Ω, F , F t , X t , θ t , P x ) be the right process associated with p ij (t).
In this paper we consider the Markov chain which is an exponential ergodicity, that means, there is a unique stationary distribution π = (π j ) (j ∈ E), constants R i < ∞ and α >  such that j p ij (t)π i ≤ R i e -αt for all i, j ∈ E. Our goal is to find out the computable bounds of the constants R i and α, especially α.
There has been considerable recent work on the problem of computable bounds for convergence rates of Markov chains. Recently, the authors (see [ However, their methods are not fitted for the general continuous time Markov chains, especially when the symmetric condition, coupling condition or stochastically monotone one is not satisfied. For example, the bounds of Markov chains with instantaneous states such as Kolmogorov matrix, or the regular birth and death process. In this paper, we discuss this problem.
Let i ∈ E and suppose that X  = i, define then there exists R i < ∞ for some (and then for all) i such that j p ij (t)π j ≤ R i e -αt .
In this paper we shall first develop the methods in [] to the continuous time situation, which leads to considerable improvements of convergence rates. And this result shall be in a wider range of application than existing results in [-]. Next we shall give some fundamental lemmas and the proof of the main theorem in this paper. Finally, we shall apply our result and the Itô excursion theorem to compute two examples in Section , which will show the advantages of our result.

Proof of Theorem 1 2.1 Definitions and some fundamental lemmas
Let {Y n } ∞ n= be a time homogeneous Markov chain with one-step transition matrix Π = (Π ij ) on the state space E. Suppose that {Y n } ∞ n= is an aperiodic, irreducible ergodic Markov chain with a transition function Π ij and stationary distribution π j (j ∈ E). Let Π = (Π ij (n)) be an n-step transition matrix and Definition  We say that {Y n } ∞ n= is ρ-geometrically ergodic (for short, geometrically ergodic) if there exits a number ρ with  < ρ <  such that for any n ∈ N and i, j ∈ E, where ρ is called ergodic index.
Lemma  Suppose Π ij and π j are defined as above, m ∈ E is a fixed state, Proof From () together with Theorems . and . in [], we can get the proof of Lemma .
Definition  Given a number h > , the discrete time Markov chain {X nh } ∞ n= having a one-step transition function p ij (h) (and therefore an n-step transition function p ij (nh)) is called the h-skeleton of {X t , t ≥ }.

Lemma  Suppose that p ij (t) is an irreducible and ergodic transition function, m ∈ E is a fixed state; for a constant
Proof It is obvious that m is not an absorbing state, otherwise p ij (t) is reducible. Suppose where k = , , . . . and m is recurrent. Then the stopping times mentioned above are almost surely finite and From the strong Markov property of X, it is easily known that γ τ  , γ τ  , γ τ  , . . . are independent identically distributed exponential random variables with mean q m . So We can easily get If i = m, then we have If i = m and τ  = , then we have If i = m, then we have, for each k ≥ , By () and the equations above we have And by () we have So Lemma  is proved.
Remark  From () we get that when h ↓  and i = m, . .} be at most countable collection of unbounded open subsets of (, ∞). Then, in any nonempty open subinterval I of (, ∞), there exits a number h with the property that for each k, nh ∈ G k for infinitely many integers n.

Proof of Theorem 1
Similarly we can get By () and () we obtain which gives Hence, for any We have for any l > , which is a continuous and unbounded function on (, ∞). Then we know that for i ∈ E and l > , which is a class of nonempty and unbounded open sets on (, ∞). From Proposition , for every G il , there exists  < h < ε such that there are infinitely many nh belonging to G il , where n = , , . . . . If nh ∈ G il , then by () we have By the arbitrariness of l and α, we get From the definition of β i , it is easy to know that for any α > ,

Definition  Let
The constant α * is called the maximal exponentially ergodic constant of a transition function p ij (t).
Remark  If p ij (t) is irreducible, m is a stable state and λ >  which make E m {e λτ + m } < ∞, then we know that p ij (t) is still ergodic and the result in Theorem  remains valid from the proof of Theorem .

Two examples
In this section we compute the maximal exponentially ergodic constants for two types of chains: a kind of singular Markov chain in which all states are not conservative and Kolmogorov matrix in which state  is an instantaneous state.

A kind of singular Markov chain
where  < q  < q  < · · · < q n < · · · < ∞ and inf i q - i =  for i = , , . . . . In this case the transition function with Q-matrix above is not unique (see [, ]), but the honest transition function p ij (t) with Q-matrix is unique and its resolvent is where a k (k ∈ E) are sequences of nonnegative real numbers such that k∈E a k = ∞ and k∈E m∈E a k R min km () = , where R min ij (λ) is the resolvent of the minimal transition function p min ij (t). From [] it is known that this chain is not symmetric, so we cannot discuss its ergodicity with coupling theory. We also cannot adapt existing results to this chain. The following are our main methods and result.

.. Ray-Knight compactification
Theorem  For the Markov chain with Q-matrix (), the Ray-Knight compactification of the state space E is E = E ∪ {∞}, X = (Ω, F , F t , X t , θ t , P x ) is the right processes with the transition function p ij (t).

Then we have
Then by () we have

R E T R A C T E D A R T I C L E
After the Ray-Knight compactification, the Markov chain X = (Ω, F , F t , X t , θ t , P x ) is the right process with the transition function p ij (t).
Remark  This chain also holds the strong Markov property.

.. Excursion leaving from state
be the hitting time of state i. By T  , σ and σ i defined above, we have P i [σ < ∞] =  (i ∈ E).
Consider excursion leaving from state ∞ of X, and let ϕ( And then we have the following result.

Theorem  There exists a continuous additive function L t of X such that
where η ∞ (w) = inf{t|t > , w(t) = ∞}. We write U for Boolean algebra on U, {W t } t≥ for coordinate process, {U t } t> for natural filtration and θ t for shift operator. (U, U) is called the excursion space.
For any t ≥ , define β t = inf{s|L s > t}, {β t } is the right reverse of local time L t . Let D p (ω) = {t|β t-< β t }. We have known that between excursion leaving from state ∞ of X and D p (ω) is one-to-one correspondence (see [, ]).

Theorem  For any t
Remark  This means that the characteristic measureP(·) has the same distribution as k∈E a k P k {·}.
Remark  () The state space of the right process X is E ∪ {∞}, where ∞ is the branching point; The local time L t of X on S ∞ is continuous; () The excursion measureP(·) is σ -finite and satisfieŝ

.. Maximal exponentially ergodic constant
The following theorem gives the stationary distribution.

Theorem  The transition function p ij (t) defined above is ergodic, and its stationary distribution is
Then we complete the proof.
In the following we discuss the conditions of exponential ergodicity and convergence rate of exponential ergodicity.

Theorem  If
then for some (and then for all) i ∈ E, E i [e λσ  ] < ∞, p ij (t) is exponentially ergodic. Moreover, for this λ, if Proof () For any i ∈ E (i = ), In the following we compute E ∞ [e λσ  ]. Consider the coordinate process W (s) on the excursion space (U, U ). For any i ∈ E, define Obviously C  , C  ∈ U and C  ∪ C  = U. Let τ = inf{t|β t > σ  } (t > ) and where A denotes the cardinal of A, then we have which gives that τ is exponential random variable with mean a  . And hence we have From the computation of the Poisson point process, we know that Hence , Lemma ., we know that p ij (t) is exponentially ergodic.
() If i = , then τ +  = σ  . If λ satisfies (), then we have for any i ∈ E from Theorems  and . If m =  in Theorem , by the method of () above, we have We complete the proof of this theorem.

Kolmogorov matrix
This following example contains an instantaneous state. Suppose that q  , q  , . . . are sequences of positive real numbers and consider Q-matrix as follows: where ∞ i= q i - < ∞. This matrix is called the Kolmogorov matrix. There are infinitely many dishonest processes with this Q-matrix. The authors (see [, , ]) have shown that the process with the following resolvents is the only honest one.

R E T R
and where λ >  and the state space is E = {, , , . . .}. Obviously, the transition function p ij (t) which corresponds with the resolvents above is the only honest one. Though this chain is weakly symmetric, its convergence rate is still unknown because of its instantaneous state.

.. Ray-Knight compactification
Theorem  For the Markov chain with Q-matrix above, the Ray-Knight compactification of the state space E is still E, X = (Ω, F , F t , X t , θ t , P x ) is the right process with the transition function p ij (t).
By using the methods in [], we show that E = E. In the Ray-Knight topology, instantaneous state  is the limit point of sequences {, , . . .}. So we know that the Markov chain X = (Ω, F , F t , X t , θ t , P x ) is the right process with the transition function p ij (t) (see [, ]).

Remark  This chain holds the strong Markov property.
.. Excursion leaving from state  For each i ∈ E, the definition of T  , σ , σ i as above. Then obviously for each i ∈ E, we have P i [σ < ∞] = , which means that instantaneous state  is a recurrent state of X.
Consider excursion leaving from state  of X, and let for all x ∈ E; it is easily verified that ϕ(·) is a -excessive function of X. And then we have the following result.
Theorem  There exists a continuous additive function L t of X such that where η  (w) = inf{t|t > , w(t) = }. We write U for Boolean algebra on U, {W t } t≥ for coordinate process, {U t } t> for natural filtration and θ t for shift operator. (U, U) is called the excursion space. For any t ≥ , let β t = inf{s|L s > t}, {β t } be the right reverse of local time L t . Let D p (ω) = {t|β t-< β t }. We have known that between excursion leaving from state  of X and D p (ω) is a one-to-one correspondence.
then {Y t ; t ∈ D p } is the Poisson point process on the excursion space (U, U ), and the characteristic measureP(·) satisfieŝ P {W t  = i  , . . . , W t n = i n } = k∈E p min ki  (t  ) · · · p min i n- i n (t nt n- ).
Remark  Theorem  means that the characteristic measureP(·) has the same distribution as k∈E P k {·}.
Remark  () The state space of the right process X is E, where  is the branching point; () The local time L t of X on S  is continuous; () The excursion measureP(·) is σ -finite and satisfieŝ

.. Maximal exponentially ergodic constant
The following theorem will give the stationary distribution.
Theorem  The transition function p ij (t) defined above is ergodic, and we know that its stationary distribution is Thus the proof is completed.
In the following we discuss the conditions of exponential ergodicity and the convergence rates of exponential ergodicity.

Theorem  If
then for some (and then for all) i ∈ E, E i [e λσ  ] < ∞, so p ij (t) is exponentially ergodic. Moreover, for this λ, if then there exists R i < ∞ for any i ∈ E such that j∈E p ij (t)π j ≤ R i e -αt .
Proof () For any i ∈ E (i ≥ ), we know Then we will compute E  {e λσ  }. Consider the coordinate process W (s) on the excursion space (U, U ). For any i ∈ E, define η i = inf s|W (s) = i . Let τ = inf{t|β t > σ  } (t > ) and we have which gives τ is exponential random variable with mean  and So Therefore, if () is satisfied, then we get that E  [e λσ  ] < ∞ and E i {e λσ  } < ∞. From Lemma . in [], p., we know that p ij (t) is exponentially ergodic.
() If i ≥ , then we have τ +  = σ  . If λ satisfies (), then we have E i e λτ +  = E i e λσ  < ∞ for any i ∈ E from Theorem .

A R T I C L E
Let m =  in Theorem , by the method of () above, we have So the result is proved from Theorem .
Remark  According to the results above, the maximal exponentially ergodic constant of this example satisfies .