K -Correspondences, USCOs, and fixed point problems arising in discounted stochastic games

We establish a fixed point theorem for the composition of nonconvex, measurable selection valued correspondences with Banach space valued selections. We show that if the underlying probability space of states is nonatomic and if the selection correspondences in the composition are K-correspondences (meaning correspondences having graphs that contain their Komlos limits), then the induced measurable selection valued composition correspondence takes contractible values and therefore has fixed points. As an application we use our fixed point result to show that all nonatomic uncountable-compact discounted stochastic games have stationary Markov perfect equilibria – thus resolving a long-standing open question in game theory.


Introduction
We make two contributions. First, we establish a fixed point theorem for the composition of measurable selection valued correspondences with Banach space valued selections. a We show that if the underlying probability space of states is nonatomic and if the selection correspondences in the composition are K -correspondences (meaning correspondences having graphs that contain their Komlos limits), then the induced nonconvex, measurable selection valued composition correspondence is approximable and therefore has fixed points. b Second, we apply our fixed point result to show that all nonatomic, uncountablecompact discounted stochastic games (DSGs) satisfying the assumptions of the Nowak-Raghavan DSG model have stationary Markov perfect equilibria (SMPE) -thus, resolving a long standing open question in game theory (see Nowak and Raghavan [1]). c Regarding our first contribution, the key step allowing us to establish our fixed point result is to show that if the probability measure on the state space is nonatomic and if the selection correspondences in the composition are K -correspondences (or equivalently, are weak * upper semicontinuous correspondences taking nonempty compact values (US-COs), as we show here), then the composition correspondence takes contractible values in its set of selections where contractibility is with respect to the compatibly metrized weak * topologies. d It then follows from results in Gorniewicz, Granas, and Kryszewski [2] that the composition correspondence is approximable.
Regarding our second contribution, we first use our fixed point result to show, under mild conditions on primitives, that all nonatomic, parameterized state-contingent games (PSGs) have Blackwell equilibria. An excellent example of a nonatomic PSG is the oneshot game underlying an uncountable-compact nonatomic discounted stochastic game (DSG) satisfying the assumptions of Nowak and Raghavan [1]. By Blackwell's classical result [3] (extended to games), a DSG has a stationary Markov perfect equilibrium if and only if its underlying one-shot PSG has a Blackwell equilibrium. By our fixed point result any one-shot PSG belonging to a nonatomic Nowak-Raghavan DSG has a Blackwell equilibrium -therefore, by Blackwell [3], all nonatomic Nowak-Raghavan DSG have stationary Markov perfect equilibria.
We note that in a nonatomic DSG, because the underlying Nash selection correspondence is contractibly valued, the key pathology underlying the recent counterexamples to existence due to Levy [4] and Levy and McLennan [5] is ruled out. In particular, because the Nash selection correspondence is contractibly valued, Nash equilibria homeomorphic to the unit circle cannot arise. Also, we note that in the Levy and McLennan [5] discounted stochastic game model, the state space is not nonatomic. However, as noted by Jaskiewicz and Nowak [6], it is easy to modify the Levy-McLennan model so as to allow for a nonatomic state space while preserving the nonapproximability of the Nash payoff selection correspondence -thereby extending the Levy-McLennan counterexamples to the nonatomic case (also see Jaskiewicz and Nowak [7]). Here we will assume that the state space is nonatomic, and under this assumption, together with our result showing that the Nash payoff selection correspondence belonging to any Nowak-Raghavan DSGs is a K -correspondence, we will establish that the Nash payoff selection correspondence belonging to any Nowak-Raghavan DSGs is approximable and therefore has fixed points. Finally, we note that if a DSG has a Nash payoff selection correspondence that is approximable, then it will have fixed points, and therefore, by Blackwell's theorem [3] the DSG will have stationary Markov perfect equilibria (e.g., see Page [8,9]). By constructing a DSG having no stationary Markov perfect equilibria, and therefore, a DSG having a Nash payoff selection correspondence without fixed points, we can infer that the Nash payoff selection correspondence belonging to a Levy-McLennan DSG model is not approximable. Thus, while not all nonapproximable DSGs have SMPE, as shown by Levy [4] and Levy and McLennan [5], all approximable DSGs do, as shown by Page [8,9]. Moreover, as we show here, all DSGs satisfying the Nowak-Raghavan assumptions are approximable, and as a consequence, all such DSGs escape the Levy-McLennan counterexample, and possess stationary Markov perfect equilibria in behavioral strategies.

Spaces
Let ( , B , μ) be a probability space of states ω, where is a complete, separable metric space with probability measure μ defined on the Borel σ -field B , and let Y be a normbounded, weak * -closed (i.e., w * -closed), convex subset of F * , the separable norm dual of a separable Banach space F. Equip Y with metric ρ * Y compatible with the w * -topology on Y inherited from F * .
Next, let L ∞ Y denote the set of μ-equivalence classes of F * -valued, Bochner integrable Similarly, let X be a norm-bounded, weak * -closed (i.e., w * -closed), convex subset of E * , the separable norm dual of a separable Banach space E, and equip X with metric ρ * X compatible with the w * -topology on X inherited from E * . f Also, let L ∞ X denote the set of μequivalence classes of E * -valued, Bochner integrable functions x (·) , with x ω ∈ X a.e. [μ]. Denote by L ∞ X the set consisting of all E * -valued, Bochner integrable functions x (·) with x ω ∈ X a.e. [μ] (i.e., L ∞ X is the prequotient of L ∞ X ). We will equip L ∞ X with a metric ρ * L ∞ X compatible with the weak * topology inherited from L ∞ E * . We note that the set L ∞ X is decomposable.
Finally, equip the spaces Y and X with the Borel σ -fields B * Y and B * X , generated by the ρ * Y -and ρ * X -open sets in Y and X, respectively.

Convergences
We will begin by discussing W * -convergence in L ∞ F * and w * -K -convergence in its prequotient space, L ∞ F * . Then we will present a result clarifying how the two are related.

W * -Convergence in L ∞
F * By Diestel and Uhl (p. 98) [11], because F * is separable, L ∞ F * is the norm dual of L 1 F . We have the following definitions: In L ∞ Y a sequence {v n (·)} n W * -converges to v * (·) if and only ρ * (2020) 2020:14 Page 4 of 28 be the corresponding sequence of arithmetic mean functions. Because F * is convex for Definition 2 (w * -K -Sequences, w * -K -Convergence, and w * -K -Limits) We say that a se- By Theorem 3.2 in Balder [12], if the sequence, {v n (·)} n ⊂ L ∞ F * , is such that then there exists a subsequence {v n k (·)} k of {v n (·)} n and a function v(·) ∈ L ∞ F * such that any subsequence{v n kr (·)} r of {v n k (·)} k w * -K -converges to w * -K -limit, v(·).

W * -Convergence and w * -K -convergence
Our first result is about the relationship between W * -convergence and w * -K -convergence in ∞ Y .
Theorem 1 (W * -Convergence and w * -K -Convergence) Let {v n (·)} n be any sequence in L ∞ Y . Then the following statements are true: and by the dominated convergence theorem we have for each l ∈ L 1 F , Because {v n k (·)} k W * -converges to v * (·), we have for each l ∈ L 1 F , implying that for each l ∈ L 1 F , . Now we will prove part (1). By part (2), we can conclude that L ∞ Y is convex and W * - Let v ∞ (·) be any W * -limit point of the sequence {v n (·)} n and let {v n k (·)} k be a subsequence W * -converging to v ∞ (·). By w * -K -convergence we know that this subsequence also w * -K -converges to v(·), and hence by part (2)

Contractible continua
Let (X, ρ X ), (Y , ρ Y ), and (Z, ρ Z ) be metric spaces. Consider a function, f : Z − → X, from Z onto X. If f is continuous and one-to-one, and if its inverse f -1 is also continuous, then we say that f is a homeomorphism and that the metric spaces Z and X are homeomorphic. If (Z, ρ Z ) is compact, then any continuous, one-to-one mapping f from Z onto X is a homeomorphism. A continuous function f : Given metric space (Z, ρ Z ), a set E ⊆ Z is connected if E cannot be written as the union of two disjoint open sets (or two disjoint closed sets). A set E ⊆ Z is locally connected at e ∈ E if each neighborhood U e of e contains a connected neighborhood V e of e. Also E is locally connected if it is locally connected at each e ∈ E. g We note that in any metric space (Z, ρ Z ), the condition of being (i) a locally connected continuum and (ii) the continuous image of an interval are equivalent (this is the Mazurkiewicz-Moore theorem -see Kuratowski [13]).
If the metric space (Z, ρ Z ) is compact and connected, it is called a continuum. A nonempty, closed, connected subset of Z is called a subcontinuum. We will denote by C(Z) the collection of all subcontinua of Z. If in addition, the continuum (Z, ρ Z ) is locally connected, it is called a Peano continuum. Finally, (i) if the Peano continuum (Z, ρ Z ) is unicoherent, meaning that for all subcontinua A and B of Z such that Z = A ∪ B, the intersection A ∩ B is connected, and (ii) if all subcontinua of Z are unicoherent (i.e., if property (i) is hereditary), then (Z, ρ Z ) is a dendrite. Thus, (Z, ρ Z ) is a dendrite if it is a compact metric space that is connected, locally connected, and hereditarily unicoherent (see Charatonik and Charatonik [14] for more details). h A retraction r(·) is a continuous function from a space Z into Z such that r(·) is the identity on its range (i.e., r(r(z)) = r(z) for all z ∈ Z). A subset W of Z is said to be a retract of Z provided there is a retraction of Z onto W .
The notion of a homotopy will be important in what follows. A homotopy is a function that essentially provides us with a way to index a set of continuous functions. We have the following formal definition: The indexed collection, C h (Z, X), can be thought of as an arc, α h , in the continuum of continuous functions, C(Z, X), equipped with the sup metric. The continuous functions f and g in C(Z, X) are homotopically related, or homotopic, if f and g are the endpoints of an arc α h whose arc type is identified by some function, h ∈ C(Z × [0, 1], X). In particular, if f , g ∈ C(Z, X) are homotopic, then there is an arc of type h ∈ C(Z × [0, 1], X) running from continuous function f (·) = h(·, 0) to continuous function g(·) = h(·, 1). We denote this h-arc then f is said to be inessential. Moreover, if for some pair of compact metric spaces (Z, ρ Z ) and (X, ρ X ), all pairs of functions f , g ∈ C(Z, X) are homotopic, then, in particular, f , g x ∈ C(Z, X) are homotopic for some h-arc and some x ∈ X -and this means that for this pair of compact metric spaces, (Z, ρ Z ) and (X, ρ X ), all functions f ∈ C(Z, X) are inessential (i.e., We say that X is contractible if X is contractible in X. Note that if X is contractible, then for any Z ⊆ X, Z is contractible in X. Two useful facts related to the contractibility of continua are the following: (1) If X is contractible and Z ⊆ X is a retraction of X, then Z is also contractible. Thus if r : X − → Z, r ∈ C(X, Z) where r(z) = z for all z ∈ Z, then Z is also contractible. (2) If X is contractible, then X is unicoherent (see Corollary A.12.10 in van Mill [15])implying that all pairs of functions, f , g ∈ C(X, S 1 ), are homotopic for the unit circle, Thus, if X is contractible, then all continuous functions f : X − → S 1 are inessential, and we can conclude that X contains no simple closed curves. A compact metric space K is called an absolute retract (AR) provided that whenever K is embedded in a metric space Y , the embedded copy of K is a retract of Y . A compact metric space is called an absolute extensor (AE) provided whenever B is a closed subset of a metric space M and f : B − → K is continuous, then f can be extended to a continuous function F : M − → K (F being an extension of f means that F| B = f ). By Borsuk's theorem, a compact metric space K is an AR if and only if K is an AE (see Borsuk [16], see also, 9.1 in Illanes and Nadler [17]).
A compact metric space K is called an absolute neighborhood retract (ANR) provided that whenever K is embedded in a metric space Y the embedded copy K of K is a retract of some neighborhood of K in Y . A compact metric space K is called an absolute neighborhood extensor (ANE) provided that whenever B is a closed subset of a metric space M and f : B − → K is continuous, then there is a neighborhood U of B in M such that f can be extended to a continuous function F : U − → K . In parallel to Borsuk's theorem, a compact metric space K is an ANR if and only if K is an ANE (see Borsuk [16], see also, 19.5 in Illanes and Nadler [17]).

USCOs and approximability
For compact metric spaces (X, ρ X ) and (Y , ρ Y ), let U ρ Y -ρ X denote the collection of all upper semicontinuous correspondences (·) defined on Y taking nonempty, ρ X -closed (and hence ρ X -compact) values in X. Equip the product space Y × X with the sum metric, ρ Y ×X := ρ Y + ρ X . Following the literature, all such mappings are USCOs (e.g., see Hola and Holy [18]). We say that the USCO ( Bringing together the relevant parts of (5.6) and (5.12) from Gorniewicz, Granas, and Kryszewski [2], we have the following result concerning the approximability of USCOs: We will denote this special set of USCOs -USCOs with respect to weak * topologies -by U * or by USCO * . Also, we have the collection of all USCO * s with convex values (CUSCO * ).

K-correspondences and USCO * s
The formal definition of a K -correspondence is the following: Definition 4 (K -Limit Property and K -Correspondences) We say that a correspondence E(·) : L ∞ Y − → P(L ∞ X ) has the K -limit property, or is a K -correspondence, if for any sequence As our next result makes clear, because of the near equivalence of w * -K -convergence and W * -convergence in L ∞ Y and L ∞ X , K -correspondences and USCO * s are equivalent. We note that each

Contractibility
Our key result is the following: Given that μ is nonatomic, we know by Fryszkowski [19] that Lyapunov's theorem [20] on the range of a vector measure guarantees the existence of a family of measurable sets Using the properties of the system {A t } t∈ [0,1] , and the decomposability of We must show that In particular, we must show that for each l(·) ∈ L 1 E , For expressions (9(a)) and (9(b)) above we, have that and For expression (9(c)), because l(·) Because μ(A t n A t * ) − → 0, we have that M X l(·) 1 [ 1 r r q=1 μ(A t n kq A t * )] − → 0, implying that in expression (9) (a) − → 0 and (b) − → 0, and we have already from expression (12) that (c) − → 0. Thus, we have that for each l(·) ∈ L 1 E , . Thus, given the properties of the Lyapunov system (5) for each v, the function, h v (·, ·), given in (6) ·) is a homotopy for the set of measurable selections, E(v), and therefore, for each v, E(v) is contractible.
Our proof that E(v) is contractible is inspired by the contractibility result given by Mariconda [21] showing that if the underlying probability space is nonatomic, then any decomposable subset of E-valued, Bochner integrable functions in L 1 E is contractible (where E is a Banach space). In Mariconda's result, the space of functions is equipped with the norm in L 1 E , while here our space of functions (with each function taking values in X ⊂ E * ) is equipped with the W * topology -a topology metrized by ρ * L ∞ X .

Approximability
The importance of ( , B , μ) being nonatomic and E(·) being a K -correspondence is that together they guarantee that E(·) is an USCO * with contractible values (Theorem 3 above), and this in turn guarantees the ρ * , as our next result shows.
Proof By Lemma 1 and Theorem 3 above, E(·) is an USCO * taking contractible values. By Theorem 2 above (see 5.6 and 5.12 in Gorniewicz, Granas, and Kryszewski [2]), because E(·) is an USCO * defined on the ANR space of value functions L ∞ Y taking nonempty, compact, and contractible values in the ANR space,

K -Correspondences
Consider the correspondence U(·) from L ∞ Y with nonempty set values in L ∞ Y .

Theorem 5 (Fixed Point Theorem for
Because each of the functions, g n , is ρ *  [22], 17.56), that each g n has a fixed point, v n ∈ L ∞ Y (i.e., for each n there exists some v n ∈ L ∞ Y such that v n = g n (v n )). Let {v n } n be a fixed point sequence corresponding to the sequence of ρ * -continuous approximating functions, {g n (·)} n . Expression (14) can now be rewritten as follows: for each v n in the fixed point sequence, there is a corresponding pair, (v n , u n ) ∈ GrU (·), such that and therefore such that and therefore by part B of (15), as n − → ∞ we have

Composition correspondences
Finally, let us consider the correspondences, and the composition correspondence, Here, the notation P(L ∞ X ) denotes the collections of all nonempty subsets of L ∞ X (and similarly for P(L ∞ Y )). We will use the notation P * f (L ∞ X ) to denote the hyperspace of all nonempty, W * -closed (and hence, W * -compact) subsets of L ∞ X (and similarly for P * f (L ∞ Y )). Let denote the collection of all such upper semicontinuous correspondences. We note that, by Lemma 1 above, T (·, ·) is a K -correspondence. Next, let We can now state our main fixed point result.

Theorem 6 (Fixed Point Theorem for Compositions of
The following statements are true: , and, by Lemma 1, (2) Because ( , B , μ) is nonatomic and U(·) is a K -correspondence, we have by Theorem 5 above that U(·) has fixed points. Therefore, there exists

Parameterized, state-contingent games (PSGs) 4.1.1 Primitives, assumptions, and Blackwell equilibria of PSGs
We must first expand the number of spaces to take into account that now we have m players indexed by d = 1, 2, . . . , m. For each player d, let Y d be a closed bounded interval [-M, M], M > 0, the same for all players and equip Y d with the absolute value metric -a metric we will continue to denote by ρ * Y d . Let L ∞ R denote the Banach space of μ-equivalence classes of real-valued, essentially bounded measurable functions equipped with the weak * topology (i.e., the w * -topology), and let L ∞ Y d be the nonempty, convex, w * -compact, and metrizable subset of Equip the product spaces Y := Y 1 ×· · ·×Y m and X := X 1 ×· · ·×X m with the sum metrics, denoted by ρ * Y := d ρ * Y d and ρ * X := d ρ * X d , respectively. Also equip the product spaces Y and X with the Borel product σ -fields, , and again note that on the

Label the assumptions above [A-1].
A parameterized, state-contingent game (denoted by PSG) consists of a vparameterized collection of strategic form games, Letting ( , B , μ) be the underlying probability space of states ω, where is a complete, separable metric space, B is the Borel σ -field, and μ is a nonatomic probability measure, in the v-game G v , each player d ∈ D, seeks to choose a feasible state-contingent action (i.e., feasible strategy) so as to maximize player d 's expected payoff given player d's value function v d ∈ L ∞ Y d , and the state-contingent actions x -d(·) ∈ L ∞ X -d of the other players. i Also S ∞ ( d (·, v d )) denotes the collection of μ-equivalence classes of measurable selections of the state-contingent constraint correspondence, Here, Y := Y 1 × · · · × Y m ⊂ R m is the set of all possible player payoff profiles, and for each player d = 1, 2, . . . , m, we have for each possible state-contingent action profile ( We will assume the following concerning a parameterized game
An excellent example of a PSG satisfying assumptions [PSG-1] as well as assumptions [A-1] is the one-shot game underlying any nonatomic discounted stochastic game satisfying the Nowak-Raghavan assumptions (see Nowak and Raghavan [1]). We will provide just such an example below.
A Blackwell equilibrium of a parameterized game G L ∞ Y is defined as follows: By Theorem 2.2 in Hiai and Umegaki [23], we have that Thus, in state-contingent form, an equivalent way of writing down the definition of a Blackwell equilibrium of a parameterized game G L ∞ Y is the following: X is a Blackwell equilibrium provided for each player d and a.e. [μ] in ω, The first condition in expression (24) given players' Nash equilibrium strategy profile The second condition in expression (24) above requires that each player's strategy given player d s value function v d ∈ L ∞ Y d , and the strategies of the other players

Under assumptions [A-1] and [PSG-1], we know that each (ω, v)-game,
has a nonempty, ρ * X -compact set of Nash equilibria, denoted by N (ω, v). Moreover, by the Berge maximum theorem and the measurable maximum theorem (see 17.31 and 18.19 in Aliprantis and Border [22]), we know that the collection of (ω, v)-games G ×L ∞ N (ω, v) and an associated Nash payoff correspondence (ω, v) − → P(ω, v) that are upper Caratheodory (i.e., jointly measurable in (ω, v) and upper semicontinuous in v with nonempty compact values). Here, for each (ω, v) ∈ × L ∞ Y , we have that Let G ×L ∞ Y denote the collection of (ω, v)-games underlying the parameterized game Moreover, letting and (2020) 2020: 14 Page 16 of 28 be the Nash selection correspondences, we have that where S ∞ (P (·) ) is the Nash payoff selection correspondence and S ∞ (N (·) ) is the Nash selection correspondence.
Finally, we note that for any u ∈ S ∞ (P v ), we can deduce the existence of Nash strategy profile u(ω, v, x ω ) a.e. [μ] using implicit measurable selection (see Theorem 7.1 in Himmelberg [24]). Thus, a necessary and sufficient condition

Best strategy response correspondences
Given a Blackwell equilibrium (v * , x * (·) ), belonging to a parameterized G L ∞ Y , we have by the Nash condition above (26) and by Theorem 2.2 in Hiai and Umegaki [23] that for each player d and a.e. [μ] in ω, Under assumptions [A-1] and [PSG-1], we have for each player d that the strategy con- we have by Berge's maximum theorem (see 17.31 in Aliprantis and Border [22]) that player d 's best strategy response correspondence (i.e., the argmax correspondence) is upper semicontinuous and takes nonempty and ρ * is convex-valued, we have that player d 's best strategy response correspondence B d (·, ·) is a CUSCO * (i.e., is a convex valued USCO * ; see Hola and Holy [18]). Thus, the best strategy response cor- where for each player d = 1, . . . , m, is also a CUSCO * but one defined on L ∞ Y × L ∞ X with nonempty, convex, and ρ * implying that players' best strategy response correspondence B(v, ·) is a CUSCO * for each possible profile of player value functions v. Moreover, the players' best strategy response CUSCO * has associated with it a fixed point correspondence (see 17.28 in Aliprantis and Border [22]). By the Kakutani-Fan-Glicksberg fixed point theorem (see 17.55 in Aliprantis and Border [22]), we know that each v-BSR CUSCO * B(v, ·) : L ∞ X − → P * fc (L ∞ X ) has a nonempty, ρ * L ∞ X -compact set of fixed points. Thus, E B (·) takes nonempty values, and because B(·, ·) is a CUSCO * , and therefore because B(·, ·) has a weak * closed graph in is an USCO * (see Hola and Holy [18]) and takes decomposable values in L ∞ X . By Lemma 1 above E B (·) is a K -correspondence. It then follows from Theorems 1-6 above that the correspondence is the game's Nash selection correspondence and where S ∞ (P (·) ) is the game's Nash payoff selection correspondence, we know that the game has Blackwell equilibria, Thus, for any Blackwell equilibrium (v * , x * (·) ), we have, for each player d and for μ-almost As we will see in the next subsection, the key facts about a one-shot PSG belonging to a discounted stochastic game (DSG) is that it has an equilibrium strategy correspondence E B (·) and an equilibrium payoff function correspondence T (·, E B (·)), that are Kcorrespondences with decomposable values. These facts are particularly important because they guarantee that equilibrium payoff correspondence T (·, E B (·)) is an USCO * , and if μ is nonatomic then T (·, E B (·)) is an USCO * with contractible values -guaranteeing that T (·, E B (·)) is approximable -and finally, that T (·, E B (·)) has fixed points in L ∞ Y . Moreover, given that for all v ∈ L ∞ Y , for the Nash selection correspondences S ∞ (N (·) ) and S ∞ (P (·) ), belonging to the parameterized game G L ∞ Y , we have that the Nash payoff selection correspondence S ∞ (P (·) ) is approximable and has fixed points.

Discounted stochastic games (DSGs)
For the one-shot PSG underlying any discounted stochastic game DSG NR , satisfying the Nowak-Raghavan assumptions, each v-game G NR v is specified by the primitives The  ( d (ω)) a.e.

Main conclusions
Focusing on composition correspondences of the following type: where are K -correspondences, we show that if the probability space of states ( , B , μ), underlying the function spaces L ∞ Y and L ∞ X , is equipped with a nonatomic probability measure μ, then for each v ∈ L ∞ Y , T (v, E(v)) is contractible with respect to the compatibly metrized weak * topologies, and, as a consequence, the composition correspondence T (·, E(·)) is approximable and has fixed points.
Next, we consider a discounted stochastic game DSG, satisfying the Nowak-Raghavan assumptions, with underlying parameterized, state-contingent, one-shot game G NR L ∞ Y , with Nash selection correspondence v − → S ∞ (N v ), and Nash payoff selection correspondence v − → S ∞ (P v ). We show that if the correspondence E(·) in expression (52) is given by is the best strategy response correspondence (a CUSCO * ) belonging to the DSG s parameterized, state-contingent, one-shot game G NR L ∞ Y , then E B (·) and T (·, E B (·)) are K -correspondences with Therefore, if the DSG has a probability space of states ( , B , μ), equipped with a nonatomic probability measure μ, then, by our Theorem 6, the Nash payoff selection correspondence v − → S ∞ (P v ) = T (v, E B (v)) has fixed points. By Blackwell's theorem [3], it then follows that the nonatomic, Nowak-Raghavan DSG to which S ∞ (P (·) ) belongs has stationary Markov perfect equilibria. All of this is true provided the DSG satisfies the Nowak-Raghavan assumptions. It then suffices to show that each player's state-contingent payoff function is sequentially weak * continuous. Thus, we show in Theorem 7 that if v n − → ρ w * v * and σ n (·) ⇒ ρ W * ca σ * (·), then u(·, v n , σ n (·)) − → ρ w * u(·, v * , σ * (·)).

Appendix 1: Measurability and integrability of Banach space valued functions
Recall the set-up: ( , B , μ) is a probability space of states (not necessarily nonatomic), where is Polish with probability measure μ defined on the Borel σ -field B , and Y is a norm-bounded, weak * -closed (i.e., w * -closed), convex subset of F * , the norm dual of a separable Banach space F, with F * having the Radon-Nikodym property.
(1) Because F is separable, F * is (norm-) separable if and only if F * has the Radon-Nikodym property (Bourgin [27], Theorem 5. where B w * is the Borel σ -field generated by the w * -topology in F * . By Lemma 11.37 in Aliprantis and Border [22], if v(·) is strongly measurable, then v(·) is (B , B w * )-measurable. By the Pettis measurability theorem (Diestel and Uhl [11], p. 42) if v( \N) is norm separable for N ∈ B with μ(N) = 0 (i.e., off a set of μ-measure zero), then the range of v(·) is norm separable and if v(·) is (B , B w * )-measurable, then v(·) is strongly measurable. In addition, by Proposition A.1 in Cornet and Martin-da-Rocha [28], v(·) is (B , B w * )measurable if and only if v(·) is scalar measurable. Thus, letting L ∞ Y (the prequotient of L ∞ Y ) be the set of all (B , B w * )-measurable functions defined on taking values a.e.
[μ] in the w * -closed and · * -bounded subset Y of the norm dual F * , we have for each v(·) ∈ L ∞ Y that v(·) is strongly measurable because v( \N) ⊆ Y for N ∈ B with μ(N) = 0 and, by Theorem 7.7 in Kahn [29], Y is · * -separable. Thus, each function v(·) in the prequotient space L ∞ Y of (B , B w * )-measurable functions defined on and taking values a.e.
[μ] in the w * -compact subset Y of the norm dual F * is not only (B , B w * )-measurable, but also scalarly measurable, as well as strongly measurable.
Because F * has the Radon-Nikodym property, it follows from Theorem 1 of Diestel and Uhl [11] (p. 98) that L ∞ F * is the norm dual of L 1 F .