Second order Markov with toolkit

1

I am interested in solving a lifecycle model with health shocks where health follows a second-order Markov process (as a standard reference, see De Nardi et al. Lifetime costs of bad health: https://academic.oup.com/restud/article-abstract/92/3/1987/7730359 )

Health is denoted by h_t, where

h_t \in \{0,1\},

with

0 = \text{bad health}, \qquad 1 = \text{good health}.

For a second-order Markov process, the transition probability depends on both current and lagged health:

\Pr(h_{t+1}\mid h_t,h_{t-1}).

Equivalently, define the enlarged state variable

s_t = (h_{t-1},h_t).

Since h_t takes two possible values, the enlarged state s_t takes four possible values:

s_t \in \{(0,0),(0,1),(1,0),(1,1)\}.

To me it seems that the most straightforward way of setting this up in the toolkit is to use s_t as exogenous Markov state with a 4*4 transition matrix.

This matrix will have some zeros of course: for example, if the current state is (0,0), the next state can only be (0,0) or (0,1).

Does this make sense? Suggestions are welcome?

A related question: in my example, health takes only two values, but what if it is a continuos variable like an AR(2)? Any toolkit example with AR(2)?
Thanks!

1 Like
2

As you say, any second-order markov, call it z, that has a one-dimensional state Pr(z_t|z_{t-1},z_{t-2}) can be rewritten as a first-order markov with a two dimensional state, defining y_t=[z_t, z_{t-1}]' we get that y is a first-order markov Pr(y_t|y_{t-1}).

Note that the transition matrix for y will always be pretty sparse as the transitions from z_{t-2} to z_{t-1} are just a diagonal.

This is standard everywhere, and is also what you should do for VFI Toolkit.

An example is in the Intro to Life-Cycle Models:
A.14 Life-Cycle Model A10: Second-Order Markov Processes (implementing an
AR(2) persistent shock)

3 Likes