I typed a relatively lazy prompt in chatgpt asking how to solve the stochastic neoclassical growth model with the vfi toolkit developed by R Kirkby. I was curious to see how well AI is aware of the toolkit. This is the answer and is pretty good, given that I got it at the first try: no follow-up interactions, no context like “where to find the toolkit repo”, how it works etc.
My prompt:
Can you explain how to solve a value function iteration problem (e.g. neoclassical growth model with tfp shocks) using the vfi toolkit by Robert Kirkby?
Chatgpt answer:
Solving a VFI Problem with VFI Toolkit
The key idea is: with VFI Toolkit you do not write the Bellman loop yourself. You give the toolkit:
- grids for endogenous and exogenous states;
- the Markov transition matrix for shocks;
- a return function;
- parameter names;
- options;
and then call ValueFnIter_Case1.
The toolkit is designed for fully discretized dynamic programming problems and can automatically use CPU/GPU parallelization.
1. Example model
Take the stochastic neoclassical growth model:
V(k,z)=\max_{k'\in \mathcal K} \left\{ u(c)+\beta \sum_{z'} \pi(z'|z)V(k',z') \right\},
subject to
c = \exp(z) k^\alpha + (1-\delta)k - k',
with
z' \sim \Pi(z'|z).
Here:
- endogenous state: (k);
- exogenous state: (z);
- endogenous next-period state / choice: (k’);
- no additional static choice (d).
In VFI Toolkit terminology, this is a Case 1 problem: one endogenous state that evolves according to the chosen next-period value, plus an exogenous Markov state.
2. Write the return function
The return function should take current state values and choice values as scalar inputs. For this model, the return function is something like:
function F = StochasticGrowth_ReturnFn(kprime_val, k_val, z_val, gamma, alpha, delta)
F = -Inf;
c = exp(z_val)*k_val^alpha + (1-delta)*k_val - kprime_val;
if c > 0
if gamma == 1
F = log(c);
else
F = c^(1-gamma)/(1-gamma);
end
end
end %end function
The important convention is:
ReturnFn(kprime, k, z, parameters...)
because (k’) is the choice and (k,z) are current states.
3. Define parameters
You typically put parameters in a structure:
Params.beta = 0.96;
Params.gamma = 2.0;
Params.alpha = 0.36;
Params.delta = 0.08;
Then you tell the toolkit which parameter enters the discount factor:
DiscountFactorParamNames = {'beta'};
and which parameters enter the return function:
ReturnFnParamNames = {'gamma','alpha','delta'};
This is one of the toolkit’s main design choices: instead of hard-coding parameters into the return function, you tell the toolkit which fields of Params to pass to the return function.
4. Create grids
For capital:
n_k = 500;
k_min = 0.1;
k_max = 50;
k_grid = linspace(k_min, k_max, n_k)';
For TFP shocks, suppose you already discretized an AR(1)
z' = \rho z + \varepsilon', \qquad \varepsilon' \sim N(0,\sigma^2).
You need:
z_grid % n_z by 1 vector
pi_z % n_z by n_z transition matrix
For example:
n_z = 7;
rho = 0.9;
sigma = 0.02;
[z_grid, pi_z] = TauchenMethod(n_z, 0, rho, sigma, 3);
The exact Tauchen/Rouwenhorst function depends on what you use. The important point is that both (k) and (z) must be discretized.
5. Tell the toolkit the dimensions
For this problem:
n_d = 0; % no additional discrete/static decision d
n_a = n_k; % endogenous state grid size
n_z = length(z_grid);
The toolkit uses the notation a for the endogenous state, so in a growth model you can think of
a \equiv k.
Then:
d_grid = [];
a_grid = k_grid;
6. Call ValueFnIter_Case1
The core call is:
ReturnFn = @StochasticGrowth_ReturnFn;
vfoptions = struct();
[V, Policy] = ValueFnIter_Case1( ...
n_d, n_a, n_z, ...
d_grid, a_grid, z_grid, pi_z, ...
ReturnFn, Params, ...
DiscountFactorParamNames, ReturnFnParamNames, ...
vfoptions);
Conceptually, this computes:
So V is the value function on the grid, and Policy gives the optimal index/value of (k’), depending on the toolkit version and options. In most VFI Toolkit examples, the policy is stored in an index form that you can convert back to levels using the relevant grid.
7. Interpret the policy
Suppose Policy contains indices of optimal (k’). Then you can recover the policy in levels as:
KprimePolicy = k_grid(Policy);
Then you can plot, for example, the policy at the median TFP shock:
z_mid = ceil(n_z/2);
figure;
plot(k_grid, KprimePolicy(:,z_mid), 'LineWidth', 2);
xlabel('k');
ylabel('k'' policy');
title('Capital policy function');
You can also compute consumption:
CPolicy = zeros(n_k,n_z);
for iz = 1:n_z
CPolicy(:,iz) = exp(z_grid(iz))*k_grid.^Params.alpha ...
+ (1-Params.delta)*k_grid ...
- KprimePolicy(:,iz);
end
8. Full skeleton
Putting the main pieces together:
clear; clc;
%% Add VFI Toolkit to path
% addpath(genpath('C:\path\to\VFIToolkit-matlab'));
%% Parameters
Params.beta = 0.96;
Params.gamma = 2.0;
Params.alpha = 0.36;
Params.delta = 0.08;
DiscountFactorParamNames = {'beta'};
ReturnFnParamNames = {'gamma','alpha','delta'};
%% Capital grid
n_k = 500;
k_min = 0.1;
k_max = 50;
k_grid = linspace(k_min, k_max, n_k)';
%% TFP grid
n_z = 7;
rho = 0.9;
sigma = 0.02;
% You need to provide this using your preferred Tauchen/Rouwenhorst function.
% z_grid: n_z by 1
% pi_z: n_z by n_z
[z_grid, pi_z] = TauchenMethod(n_z, 0, rho, sigma, 3);
%% VFI Toolkit dimensions
n_d = 0;
n_a = n_k;
d_grid = [];
a_grid = k_grid;
%% Return function
ReturnFn = @StochasticGrowth_ReturnFn;
%% Options
vfoptions = struct();
%% Solve
[V, Policy] = ValueFnIter_Case1( ...
n_d, n_a, n_z, ...
d_grid, a_grid, z_grid, pi_z, ...
ReturnFn, Params, ...
DiscountFactorParamNames, ReturnFnParamNames, ...
vfoptions);
%% Recover policy in levels, if Policy stores indices
KprimePolicy = k_grid(Policy);
%% Plot
z_mid = ceil(n_z/2);
figure;
plot(k_grid, KprimePolicy(:,z_mid), 'LineWidth', 2);
xlabel('k');
ylabel('k''');
title('Capital policy function, median TFP state');
and the return function file:
function F = StochasticGrowth_ReturnFn(kprime_val, k_val, z_val, gamma, alpha, delta)
F = -Inf;
c = exp(z_val)*k_val^alpha + (1-delta)*k_val - kprime_val;
if c > 0
if gamma == 1
F = log(c);
else
F = c^(1-gamma)/(1-gamma);
end
end
end %end function
9. What the toolkit is doing internally
Internally, the toolkit is essentially doing what you would do by hand:
- For each current state ((k,z)),
- loop over feasible (k’) on the grid,
- compute current utility,
- compute expected continuation value,
- maximize over (k’),
- iterate until convergence.
The advantage is that you do not manually write the large loops, the expectation step, or the CPU/GPU parallelization.
10. Relation to hand-written VFI
Hand-written VFI:
for ik = 1:n_k
for iz = 1:n_z
for ikp = 1:n_k
...
end
end
end
VFI Toolkit version:
[V, Policy] = ValueFnIter_Case1(...);
But you still need to supply the economic structure correctly:
| Object | Meaning in growth model |
|---|---|
a_grid |
capital grid (k) |
z_grid |
TFP grid (z) |
pi_z |
Markov transition matrix for (z) |
ReturnFn |
one-period utility (u(c)) |
Params |
parameters |
DiscountFactorParamNames |
usually {'beta'} |
ReturnFnParamNames |
e.g. {'gamma','alpha','delta'} |
Policy |
optimal (k’) choice, usually as grid index |
The main mental shift is that (k’) is not listed as a separate decision grid d_grid. In Case 1, the next-period endogenous state (a’) is automatically treated as a choice over a_grid. You only use d_grid for additional choices, such as labor (n), portfolio shares, default decisions, etc.