GitHub

Value Function Iteration

    Introduction

    The Value Function Iteration (VFI) algorithm is a method to solve dynamic programming problems. It is particularly useful when the state space is low-dimensional. The VFI algorithm is a brute-force method that iteratively applies the Bellman operator to the value function until convergence. The VFI algorithm is computationally expensive because it requires solving the Bellman equation at each iteration. However, it is a simple and robust method that can be used as a benchmark for more sophisticated algorithms.

    API

    BellmanSolver.VFI.do_VFI — Method
    do_VFI(
    flow_value, k_grid, p_grid, trans_mat, β;
    tol=1e-6, max_iter=1000, kwargs...
)

    Do value function iteration for the case where we optimise over one variable, and there is one stochastic variable in the value function. The stochastic variable is represented by a markov process on a grid.

    Arguments

    • flow_value::Function: Function to compute the flow value
    • k_grid::Vector{Float64}: Grid points for the capital stock
    • p_grid::Vector{Float64}: Grid points for the price of capital
    • trans_mat::Array{Float64, 2}: Transition matrix for the price process
    • β::Real: Discount factor
    • tol::Real=1e-6: Tolerance for convergence
    • max_iter::Int=1000: Maximum number of iterations
    • kwargs...: Additional keyword arguments for flow_value

    Returns

    • k_grid::Vector{Float64}: Grid points for the capital stock
    • p_grid::Vector{Float64}: Grid points for the price of capital
    • Kp_mat::Array{Float64, 2}: Policy function for the capital stock at time t+1
    • V::Array{Float64, 2}: Value function
    source
    BellmanSolver.VFI.do_VFI — Method
    do_VFI(flow_value, k_grid, β, interp; tol=1e-6, max_iter=1000, kwargs...)

    Do Value Function Iteration for the case where we optimise over one variable and there are no stochastic elements, using interpolation to optimise over an interval for each grid point.

    Arguments

    • flow_value::Function: Function to calculate the flow value
    • k_grid::Vector{Float64}: Grid points for the choice variable
    • β::Real: Discount factor
    • interp::AbstractInterpolator: Interpolator to use
    • tol::Real=1e-6: Tolerance for convergence
    • max_iter::Int=1000: Maximum number of iterations
    • kwargs...: Keyword arguments to pass to flow_value. Should be parameters of flow_value

    Returns

    • k_grid::Vector{Float64}: Grid points for the choice variable
    • kp_vct::Vector{Float64}: Optimal choice
    • V::Vector{Float64}: Value function
    source
    BellmanSolver.VFI.do_VFI — Method
    do_VFI(flow_value, k_grid, β; tol=1e-6, max_iter=1000, kwargs...)

    Do value function iteration for the case where we optimise over one variable, and there are no stochastic elements.

    Arguments

    • flow_value::Function: Function to calculate the flow value
    • k_grid::Vector{Float64}: Grid points for the choice variable
    • β::Real: Discount factor
    • tol::Real=1e-6: Tolerance for convergence
    • max_iter::Int=1000: Maximum number of iterations
    • kwargs...: Keyword arguments to pass to flow_value. Should be parameters of flow_value

    Returns

    • k_grid::Vector{Float64}: Grid points for the choice variable
    • kp_vct::Vector{Float64}: Optimal choice
    • V::Vector{Float64}: Value function
    source