GitHub

Markov Chain

Introduction

The markov chain model provides methods to set up discretised random processes for the random variable in dynamic programming problems. We use the tauchen method to create a markov chain from a continuous diffusion process.

API

BellmanSolver.Markov.make_deterministic_chain — Method
make_deterministic_chain(N, min, max)

Make a deterministic markov chain chain with N grid points between min and max.

This will be used to discretise the price state space in the deterministic case. While, strictly speaking, it might be both more readable and more performant to modify the value function instead, it is easier to code this method, and the performance of my algorithm is not currently a bottleneck.

This would be an identity matrix, but we need to account for the fact that η = 0.95, so prices will decay back to 1.

Arguments

  • N::Int: Number of grid points
  • min::Real: Minimum value of the grid
  • max::Real: Maximum value of the grid

Returns

  • y_grid::Vector{Float64}: Grid points for the state space
  • trans_mat::Matrix{Float64}: Transition matrix for the markov chain
source
BellmanSolver.Markov.tauchen — Method
tauchen(N, m, μ, σ, λ)

Tauchen's method for discretizing a state space for a AR(1) process.

Arguments

  • N::Int: Number of grid points
  • m::Real: Multiple of standard deviation for the grid
  • μ::Real: Drift of the AR(1) process
  • σ::Real: Standard deviation of the error term of the AR(1) process
  • λ::Real: Persistence parameter of the AR(1) process

Returns

  • y_grid::Vector{Float64}: Grid points for the state space
  • trans_dict::Dict{Float64, Vector{Float64}}: Transition matrix for the AR(1)

process

source
BellmanSolver.Markov.tauchen_unit_root — Method
tauchen_unit_root(N, m, σ)

Tauchen's method for discretizing a state space for a unit root AR(1) process.

Using a separate method as I am unsure how to handle the mean.

Arguments

  • N::Int: Number of grid points
  • m::Real: Multiple of standard deviation for the grid
  • σ::Real: Standard deviation of the error term of the AR(1) process

Returns

  • y_grid::Vector{Float64}: Grid points for the state space
source