contents.m

approximate reinforcement learning

% Approximate reinforcement learning functions
%
% fuzzyqi           - Implements model-based, offline Q-iteration with fuzzy approximation
% tilingqi          - Tile-coding Q-iteration
% startupapproxrl   - Startup script for the approximate RL package
%
% Subdirectories:
% /systems          - Contains system (problem) definitions for approximate RL
% /tiling           - Tiling implementation
% /demo             - Demonstration scripts
% /priv				- Private functions (plot history)

% ----------------------
% Other information of general interest follows

% Conventional variable names used in approximate RL scripts:
% 
% Vectors / structures:
% theta       - parameter vector / matrix of the approximator
% PHI`        - activation matrix for basis functions approximator
% P           - projection matrix, if needed
% MDP         - precomputed MDP structure. Field F record the next state (or activation of next state), 
%               R the reward for all (xi, uj) in X0 x U0
% 
% Dimensionality:
% DIMS        - structure recording all dimensions needed in the problem
% X, U        - for state /input quantization for the approximator
% X0, U0      - for state /input samples if different from the quantization
% N, M        - number of params of the approximator accross state and action dimensions
% N0, M0      - number of state / input samples, if different from the above
% dimx, dimu  - n-d dimensionality of X and U
% dimx0, dimu0- n-d dimensionality of X0 and U0, if different from the above
% p, q        - number of state / output variables (also indices)
% i, j        - flat indices of state/output sample, or state/input approximator parameters
% ii, jj      - n-d indices of state/output sample, or state/input approximator parameters