q_from_v.m

来自「实现马尔可夫决策过程模型的算法」· M 代码 · 共 12 行

function Q = Q_from_V(V, T, R, discount_factor)
% Q(s,a) = R(s,a) + sum_s' T(s,a,s') * gamma * V(s')

S = size(T,1);
A = size(T,2);
Q = zeros(S,A);

for a=1:A
  Q(:,a) = R(:,a) + squeeze(T(:,a,:))*discount_factor*V;
end