Q-learning：奖励计算的正确状态是什么

Question

Q学习 - 奖励

我正在努力解释Q学习算法的伪代码：

1  For each s, a initialize table entry Q(a, s) = 0
2  Observe current state s
3  Do forever:
4     Select an action a and execute it
5     Receive immediate reward r
6     Observe the new state s′ ← δ(a, s)
7     Update the table entry for Q(a, s) as follows:
8        Q( a, s ) ← R( s ) + γ * max Q( a′, s′ )
9     s ← s′

是否应从后续状态s'或当前状态s收集奖励？

Answer 1

应在执行操作a后输入的后续状态中收集奖励。