我是强化学习和pytorch的新手。我正在向Udemy学习。但是,我的代码与显示的代码相同,但是出现错误。我猜这是一个pytorch错误,但无法调试。如果有人帮助,我们将不胜感激。
import gym
import time
import torch
import matplotlib.pyplot as plt
from gym.envs.registration import register
register(id='FrozenLakeNotSlippery-v0',entry_point='gym.envs.toy_text:FrozenLakeEnv',kwargs={'map_name': '4x4', 'is_slippery':False})
env = gym.make('FrozenLakeNotSlippery-v0')
number_of_states = env.observation_space.n
number_of_actions = env.action_space.n
Q = torch.zeros([number_of_states,number_of_actions])
num_episodes = 1000
steps_total = []
gamma = 1
for i in range(num_episodes):
state = env.reset()
step = 0
while True:
step += 1
#action = env.action_space.sample()
random_values = Q[state]+torch.rand(1,number_of_actions)/1000
action = torch.max(random_values,1)[1][0]
new_state, reward, done, info = env.step(action)
Q[state, action] = reward + gamma * torch.max(Q[new_state])
state = new_state
#time.sleep(0.4)
#env.render()
if done:
steps_total.append(step)
print ("Episode Finished after %i steps" %step)
break
print ("Average Num Steps: %2f" %(sum(steps_total)/num_episodes))
plt.plot(steps_total)
plt.show()
我遇到的错误是以下错误
KeyError Traceback (most recent call last)
<ipython-input-11-a6aa419c3767> in <module>
8 random_values = Q[state]+torch.rand(1,number_of_actions)/1000
9 action = torch.max(random_values,1)[1][0]
---> 10 new_state, reward, done, info = env.step(action)
11 Q[state, action] = reward + gamma * torch.max(Q[new_state])
12 state = new_state
c:\users\souradip\appdata\local\programs\python\python36\lib\site-packages\gym\envs\toy_text\discrete.py in step(self, a)
53
54 def step(self, a):
---> 55 transitions = self.P[self.s][a]
56 i = categorical_sample([t[0] for t in transitions], self.np_random)
57 p, s, r, d= transitions[i]
KeyError: tensor(3)
答案 0 :(得分:1)
下面的代码
action = torch.max(random_values,1)[1][0]
得到一个0-dim
张量,但是env.step()
期望一个python数,它基本上是动作空间中的一个动作。因此,如评论中的@a_guest所示,请使用a.item()
将0-dim
张量转换为如下所示的python数字:
new_state, reward, done, info = env.step(action.item())