Question

我正在尝试使用OpenAI Gym进行多处理。我知道那里有现成的解决方案，但是我这样做是为了在Gym和multiprocessing上得到一些实践。

问题是我正在运行一个错误，其原因我无法确定。这是代码：

import gym
import numpy as np
import multiprocessing as multip


class Environments:
    gym_environment = 'CarRacing-v0'

    def __init__(self, environments_n):
        self.environments_n = environments_n
        self.cores_count = multip.cpu_count()
        self.envs = []
        self.reset_all_environments()

    def __enter__(self):
        self.pool = multip.Pool(self.cores_count)
        return self

    def __exit__(self, exc_type, exc_value, tb):
        self.pool.close()
        self.pool.join()

    def reset_all_environments(self):
        for env in self.envs:
            env.close()
        self.envs = [gym.make(self.gym_environment) for _ in range(self.environments_n)]
        self.dones = [False for _ in range(self.environments_n)]
        self.last_obs = [env.reset() for env in self.envs]

    @staticmethod
    def step_single(env, action):
        print(action, env)
        return env.step(action)

    def step_all(self):
        actions = [env.action_space.sample() for env in self.envs]
        results = self.pool.starmap(self.step_single, zip(self.envs, actions))


if __name__ == '__main__':
    with Environments(2) as envs:
        print(envs.step_all())

在我期望的情况下，这应该只在创建的2个环境中运行一个步骤，但是我却得到了这个错误：

Track generation: 1134..1423 -> 289-tiles track
retry to generate track (normal if there are not many of this messages)
Track generation: 1153..1445 -> 292-tiles track
Track generation: 1188..1492 -> 304-tiles track
retry to generate track (normal if there are not many of this messages)
Track generation: 1233..1554 -> 321-tiles track
[0.01238655 0.20499504 0.07908794] <TimeLimit<CarRacing instance>>
[0.8294716 0.5289044 0.7956768] <TimeLimit<CarRacing instance>>
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 47, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "C:\Users\Nadni\PycharmProjects\ActorCritic\test.py", line 33, in step_single
    return env.step(action)
  File "C:\Program Files\Python37\lib\site-packages\gym\wrappers\time_limit.py", line 15, in step
    observation, reward, done, info = self.env.step(action)
  File "C:\Program Files\Python37\lib\site-packages\gym\envs\box2d\car_racing.py", line 315, in step
    self.car.steer(-action[0])
AttributeError: 'NoneType' object has no attribute 'steer'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "test.py", line 42, in <module>
    print(envs.step_all())
  File "test.py", line 37, in step_all
    results = self.pool.starmap(self.step_single, zip(self.envs, actions))
  File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 276, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 657, in get
    raise self._value
AttributeError: 'NoneType' object has no attribute 'steer'
Exception ignored in: <function Viewer.__del__ at 0x000002DCC71E8A60>
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\site-packages\gym\envs\classic_control\rendering.py", line 152, in __del__
  File "C:\Program Files\Python37\lib\site-packages\gym\envs\classic_control\rendering.py", line 71, in close
  File "C:\Program Files\Python37\lib\site-packages\pyglet\window\win32\__init__.py", line 305, in close
  File "C:\Program Files\Python37\lib\site-packages\pyglet\window\__init__.py", line 770, in close
ImportError: sys.meta_path is None, Python is likely shutting down

即使从print（）可以清楚地知道AttributeError: 'NoneType' object has no attribute 'steer'变量正确地是env的成员，我也不知道为什么会说<TimeLimit<CarRacing instance>>

Answer 1

我不确定是什么原因导致此问题，但是使用Ray时遇到了相同的错误。我通过如下更改来“修复”它：

抛出错误的行：

env = gym.make(env_name).unwrapped
env.step([0,0,0])

不引发错误的行：

env = gym.make(env_name)
env.reset()
env.step([0,0,0])

希望这可以帮助某人跌倒。

在OpenAI Gym上实施多重处理时出错，“ NoneType”对象没有属性“ steer”

1 个答案: