Question

0

使用来自pyBrain的强化学习我们正试图解决一个游戏。我们使用NFQ和ActionValueNetwork作为控制器。我们有自己的任务，正在使用pyBrain的实验设置来训练。我们的经纪人会在每集后重置。

我们看到一集的方式是在一个级别上运行。每一集我们称为experiment.doInteractions（），直到级别返回失败或完成。之后我们执行agent.learn（）和agent.reset（）

起初我们认为这个错误是由于因为运行一集太长而造成的内存溢出。当我们第一次遇到错误时，我们插入的移动最大值为10.000。我现在把它降低到50，这使程序运行更多剧集，但仍然给出了错误。

这让我们相信，不是移动的数量是问题，而是累积历史被保存在某处并且过度增加其允许的内存大小。我们相信agent.reset（）会从记忆中清除所有历史，这不是这样吗？我们可以阻止历史被保存在某个地方吗？

或者有人有其他建议吗？

编辑：这是异常追踪。

Running level: Single box block. Try: 125
Traceback (most recent call last):
  File "wrapperSokoban.py", line 168, in <module>
    wrapperSokoban().run_Qlearning()
  File "wrapperSokoban.py", line 154, in run_Qlearning
    self.run_Qlearning_level() 
  File "wrapperSokoban.py", line 122, in run_Qlearning_level
    self.experiment.doInteractions()
  File "/usr/local/lib/python2.7/dist-packages/PyBrain-0.3.1-py2.7.egg/pybrain/rl/experiments/experiment.py", line 18, in doInteractions
    self._oneInteraction()
  File "/usr/local/lib/python2.7/dist-packages/PyBrain-0.3.1-py2.7.egg/pybrain/rl/experiments/experiment.py", line 29, in _oneInteraction
    self.agent.giveReward(reward)
  File "/usr/local/lib/python2.7/dist-packages/PyBrain-0.3.1-py2.7.egg/pybrain/rl/agents/logging.py", line 62, in giveReward
    self.history.addSample(self.lastobs, self.lastaction, self.lastreward)
  File "/usr/local/lib/python2.7/dist-packages/PyBrain-0.3.1-py2.7.egg/pybrain/datasets/reinforcement.py", line 40, in addSample
    self.appendLinked(state, action, reward)
  File "/usr/local/lib/python2.7/dist-packages/PyBrain-0.3.1-py2.7.egg/pybrain/datasets/dataset.py", line 216, in appendLinked
    self._appendUnlinked(l, args[i])
  File "/usr/local/lib/python2.7/dist-packages/PyBrain-0.3.1-py2.7.egg/pybrain/datasets/dataset.py", line 196, in _appendUnlinked
    self._resize(label)
  File "/usr/local/lib/python2.7/dist-packages/PyBrain-0.3.1-py2.7.egg/pybrain/datasets/dataset.py", line 181, in _resize
    self.data[l] = self._resizeArray(self.data[l])
  File "/usr/local/lib/python2.7/dist-packages/PyBrain-0.3.1-py2.7.egg/pybrain/datasets/dataset.py", line 188, in _resizeArray
    return resize(a, shape)
  File "/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 864, in resize
    a = concatenate( (a,)*n_copies)
MemoryError

运行pyBrain NFQ学习器几分钟后出现内存错误

0 个答案: