Pandas与多线程不一致“从文件初始化失败”

时间:2014-09-02 13:51:58

标签: python python-2.7 pandas

我有一些代码如下:

df = pd.read_csv(filepath, header=None, parse_dates=[[0,1]])

这是从看门狗线程调用的。另一个进程每分钟写一次文件,并且 然后通过看门狗" on_modified"来调用上面的代码。并做一些处理。 它工作了几次,大部分时间我得到以下追溯:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "E:\Portable Python 2.7.6.1\App\Lib\threading.py", line 810, in __bootstr
ap_inner
    self.run()
  File "E:\Portable Python 2.7.6.1\App\lib\site-packages\watchdog-0.8.1-py2.7.eg
g\watchdog\observers\api.py", line 236, in run
    self.dispatch_events(self.event_queue, self.timeout)
  File "E:\Portable Python 2.7.6.1\App\lib\site-packages\watchdog-0.8.1-py2.7.eg
g\watchdog\observers\api.py", line 406, in dispatch_events
    self._dispatch_event(event, watch)
  File "E:\Portable Python 2.7.6.1\App\lib\site-packages\watchdog-0.8.1-py2.7.eg
g\watchdog\observers\api.py", line 401, in _dispatch_event
    handler.dispatch(event)
  File "E:\Portable Python 2.7.6.1\App\lib\site-packages\watchdog-0.8.1-py2.7.eg
g\watchdog\events.py", line 343, in dispatch
    _method_map[event_type](event)
  File "tx_stats.py", line 272, in on_modified
    dur, tail_skip_count)
  File "tx_stats.py", line 209, in read_file
    df = pd.read_csv(filepath, header=None, parse_dates=[[0,1]])
  File "E:\Portable Python 2.7.6.1\App\lib\site-packages\pandas-0.14.1-py2.7-win
32.egg\pandas\io\parsers.py", line 452, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "E:\Portable Python 2.7.6.1\App\lib\site-packages\pandas-0.14.1-py2.7-win
32.egg\pandas\io\parsers.py", line 234, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "E:\Portable Python 2.7.6.1\App\lib\site-packages\pandas-0.14.1-py2.7-win
32.egg\pandas\io\parsers.py", line 542, in __init__
    self._make_engine(self.engine)
  File "E:\Portable Python 2.7.6.1\App\lib\site-packages\pandas-0.14.1-py2.7-win
32.egg\pandas\io\parsers.py", line 679, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "E:\Portable Python 2.7.6.1\App\lib\site-packages\pandas-0.14.1-py2.7-win
32.egg\pandas\io\parsers.py", line 1041, in __init__
    self._reader = _parser.TextReader(src, **kwds)
  File "parser.pyx", line 332, in pandas.parser.TextReader.__cinit__ (pandas\par
ser.c:3218)
  File "parser.pyx", line 560, in pandas.parser.TextReader._setup_parser_source
(pandas\parser.c:5608)
IOError: Initializing from file failed

可能有什么不对?最令人费解的部分是它在某些时候有效,但不是每次都有效。可能是因为此进程读取文件与关闭文件的其他进程之间存在竞争条件吗?

该文件包含一些统计信息,如:

2014.09.02,00:15,111.7,63,159,134.11261, and so on
2014.09.02,00:16,126.1,08,1235,154.11353,
2014.09.02,00:17,123.5,65,100,153.11313,
2014.09.02,01:18,114.7,59,101,152.11334,
2014.09.02,01:19,111.3,42,1229,153.11283,

0 个答案:

没有答案