Question

我有一些python代码，其中没有触发APScheduler作业。作为上下文，我还有一个处理程序，它使用eventlet / GreenPool来查找文件修改目录，以进行多线程处理。根据一些故障排除，似乎APScheduler和eventlet之间存在某种冲突。

我的输出如下：

2016-12-26 02:30:30 UTC（+0000）：完成下载通行证   2016-12-26 02:46:07 UTC（+0000）：由于控制-C或其他退出信号而退出   Jobstore默认值：
      时间激活下载（触发：间隔[0:05:00]，下次运行时间：2016-12-25 18:35:00太平洋标准时间）   2016-12-26 02:46:07 UTC（+0000）：1

（太平洋标准时间18:35 = UTC时间02:35）...所以它应该在我按下control-C之前11分钟被解雇

from apscheduler import events ## pip install apscheduler
from apscheduler.schedulers.background import BackgroundScheduler

# Threading
from eventlet import patcher, GreenPool ## pip install eventlet
patcher.monkey_patch(all = True)

def setSchedule(scheduler, cfg, minutes = 60*2, hours = 0):
  """Set up the schedule of how frequently a download should be attempted.
  scheduler object must already be declared.
  will accept either minutes or hours for the period between downloads"""
  if hours > 0:
    minutes = 60*hours if minutes == 60 else 60*hours+minutes
  handle = scheduler.add_job(processAllQueues,
                            trigger='interval',
                            kwargs={'cfg': cfg},
                            id='RQmain',
                            name='Time-Activated Download',
                            coalesce=True,
                            max_instances=1,
                            minutes=minutes,
                            start_date=dt.datetime.strptime('2016-10-10 00:15:00', '%Y-%m-%d %H:%M:%S') # computer's local time
  )
  return handle

def processAllQueues(cfg):
  SQSpool = GreenPool(size=int(cfg.get('GLOBAL','Max_AWS_Connections')))
  FHpool = GreenPool(size=int(cfg.get('GLOBAL','Max_Raw_File_Process')))
  arSects = []
  dGlobal = dict(cfg.items('GLOBAL'))
  for sect in filter(lambda x: iz.notEqualz(x,'GLOBAL','RUNTIME'),cfg.sections()):
    dSect = dict(cfg.items(sect)) # changes all key names to lowercase
    n = dSect['sqs_queue_name']
    nn = dSect['node_name']
    fnbase = "{}_{}".format(nn,n)
    dSect["no_ext_file_name"] = os.path.normpath(os.path.join(cfg.get('RUNTIME','Data_Directory'),fnbase))
    arSects.append(mergeTwoDicts(dGlobal,dSect)) # section overrides global
  arRes = []
  for (que_data,spec_section) in SQSpool.imap(doQueueDownload,arSects): 
    if que_data: fileResult = FHpool.spawn(outputQueueToFiles,spec_section,que_data).wait()
    else: fileResult = (False,spec_section['sqs_queue_name'])
    arRes.append(fileResult)
  SQSpool.waitall()
  FHpool.waitall()
  pr.ts_print("Finished Download Pass")
  return None

def main():
  cfgglob = readConfigs(cfgdir, datdir)
  sched = BackgroundScheduler()
  cron_job = setSchedule(sched, cfgglob, 5)  
  sched.start(paused=True)
  try:
    change_handle = win32file.FindFirstChangeNotification(cfgdir, 0, win32con.FILE_NOTIFY_CHANGE_FILE_NAME | win32con.FILE_NOTIFY_CHANGE_LAST_WRITE)
    processAllQueues(cfgglob)
    sched.resume() # turn the scheduler back on and monitor both wallclock and config directory.
    cron_job.resume()
    while 1:
      SkipDownload = False
      result = win32event.WaitForSingleObject(change_handle, 500)
      if result == win32con.WAIT_OBJECT_0: # If the WaitForSO returned because of a notification rather than error/timing out
        sched.pause() # make sure we don't run the job as a result of timestamp AND file modification
        while 1:
          try:
            win32file.FindNextChangeNotification(change_handle) # rearm - done at start because of the loop structure here
            cfgglob = None
            cfgglob = readConfigs(cfgdir,datdir)
            cron_job.modify(kwargs={'cfg': cfgglob}) # job_id="RQmain",
            change_handle = win32file.FindFirstChangeNotification(cfgdir, 0, win32con.FILE_NOTIFY_CHANGE_FILE_NAME | win32con.FILE_NOTIFY_CHANGE_LAST_WRITE) # refresh handle
            if not SkipDownload: processAllQueues(cfgglob)
            sched.resume()
            cron_job.resume()
            break
  except KeyboardInterrupt:
    if VERBOSE | DEBUG: pr.ts_print("EXITING due to control-C or other exit signal")
  finally:
    sched.print_jobs()
    pr.ts_print(sched.state)
    sched.shutdown(wait=False)

如果我注释掉大部分processAllQueues函数以及eventlet包含在top，它会适当地触发。如果我保留

from eventlet import patcher, GreenPool ## pip install eventlet
patcher.monkey_patch(all = True)

但注释掉processAllQueues到倒数第二行的打印行，它无法触发APScheduler，表明导入修补程序和GreenPool或者使用monkey_patch语句时出现问题。评论patcher.monkey_patch(all = True)使其再次“有效”。

有谁知道在我的情况下哪个候选的monkey_patch语句会起作用？

Answer 1

您有一个显式的事件循环来监视文件更改。这阻止了eventlet事件循环的运行。您有两种选择：

在win32event.WaitForSingleObject()

eventlet.tpool.execute()

在阻止来电之前/之后运行eventlet.sleep()并确保您不会阻止太久。

eventlet.monkey_patch(thread=False)是将每个其他模块列为true的更短替代方法。通常，在使用锁或线程局部存储或线程API来生成绿色线程时，您需要thread=True。如果您真正使用操作系统线程，则可能需要thread=False，例如有趣的GUI框架。

您不应该在Windows上真正考虑使用Eventlet来运行重要项目。性能远低于POSIX。我从0.17开始就没有在Windows上运行测试。这相当于在流行的桌面平台上轻松开发。

由于eventlet monkey_patch，APScheduler任务未触发

1 个答案: