Question

我很少将问题发布到论坛，但这个问题令我难过。我很好奇是什么导致了这个问题（解决方案也很好，但大多数情况下，我想知道为什么我遇到这个问题）：

我最近编写了一个python脚本，用于包装调用由PBS作业启动的远程命令：

#! /usr/bin/env python
#
# Copyright (c) 2009 Maciej Brodowicz
# Copyright (c) 2011 Bryce Lelbach
#
# Distributed under the Boost Software License, Version 1.0. (See accompanying
# file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

from datetime import datetime

from string import letters, digits

from types import StringType

from optparse import OptionParser

from threading import Thread

# subprocess instantiation wrapper. Unfortunately older Python still lurks on
# some machines.
try:
  from subprocess import Popen, STDOUT, PIPE
  from types import StringType

  class process:
    _proc = None
    _exec = None

    def __init__(self, cmd):
      self._proc = Popen(cmd, stderr = STDOUT, stdout = PIPE,
        shell = (False, True)[type(cmd) == StringType])

    def poll(self):
      return self._proc.poll()

    def pid(self):
      return self._proc.pid

    def _call(self):
      # annoyingly, KeyboardInterrupts are transported to threads, while most
      # other Exceptions aren't in python
      try:
        self._proc.wait()
      except Exception, err:
        self._exec = err

    def wait(self, timeout=None):
      if timeout is not None:
        thread = Thread(target=self._call)
        thread.start()

        # wait for the thread and invoked process to finish
        thread.join(timeout)

        # be forceful
        if thread.is_alive():
          self._proc.terminate()
          thread.join()

          # if an exception happened, re-raise it here in the master thread 
          if self._exec is not None:
            raise self._exec

          return (True, self._proc.returncode)

        if self._exec is not None:
          raise self._exec

        return (False, self._proc.returncode)

      else:
        return (False, self._proc.wait())

    def read(self):
      return self._proc.stdout.read()

except ImportError, err:
  # no "subprocess"; use older popen module
  from popen2 import Popen4
  from signal import SIGKILL
  from os import kill, waitpid, WNOHANG

  class process:
    _proc = None

    def __init__(self, cmd):
      self._proc = Popen4(cmd)

    def poll(self):
      return self._proc.poll()

    def pid(self):
      return self._proc.pid

    def _call(self):
      # annoyingly, KeyboardInterrupts are transported to threads, while most
      # other Exceptions aren't in python
      try:
        self._proc.wait()
      except Exception, err:
        self._exec = err

    def wait(self, timeout=None):
      if timeout is not None:
        thread = Thread(target=self._call)
        thread.start()

        # wait for the thread and invoked process to finish
        thread.join(timeout)

        # be forceful
        if thread.is_alive():
          kill(self._proc.pid, SIGKILL)
          waitpid(-1, WNOHANG)
          thread.join()

          # if an exception happened, re-raise it here in the master thread 
          if self._exec is not None:
            raise self._exec

          return (True, self._proc.wait())

        if self._exec is not None:
          raise self._exec

        return (False, self._proc.wait())

      else:
        return (False, self._proc.wait())

    def read(self):
      return self._proc.fromchild.read()

def run(cmd, timeout=3600):
  start = datetime.now() 
  proc = process(cmd)
  (timed_out, returncode) = proc.wait(timeout)
  now = datetime.now()

  output = ''

  while True:
    s = proc.read()

    if s:
      output += s
    else:
      break

  return (returncode, output, timed_out)

def rstrip_last(s, chars):
  if s[-1] in chars:
    return s[:-1]
  else:
    return s

# {{{ main
usage = "usage: %prog [options]" 

parser = OptionParser(usage=usage)

parser.add_option("--timeout",
                  action="store", type="int",
                  dest="timeout", default=3600,
                  help="Program timeout (seconds)")

parser.add_option("--program",
                  action="store", type="string",
                  dest="program",
                  help="Program to invoke") 

(options, cmd) = parser.parse_args()

if None == options.program:
  print "No program specified"
  exit(1)

(returncode, output, timed_out) = run(options.program, options.timeout)

if not 0 == len(output):
  print rstrip_last(output, '\n')

if timed_out:
  print "Program timed out"

exit(returncode)
# }}}

另一个python脚本根据PBS报告的可用资源汇总命令行参数，类似于mpirun。我使用python-paramiko通过SSH启动远程命令。最初我只是直接执行命令，但是当远程运行的一个进程退出信号（例如SIGSEGV）时，我无法收到正确的退出代码。因此，需要上面的脚本。

当我的开发集群在运行时运行这个脚本时，我注意到这个脚本巧妙地无法在我的4核Debian GNU / Linux节点上运行，但它可以在我的48核RHEL / Linux节点上运行：< / p>

在Debian节点上：

wash@hermione0:~/sandbox$ python --version
Python 2.6.7
wash@hermione0:~/sandbox$ uname -a
Linux hermione0 2.6.32-5-amd64 #1 SMP Wed Jan 12 03:40:32 UTC 2011 x86_64 GNU/Linux
wash@hermione0:~/sandbox$ time ./hpx_invoke.py --program='sleep 30' --timeout=5
Program timed out

real 0m30.025s
user 0m0.016s
sys  0m0.012s
wash@hermione0:~/sandbox$

在RHEL节点上：

[22:08:23]:wash@vega:/home/wash/sandbox$ python --version
Python 2.6.6
[22:09:28]:wash@vega:/home/wash/sandbox$ uname -a
Linux vega 2.6.32-131.4.1.el6.x86_64 #1 SMP Fri Jun 10 10:54:26 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
[22:09:30]:wash@vega:/home/wash/sandbox$ time ./hpx_invoke.py --program='sleep 30' --timeout=5
Program timed out

real 0m5.053s
user 0m0.040s
sys  0m0.020s
[22:09:41]:wash@vega:/home/wash/sandbox$

导致这种情况的原因是什么？

P.S。我是这些盒子上的系统管理员。

Answer 1

我猜测可用软件包的不同之处在于导致“子进程实例化包装器”的不同分支在任一计算机上使用。在一个分支中，您将使用SIGTERM（terminate()调用）而在另一个分支中使用SIGKILL。

话虽如此，sleep似乎过早地给出了任何一个信号。可能存在其他差异，但很难说。你最好放入一些调试代码，看看在什么机器上发生了什么。

Answer 2

问题原来是调用子进程作为shell（两台机器都有子进程包）。在RHEL节点上，当/ bin / sh被杀死时，调用的程序也被终止。在Debian节点上，只有/ bin / sh进程被终止，并且它被调用的程序仍然存活。

我通过将脚本更改为不再使用shell = True来修复此问题。

在RHEL和Debian上使用Python脚本的行为不同，几乎相同的python版本

2 个答案: