织物2.3并行执行sudo命令

时间:2018-09-05 14:46:30

标签: python fabric

我正在尝试使用Fabric 2.3在一些服务器上运行一些需要sudo的任务。我的主要目标是并行化操作,因此我想到了使用''类的fabric api,但它不支持blanks(0)

为了清楚起见,下面是我的代码

ThreadingGroup

现在这不能像上面提到的那样工作,因为sudo不支持#!/usr/bin/env python from fabric import ThreadingGroup, Config from getpass import getpass sudo_pass = getpass("Enter your sudo password: ") sudo_config = Config(overrides={'sudo': {'password': sudo_pass}}) server_pool = ThreadingGroup("test1", "test2", config=sudo_config) result = server_pool.sudo('cat /etc/shadow', hide='stderr') print(result) 类支持的所有方法。

通过遍历各个主机,然后为每个主机创建连接,我可以在多台服务器上运行ThreadingGroup,但是效率不高。

那么有没有办法使它与Fabric 2.3并行?我也浏览了官方文档,但没有找到任何东西。

进一步,我根据官方文档对其进行了更多测试,并且看来Connection仅在按以下方式运行时才达到并行性

sudo

但是,如果您像下面那样运行,它不会并行运行

ThreadingGroup

因此,到目前为止,似乎在Fabric 2.3中没有对并行性的灵活支持,我可能只需要切换回Fabric版本1。

1 个答案:

答案 0 :(得分:1)

似乎某些功能未在当前版本(到目前为止为2.4)中实现。 一个可选的解决方案是在其源文件中添加一些代码。 您可以找到fabric的安装路径,然后编辑group.py。

首先,将此功能添加到group.py中:

def thread_worker_sudo(cxn, queue, args, kwargs):
    result = cxn.sudo(*args, **kwargs)
    # TODO: namedtuple or attrs object?
    queue.put((cxn, result))

,然后在ThreadingGroup类中添加sudo函数:

class ThreadingGroup(Group):
    .... original ThreadingGroup


    def sudo(self, *args, **kwargs):
        results = GroupResult()
        queue = Queue()
        threads = []
        for cxn in self:
            my_kwargs = dict(cxn=cxn, queue=queue, args=args, kwargs=kwargs)
            thread = ExceptionHandlingThread(
                target=thread_worker_sudo, kwargs=my_kwargs
            )
            threads.append(thread)
        for thread in threads:
            thread.start()
        for thread in threads:
            # TODO: configurable join timeout
            # TODO: (in sudo's version) configurability around interactive
            # prompting resulting in an exception instead, as in v1
            thread.join()
        # Get non-exception results from queue
        while not queue.empty():
            # TODO: io-sleep? shouldn't matter if all threads are now joined
            cxn, result = queue.get(block=False)
            # TODO: outstanding musings about how exactly aggregate results
            # ought to ideally operate...heterogenous obj like this, multiple
            # objs, ??
            results[cxn] = result
        # Get exceptions from the threads themselves.
        # TODO: in a non-thread setup, this would differ, e.g.:
        # - a queue if using multiprocessing
        # - some other state-passing mechanism if using e.g. coroutines
        # - ???
        excepted = False
        for thread in threads:
            wrapper = thread.exception()
            if wrapper is not None:
                # Outer kwargs is Thread instantiation kwargs, inner is kwargs
                # passed to thread target/body.
                cxn = wrapper.kwargs["kwargs"]["cxn"]
                results[cxn] = wrapper.value
                excepted = True
        if excepted:
            raise GroupException(results)
        return results

我只复制功能 run 的代码并替换此行

thread = ExceptionHandlingThread(
    target=thread_worker, kwargs=my_kwargs
)

thread = ExceptionHandlingThread(
    target=thread_worker_sudo, kwargs=my_kwargs
)

像这样对我有用:

def test_sudo(group):
    group.sudo('whoami', user='root')

$ python fabfile.py
root
root
root
root
root

但是,我不确定,它在所有情况下都能正常工作。