Question

我花了很多时间研究如何使用多处理软件包，但在QGIS的插件中找不到任何关于如何使用它的内容。我正在开发一个插件，可以对几个元素进行一些优化。我想将它并行化。

我在python插件（http://snorf.net/blog/2013/12/07/multithreading-in-qgis-python-plugins/）中找到了一个关于多线程的有用链接，但是没有使用多处理模块，这可能更容易？

我一直在尝试一个非常基本的例子。我只在这里显示插件的run函数：

def run(self):
    """Run method that performs all the real work"""
    # show the dialog
    self.dlg.show()

    # Run the dialog event loop
    result = self.dlg.exec_()

    # See if OK was pressed and run code
    if result:
        #Get number of cores
        nProcs = mp.cpu_count()

        #Start a Process
        p = mp.Pool(nProcs)

        #Define function     
        def cube(x):
            return x**3

        #Run parallel        
        results = p.map(cube, range(1,7))

当我从QGIS中的插件运行此代码时，它会打开几个QGIS窗口，然后返回错误（无法加载图层等）。我错过了什么？我是否需要先在另一个线程上启动一个worker，然后在那里使用多处理？或者我们会使用多处理中的另一个函数吗？

如果问题需要修改，请告诉我。我使用QGIS 2.10在Windows 7下工作。

谢谢，

更新我创建了一个工作类来实现该函数并将其发送到一个新线程，但是当我在该线程中使用多处理时，我遇到了同样的问题。

我创建的课程如下：

class Worker(QObject):
    '''Example worker'''
    def __init__(self, result_queue, f, attr=[], repet=None, nbCores=None):
        QObject.__init__(self)
        if not hasattr(f, '__call__'):
            #Check if not a function
            raise TypeError('Worker expected a function as second argument')
        if not isinstance(attr, list) and not repet==None:
            #Check if not a list if there is a repet command
            raise TypeError('Input problem:\nThe arguments for the function should be in a list if repet is provided')
        if not all(isinstance(elem, list) for elem in attr) and repet==None and len(inspect.getargspec(f).args) > 1:
            #Check if not a list of lists if there isn't a repet command
            raise TypeError('Input problem:\nThe arguments for the function should be a list of lists if repet is not provided')
        if not repet == None and (not isinstance(repet, int) or repet == 0):
            #Check that provided an integer greater than 0
            raise TypeError('If provided, repet should be None or a strictly positive integer')
        self.result_queue = result_queue
        self.f = f
        self.attr = attr
        self.repet = repet
        self.nbCores = nbCores

        if self.nbCores == None:
            self.nbCores = mp.cpu_count() - 1

    def fStar(self, arg):
        """Convert the function to taking a list as arguments"""
        return self.f(*arg)

    def run(self):
        ret = None
        try:
            if self.repet == 1:
                # estimates the function based on provided arguments
                ret = self.f(*self.attr) #The star unpacks the list into attributes
            else:
                pool = mp.Pool(processes=self.nbCores)

                if self.repet > 1:
                    ret = pool.map(self.fStar, itools.repeat(self.attr,self.repet))
                elif self.repet == None:
                    ret = pool.map(self.fStar, self.attr)

                pool.close()
                pool.join()
        except Exception, e:
            #I can't pass an exception, it makes qgis bug
            pass

        self.result_queue.put(ret) #Pass the result to the queue

    finished = pyqtSignal(object)
    error = pyqtSignal(Exception, basestring)

我启动worker并使用以下函数将其发送到新线程：

def startWorker(f, attr, repet=None, nbCores=None):
    #Create a result queue
    result_queue = queue.Queue()

    # create a new worker instance
    worker = Worker(result_queue, f, attr, repet, nbCores)

    # start the worker in a new thread
    thread = QThread()
    worker.moveToThread(thread)
    thread.started.connect(worker.run)
    thread.start()
    #Clean up when the thread is finished
    worker.deleteLater()
    thread.quit()
    thread.wait()
    thread.deleteLater()

    #Export the result to the queue
    res = []
    while not result_queue.empty():
        r = result_queue.get()
        if r is None:
            continue
        res.append(r)

    return res

在我最初的问题中，我刚刚通过调用results = p.map(cube, range(1,7))函数

替换了startWorker

如果您有任何想法如何使这项工作，请告诉我。我在多个线程中实现了这项工作，但使用多个内核会更快......

在qgis插件代码中使用python多处理包

0 个答案: