Question

我喜欢并行处理两个功能，一个用于图像批处理（将所有25张图像流式处理），另一个用于处理批处理图像。它们需要并行。

因此，我具有批处理图像BatchStreaming(self)和处理BatchProcessing(self, b_num)的主要功能。现在BatchStreaming运行良好。流式传输25张图像后，需要继续进行批处理。我有两个并行的过程。他们是

（1）While loop in BatchStreaming需要继续处理另一批图像。

（2）同时，需要处理当前批处理的图像。

我对应该使用进程还是线程感到困惑。我喜欢使用进程，因为我喜欢利用CPU中的所有内核。（Python的线程仅在一个CPU内核上运行）

那我有两个问题（1）程序必须重新加入主程序才能继续。但是我需要继续处理下一批图像。

（2）在下面的程序中，当调用BatchProcessing(self, b_num)并具有作为例外的情况时

Caught Main Exception
(<class 'TypeError'>, TypeError("'module' object is not callable",), <traceback object at 0x7f98635dcfc8>)

可能是什么问题？

代码如下。

import multiprocessing as MultiProcess
import time
import vid_streamv3 as vs
import cv2
import sys
import numpy as np
import os
BATCHSIZE=25
CHANNEL=3
HEIGHT=480
WIDTH=640
ORGHEIGHT=1080
ORGWIDTH=1920

class ProcessPipeline:
    def __init__(self):

        #Current Cam
        self.camProcess = None
        self.cam_queue = MultiProcess.Queue(maxsize=100)
        self.stopbit = None
        self.camlink = 'rtsp://root:pass@192.168.0.90/axis-media/media.amp?camera=1' #Add your RTSP cam link
        self.framerate = 25
        self.fullsize_batch1=np.zeros((BATCHSIZE, ORGHEIGHT, ORGWIDTH, CHANNEL), dtype=np.uint8)
        self.fullsize_batch2=np.zeros((BATCHSIZE, ORGHEIGHT, ORGWIDTH, CHANNEL), dtype=np.uint8)      
        self.batch1_is_processed=False

    def BatchStreaming(self):
        #get all cams
        time.sleep(3)
        self.stopbit = MultiProcess.Event()
        self.camProcess = vs.StreamCapture(self.camlink,
                             self.stopbit,
                             self.cam_queue,
                            self.framerate)
        self.camProcess.start()

        count=0
        try:
            while True:

                if not self.cam_queue.empty():
                    cmd, val = self.cam_queue.get()

                    if cmd == vs.StreamCommands.FRAME:
                        if val is not None:
                            print('streaming starts ')
                            if(self.batch1_is_processed == False):
                                self.fullsize_batch1[count]=val
                            else:
                                self.fullsize_batch2[count]=val
                            count=count+1
                            if(count>=25):
                               if(self.batch1_is_processed == False):#to start process for inference and post processing for batch 1
                                  self.batch1_is_processed  = True
                                  print('batch 1 process')
                                  p = MultiProcess(target=self.BatchProcessing, args=(1,))

                               else:#to start process for inference and post processing for batch 2
                                  self.batch1_is_processed  = False
                                  print('batch 2 process')
                                  p = MultiProcess(target=self.BatchProcessing, args=(2,))
                               p.start()
                               print('BatchProcessing start')
                               p.join() 
                               print('BatchProcessing join') 
                               count=0
                            cv2.imshow('Cam: ' + self.camlink, val)
                            cv2.waitKey(1)

        except KeyboardInterrupt:
            print('Caught Keyboard interrupt')

        except:
            e = sys.exc_info()
            print('Caught Main Exception')
            print(e)

        self.StopStreaming()
        cv2.destroyAllWindows()

    def StopStreaming(self):
        print('in stopCamStream')
        if self.stopbit is not None:
            self.stopbit.set()
            while not self.cam_queue.empty():
                try:
                    _ = self.cam_queue.get()
                except:
                    break
                self.cam_queue.close()
            print("before camProcess.join()")
            self.camProcess.join()
            print("after camProcess.join()")

    def BatchProcessing(self, b_num):
        print('module name:', __name__)
        if hasattr(os, 'getppid'):  # only available on Unix
            print('parent process:', os.getppid())
        print('process id:', os.getpid())


if __name__ == "__main__":
    mc = ProcessPipeline()
    mc.BatchStreaming()

Answer 1

我使用了事件信令，如下所示。对于我的应用程序来说，这更简单。

批处理循环中有足够的图像时，发出批处理信号。

export class MyComponent implements AfterViewInit {
  private visibilityChange$ = fromEvent(document, 'visibilitychange').pipe(startWith('visible'), shareReplay({ refCount: true, bufferSize: 1 }));
  private show$ = this.visibilityChange$.pipe(filter(() => document.visibilityState === 'visible'));
  private hide$ = this.visibilityChange$.pipe(filter(() => document.visibilityState === 'hidden'));

  public ngAfterViewInit() {
    const lettersStream$ = zip( // add delay for each letter
            from(['w', 'o', 'r', 'd']),
            interval(1000))
           // pause when hide$ fires, resume when show$
          .pipe(map(([letter, delayTime]) => letter))
          .subscribe(console.log);
  }
}

Python中的并行处理以进行图像批处理

1 个答案: