ZMQ Recv和recv_string

时间:2018-10-05 14:47:40

标签: python-3.x pandas time zeromq pyzmq

我在下面使用的这段代码是发布商的ZMQ子目录,向我提供了数据。它使用计数器告诉我何时每隔30秒和59秒左右将我的CSV文件写入CSV文件。

问题:我现在正在计时线程中的所有进程。其中message和message2 = socket.recv_string()需要花费半秒到20秒的时间才能接收到字符串。因此导致线程错过了我设置的30和59秒间隔。昨天没有发生这种情况。 if语句的其他计时器耗时.00001或0.0秒。所以那不是问题

我想知道什么会影响到这一点。难道是我计算机的处理能力?还是接收字符串基于等待发布者实际发送内容的时间?

我没有在开发或生产环境中运行,也不在与其他15个人及其虚拟设备共享的服务器上运行。零客户。我之前从未遇到过这个问题,在另一个脚本中,我已经设置了另一个ZMQ发布/订阅,我一直在0.01或.001秒到3秒的时间内接收消息。哪个更易于管理,但标准是.01。

任何提示或帮助都会很棒。预先感谢

import zmq
import pandas as pd
import time
import threading



df_fills = pd.DataFrame()
df_signal = pd.DataFrame()
second_v = [30,59]
s = 0
m = 0
h = 0
d = 0

def counter():
    global h,s,m,d
    while True:
        s += 1
        #print("Second:{}".format(s))
        if s >=60:
            m +=1
            s = 0
        if m >= 60:
            h += 1
            m = 0
        if h >= 24:
            d += 1
            h = 0     
        #print(s)
        time.sleep(1)


class zmq_thread(threading.Thread):
    def __init__(self,name):
        threading.Thread.__init__(self)
        self.name = name
    def run(self):  
        global df_fills, second_v,s 
        print('zmq started')
        context = zmq.Context()
        socket = context.socket(zmq.SUB)              
        socket.connect(SERVER)        
        socket.setsockopt_string(zmq.SUBSCRIBE,'F')            
        print('socket connected')     
        tickers = [a bunch of tickers] 
        while True:                                 
            try:
                start2 = time.time()                               
                if s == 30:
                    print('break')
                    if df_fills.empty == True:
                       print('running fill thread again')
                       z = zmq_thread('Start_ZMQ') 
                       #time.sleep(.7)
                       z.run()  
                    else:
                        start = time.time()
                        print('writing fills')
                        filename = "a CSV"
                        with open(filename, 'a') as f:
                            df_fills.to_csv(f, encoding = 'utf-8', index = False, header = False)
                            f.close()     
                            print('wrote fills')                                          
                            end = time.time()
                            print(end-start)
                            df_fills = df_fills.iloc[0:0]                    
                            z = zmq_thread('Start_ZMQ') 
                            z.run()                    
                    return df_fills     
                end2 = time.time()
                print(end2-start2) 
                start3 = time.time()
                message = socket.recv_string()
                message2 = socket.recv_string()  
                end3 = time.time()
                print(end3-start3, 'message timing')
                print(s)
                start1 = time.time()
                if message == 'F':
                    # message2_split = message2.split("'")
                    message2_split = message2.split(";")
                    message3_split = [e[3:] for e in message2_split]
                    message4 = pd.Series(message3_split)

                    if message4[0] in tickers:
                        df_fills = df_fills.append(message4, ignore_index=True)
                        print('fill')
                end1 = time.time()
                print(end1-start1)
            except KeyboardInterrupt:
                break              



counter = threading.Thread(target = counter)
zmq_loop = zmq_thread('Start_ZMQ')
#%%


counter.start()
zmq_loop.start()

1 个答案:

答案 0 :(得分:0)

我没有意识到ZMQ典型的recv_string默认情况下处于阻塞状态。所以我做到了

               message = socket.recv_string(flags = zmq.NOBLOCK)
               message2 = socket.recv_string(flags = zmq.NOBLOCK)               
           except zmq.ZMQError as e:               
               if e.errno == zmq.EAGAIN:
                    pass             
           else:     
                if message == 'ABA_BB':
                   message2_split = message2.split(";")
                   message3_split = [e[3:] for e in message2_split]
                   message4 = pd.Series(message3_split)
                   #print(message4)
                   if message4[2] == '300':                        
                       df_signal = df_signal.append(message4, ignore_index=True)               
                       print('Signal Appended')