我正在尝试创建一种在多个进程之间进行通信的方法，以便使用UNIX域套接字传输数据。有两个程序（progserv.py和progclient.py），每个程序都有N个（这个例子有4个）线程，每个程序在另一个进程上与相应的一个进行通信。 progclient.py发送'pickle'd形式的datetime.datetime对象。 progserv.py接收数据并确定通信延迟。这里的想法是找到进程间数据传输的通信延迟。
我面临的问题是随着N值的增加，延迟从小于1毫秒增加到有时大于250毫秒。延迟时间不一致，它一直波动很大。
progserv.py代码：
import string
import random
import threading
import time
import os
import traceback
import socket
import pickle
from datetime import datetime
def worker(num):
addr = './tmp'+str(num)
logstr = "SERVER"+str(num)+":"
if os.path.exists(addr):
os.unlink(addr)
try:
msock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM, 0)
msock.bind(addr)
msock.listen(1)
sock,_ = msock.accept()
sock.setblocking(0)
msock.close()
except:
print (logstr,"Error opening the socket for listening.Trace:", traceback.format_exc())
return
print (logstr,"Socket connection with client done")
buf=list()
count=1
while True:
try:
data = sock.recv(1500)
if len(data)<=0:
print (logstr,"Peer end probably closed down the connection")
break
else:
try:
d=(datetime.now()-pickle.loads(data)).total_seconds()*1000
buf.append((count,d))
count+=1
#print (logstr,"Received ",str(count-1),"packets")
except:
pass
except socket.error:
if len(buf)>0:
c,data=buf.pop(0)
print (logstr,"COUNTER=",c," DELTA=",data,"msec")
except:
print (logstr,"Exception in receiving data. Trace:", traceback.format_exc())
break
if isinstance(sock, socket.socket):
sock.close()
if isinstance(msock, socket.socket):
msock.close()
def main():
server_count=5
threads=list()
for i in range(server_count):
t=threading.Thread(target=worker, args=(i,))
threads.append(t)
t.start()
print ("MAIN: Started thread index",i)
time.sleep(0.1)
for t in threads:
t.join()
if __name__=='__main__':
main()
progclient.py代码：
import traceback, os, time, random, socket
import threading, pickle
from datetime import datetime
def worker(num):
addr = './tmp'+str(num)
logstr="CLIENT"+str(num)+":"
try:
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM, 0)
sock.connect(addr)
sock.setblocking(0)
except:
print (logstr,"Error opening the socket for connection. Trace:", traceback.format_exc())
return
while True:
try:
d=datetime.now()
bytes=sock.send(pickle.dumps(d))
if bytes<=0:
print (logstr,"Peer end probably closed down the connection")
break
else:
print (logstr,"Sent",bytes,"bytes")
except socket.error:
pass
except:
print (logstr,"Exception during sending. Trace:", traceback.format_exc())
break
time.sleep(1)
if isinstance(sock, socket.socket):
sock.close()
def main():
client_count=5
threads=list()
for i in range(client_count):
t=threading.Thread(target=worker, args=(i,))
threads.append(t)
t.start()
print ("CLIENT : Started thread index",i)
time.sleep(0.1)
for t in threads:
t.join()
if __name__=='__main__':
main()
每个进程有4个线程的延迟时间：
MAIN: Started thread index 0
MAIN: Started thread index 1
MAIN: Started thread index 2
MAIN: Started thread index 3
MAIN: Started thread index 4
SERVER0: Socket connection with client done
SERVER0: COUNTER= 1 DELTA= 0.7969999999999999 msec
SERVER1: Socket connection with client done
SERVER1: COUNTER= 1 DELTA= 0.28200000000000003 msec
SERVER2: Socket connection with client done
SERVER3: Socket connection with client done
SERVER2: COUNTER= 1 DELTA= 258.839 msec
SERVER4: Socket connection with client done
SERVER3: COUNTER= 1 DELTA= 238.006 msec
SERVER4: COUNTER= 1 DELTA= 477.099 msec
SERVER0: COUNTER= 2 DELTA= 0.175 msec
SERVER2: COUNTER= 2 DELTA= 0.174 msec
SERVER1: COUNTER= 2 DELTA= 108.28 msec
SERVER4: COUNTER= 2 DELTA= 35.966 msec
SERVER3: COUNTER= 2 DELTA= 146.63899999999998 msec
SERVER1: COUNTER= 3 DELTA= 76.92 msec
SERVER2: COUNTER= 3 DELTA= 116.096 msec
SERVER0: COUNTER= 3 DELTA= 317.726 msec
SERVER3: COUNTER= 3 DELTA= 15.792 msec
SERVER4: COUNTER= 3 DELTA= 294.577 msec
SERVER0: COUNTER= 4 DELTA= 6.123 msec
SERVER1: COUNTER= 4 DELTA= 95.69999999999999 msec
SERVER2: COUNTER= 4 DELTA= 0.434 msec
SERVER4: COUNTER= 4 DELTA= 63.32599999999999 msec
SERVER3: COUNTER= 4 DELTA= 322.351 msec
SERVER0: COUNTER= 5 DELTA= 185.03300000000002 msec
SERVER1: COUNTER= 5 DELTA= 84.869 msec
SERVER2: COUNTER= 5 DELTA= 33.532 msec
SERVER3: COUNTER= 5 DELTA= 101.01400000000001 msec
SERVER4: COUNTER= 5 DELTA= 152.035 msec
SERVER0: COUNTER= 6 DELTA= 33.533 msec
SERVER2: COUNTER= 6 DELTA= 0.501 msec
SERVER1: COUNTER= 6 DELTA= 101.59400000000001 msec
SERVER3: COUNTER= 6 DELTA= 91.584 msec
SERVER4: COUNTER= 6 DELTA= 32.779999999999994 msec
SERVER1: COUNTER= 7 DELTA= 42.009 msec
SERVER0: COUNTER= 7 DELTA= 142.716 msec
SERVER2: COUNTER= 7 DELTA= 137.214 msec
SERVER3: COUNTER= 7 DELTA= 124.629 msec
SERVER4: COUNTER= 7 DELTA= 190.025 msec
SERVER0: COUNTER= 8 DELTA= 0.9859999999999999 msec
SERVER1: COUNTER= 8 DELTA= 160.696 msec
SERVER2: COUNTER= 8 DELTA= 60.282000000000004 msec
SERVER4: COUNTER= 8 DELTA= 0.617 msec
SERVER3: COUNTER= 8 DELTA= 169.09 msec
SERVER0: COUNTER= 9 DELTA= 129.679 msec
SERVER1: COUNTER= 9 DELTA= 107.504 msec
SERVER2: COUNTER= 9 DELTA= 8.319999999999999 msec
SERVER3: COUNTER= 9 DELTA= 97.829 msec
SERVER4: COUNTER= 9 DELTA= 262.472 msec
SERVER0: COUNTER= 10 DELTA= 188.364 msec
SERVER2: COUNTER= 10 DELTA= 101.07 msec
SERVER3: COUNTER= 10 DELTA= 116.54100000000001 msec
SERVER1: COUNTER= 10 DELTA= 318.44899999999996 msec
SERVER4: COUNTER= 10 DELTA= 16.676 msec
SERVER0: COUNTER= 11 DELTA= 7.071 msec
SERVER1: COUNTER= 11 DELTA= 60.224 msec
SERVER2: COUNTER= 11 DELTA= 101.07199999999999 msec
SERVER3: COUNTER= 11 DELTA= 45.32 msec
SERVER4: COUNTER= 11 DELTA= 11.719999999999999 msec
SERVER0: COUNTER= 12 DELTA= 125.88199999999999 msec
SERVER1: COUNTER= 12 DELTA= 75.607 msec
SERVER2: COUNTER= 12 DELTA= 84.648 msec
SERVER3: COUNTER= 12 DELTA= 44.051 msec
SERVER4: COUNTER= 12 DELTA= 290.6 msec
SERVER0: COUNTER= 13 DELTA= 0.306 msec
SERVER1: COUNTER= 13 DELTA= 58.865 msec
SERVER2: COUNTER= 13 DELTA= 101.007 msec
SERVER3: COUNTER= 13 DELTA= 12.927 msec
SERVER4: COUNTER= 13 DELTA= 212.348 msec
SERVER0: COUNTER= 14 DELTA= 53.222 msec
SERVER1: COUNTER= 14 DELTA= 63.087 msec
SERVER2: COUNTER= 14 DELTA= 112.285 msec
SERVER4: COUNTER= 14 DELTA= 0.192 msec
SERVER3: COUNTER= 14 DELTA= 111.608 msec
SERVER0: COUNTER= 15 DELTA= 41.897999999999996 msec
SERVER2: COUNTER= 15 DELTA= 0.514 msec
SERVER3: COUNTER= 15 DELTA= 30.408 msec
SERVER1: COUNTER= 15 DELTA= 261.84499999999997 msec
SERVER4: COUNTER= 15 DELTA= 69.785 msec
SERVER0: COUNTER= 16 DELTA= 88.022 msec
SERVER2: COUNTER= 16 DELTA= 19.599999999999998 msec
SERVER1: COUNTER= 16 DELTA= 201.69500000000002 msec
SERVER3: COUNTER= 16 DELTA= 52.805 msec
SERVER4: COUNTER= 16 DELTA= 8.53 msec
SERVER0: COUNTER= 17 DELTA= 0.324 msec
SERVER1: COUNTER= 17 DELTA= 159.975 msec
SERVER2: COUNTER= 17 DELTA= 58.62 msec
SERVER4: COUNTER= 17 DELTA= 0.462 msec
SERVER3: COUNTER= 17 DELTA= 100.717 msec
SERVER0: COUNTER= 18 DELTA= 68.68100000000001 msec
SERVER1: COUNTER= 18 DELTA= 8.701 msec
SERVER2: COUNTER= 18 DELTA= 16.889999999999997 msec
SERVER3: COUNTER= 18 DELTA= 156.637 msec
SERVER4: COUNTER= 18 DELTA= 56.296 msec
SERVER0: COUNTER= 19 DELTA= 29.073 msec
SERVER2: COUNTER= 19 DELTA= 30.326999999999998 msec
SERVER3: COUNTER= 19 DELTA= 0.155 msec
SERVER1: COUNTER= 19 DELTA= 251.63200000000003 msec
SERVER4: COUNTER= 19 DELTA= 5.305 msec
当仅使用1个线程尝试相同的传输时，我看到大约1毫秒的延迟。它随着每个进程使用的线程数而不规律地增加。
该平台是基于Jessie的Raspberry Pi。有人可以解释一下这种行为吗？
答案 0 :(得分：0)
这是由Python's Global Interpreter Lock (GIL)引起的，只能限制并发处理速度。来源：https://jeffknupp.com/blog/2013/06/30/pythons-hardest-problem-revisited/
为了更快地完成任务或完成任务，你必须要么创建另一个进程（而不是线程），要么只需要移动到像c ++这样的语言，这样就可以实现并发线程和处理，而不需要＆＃34; lock＆＃34;在线程限制。