通过python套接字传输不完整的数据

时间:2015-07-30 21:01:50

标签: python sockets tcp

我遇到了使用Python编写的远程客户端和服务器通过TCP传输数据的问题。服务器位于相当慢的因特网连接(<2Mb /秒)的相当慢的区域中。当客户端在LAN上与服务器一起运行时,传输完整的字符串(2350字节);但是,当我在局域网外运行客户端时,有时字符串被截断(1485字节),有时完整字符串通过(2350字节)。截断字符串的大小似乎总是1485字节。字符串的完整大小远低于客户端和服务器的设置缓冲区大小。

我在下面复制了客户端和服务器代码的缩写版本,我试图编辑所有无关的细节:

客户端

import socket
from   time   import sleep

class FTIRdataClient():

    def __init__(self,TCP_IP="xxx.xxx.xxx.xxx",TCP_Port=xxx,BufferSize=4096):

        #-----------------------------------
        # Configuration parameters of server
        #-----------------------------------
        self.TCP_IP          = TCP_IP
        self.TCP_Port        = int(TCP_Port)
        self.RECV_BUFFER     = int(BufferSize) 

    def writeTCP(self,message):

        try:
            sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)    
            sock.connect((self.TCP_IP,self.TCP_Port))
            sock.send(message)
            incomming = sock.recv(self.RECV_BUFFER)
            sock.close()    
        except:
            print "Unable to connect to data server!!"           
            incomming = False   

        return incomming         

if __name__ == "__main__":

    #----------------------------------
    # Initiate remote data client class
    #----------------------------------
    dataClass = FTIRdataClient(TCP_IP=dataServer_IP,TCP_Port=portNum,BufferSize=4096)

    #--------------------------------
    # Ask database for all parameters
    #--------------------------------
    allParms = dataClass.writeTCP("LISTALL")

服务器

import os
import sys
import socket
import select
import smtplib
import datetime       as     dt


class FTIRdataServer(object):

    def __init__(self,ctlFvars):
        ...

    def runServer(self):

        self.server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.server_socket.bind((self.TCP_IP,self.TCP_Port))
        #self.server_socket.setsockopt(socket.IPPROTO_TCP,socket.TCP_NODELAY,1)
        self.server_socket.listen(10)         
        self.connection_list.append(self.server_socket)

        #-------------------------------------
        # Start loop to listen for connections
        #-------------------------------------
        while True:     

            #--------------------
            # Get list of sockets
            #--------------------
            read_sockets,write_sockets,error_sockets = select.select(self.connection_list,[],[],5)

            for sock in read_sockets:

                #-----------------------
                # Handle new connections
                #-----------------------
                if sock == self.server_socket:

                    #----------------------------------------------
                    # New connection recieved through server_socket
                    #----------------------------------------------
                    sockfd, addr = self.server_socket.accept()

                    self.connection_list.append(sockfd)
                    print "Client (%s, %s) connected" % addr

                #-------------------------------------
                # Handle incomming request from client
                #-------------------------------------
                else:

                    #------------------------
                    # Handle data from client
                    #------------------------
                    try:
                        data = sock.recv(self.RECV_BUFFER)

                        #------------------------------------------------
                        # Three types of call to server:
                        #  1) set   -- sets the value of a data parameter
                        #  2) get   -- gets the value of a data parameter
                        #  3) write -- write data to a file
                        #------------------------------------------------
                        splitVals = data.strip().split()

                        ...

                        elif splitVals[0].upper() == 'LISTALL':

                            msgLst = []

                            #----------------------------
                            # Create a string of all keys 
                            # and values to send back
                            #----------------------------
                            for k in self.dataParams:
                                msgLst.append("{0:}={1:}".format(k," ".join(self.dataParams[k])))

                            msg = ";".join(msgLst)

                            sock.sendall(msg)

                        ...

                        else:
                            pass

                    #---------------------------------------------------
                    # Remove client from socket list after disconnection
                    #---------------------------------------------------
                    except:
                        sock.close()
                        self.connection_list.remove(sock)
                        continue

        #-------------
        # Close server
        #-------------
        self.closeServer()


    def closeServer(self):
        ''' Close the TCP data server '''
        self.server_socket.close()

非常感谢您的帮助!!!

1 个答案:

答案 0 :(得分:0)

For anyone who is interested I found the solution to this problem. John Nielsen has a pretty good explanation here. Basically, TCP stream only guarantees that bytes will not arrive out of order or be duplicated; however, it does not guarantee how many groups the data will be sent in. So one needs to continually read (socket.recv) until all the data is sent. The previous code work on the LAN because the server was sending the entire string in one group. Over a remote connection the string was split into several groups.

I modified the client to continually loop on socket.recv() until the socket is closed and I modified the server to immediately close the socket after sending the data. There are several other ways to do this mentioned in the above link. The new code looks like:

Client

class FTIRdataClient(object):

    def __init__(self,TCP_IP="xxx.xxx.xx.xxx",TCP_Port=xxxx,BufferSize=4024):

        #-----------------------------------
        # Configuration parameters of server
        #-----------------------------------
        self.TCP_IP          = TCP_IP
        self.TCP_Port        = int(TCP_Port)
        self.RECV_BUFFER     = int(BufferSize) 

    def setParam(self,message):
        try:
            sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)    
            sock.connect((self.TCP_IP,self.TCP_Port))
            sock.sendall("set "+message)

            #-------------------------
            # Loop to recieve all data
            #-------------------------
            incommingTotal = ""
            while True:
                incommingPart = sock.recv(self.RECV_BUFFER)
                if not incommingPart: break
                incommingTotal += incommingPart

            sock.close()    
        except:
            print "Unable to connect to data server!!"           
            incommingTotal = False   

        return incommingTotal

Server class FTIRdataServer(object):

def __init__(self,ctlFvars):

    ...

def runServer(self):

    self.server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    self.server_socket.bind((self.TCP_IP,self.TCP_Port))
    #self.server_socket.setsockopt(socket.IPPROTO_TCP,socket.TCP_NODELAY,1)
    self.server_socket.listen(10)         
    self.connection_list.append(self.server_socket)

    #-------------------------------------
    # Start loop to listen for connections
    #-------------------------------------
    while True:    

        #--------------------
        # Get list of sockets
        #--------------------
        read_sockets,write_sockets,error_sockets = select.select(self.connection_list,[],[],5)

        for sock in read_sockets:

            #-----------------------
            # Handle new connections
            #-----------------------
            if sock == self.server_socket:

                #----------------------------------------------
                # New connection recieved through server_socket
                #----------------------------------------------
                sockfd, addr = self.server_socket.accept()

                self.connection_list.append(sockfd)
                print "Client (%s, %s) connected" % addr

            #-------------------------------------
            # Handle incomming request from client
            #-------------------------------------
            else:

                #------------------------
                # Handle data from client
                #------------------------
                try:
                    data = sock.recv(self.RECV_BUFFER)

                    ...                   

                    elif splitVals[0].upper() == 'LISTALL':

                        msgLst = []

                        #----------------------------
                        # Create a string of all keys 
                        # and values to send back
                        #----------------------------
                        for k in self.dataParams:
                            msgLst.append("{0:}={1:}".format(k," ".join(self.dataParams[k])))

                        msg = ";".join(msgLst)

                        sock.sendall(msg)

                    elif splitVals[0].upper() == 'LISTALLTS':   # List all time stamps

                        msgLst = []

                        #----------------------------
                        # Create a string of all keys 
                        # and values to send back
                        #----------------------------
                        for k in self.dataParamTS:
                            msgLst.append("{0:}={1:}".format(k,self.dataParamTS[k]))

                        msg = ";".join(msgLst)

                        sock.sendall(msg)                        

                    ...
                    else:
                        pass

                    #------------------------
                    # Close socket connection
                    #------------------------
                    sock.close()
                    self.connection_list.remove(sock)   

                #------------------------------------------------------
                # Remove client from socket list if client discconnects
                #------------------------------------------------------
                except:
                    sock.close()
                    self.connection_list.remove(sock)
                    continue

    #-------------
    # Close server
    #-------------
    self.closeServer()

Whatever. This is probably common knowledge and I'm just a little slow.