通过套接字发送包含文件的字典（python）

时间：2018-06-02 17:21:39

标签： python file sockets dictionary send

是否可以通过套接字将包含文件（图像或文档）的dict作为值发送？

我试过像贝娄这样的东西，但我失败了.. 

with open("cat.jpeg", "rb") as f:
    myFile = f.read(2048)

data = {"id": "1283", "filename": "cat.jpeg", "file": myFile}

dataToSend = json.dumps(data).encode("utf-8")

这会产生json错误，myFile是一个字节数组无法序列化。

我尝试使用base64编码将myFile转换为字符串，但它没有用。

部分工作是将myFile转换为字符串，如str（myFile）。 json序列化器工作，我通过套接字发送，dict是好的，但myFile数据已损坏，所以我无法重新创建图片。

是否可以使用这种方法或者我应该如何通过套接字发送文件和数据以便在另一方面轻松解析？

LE

仍然没有使用base64编码，myFile仍然是＆＃34; bytes＆＃34;格式和 json给出了这个错误：TypeError：类型＆＃39; bytes＆＃39;的对象不是JSON可序列化的

客户端

import os
import base64
import json
import socket

currentPath = os.path.dirname(os.path.abspath(__file__)) + "\\downloads\\"

with open(currentPath + "cat.png", "rb") as f:
    l = f.read()

print(type(l))   #prints <class 'bytes'>

myFile = base64.b64encode(l)

print(type(myFile))    #prints <class 'bytes'>

data = {"id": "12", "filename": "cat.png", "message": "So cute!", "file": myFile}

dataToSend = json.dumps(data).encode("utf-8")   #prints TypeError: Object of type 'bytes' is not JSON serializable

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 1234))
s.sendall(dataToSend)
s.close()

服务器：

import socket
import json
import os
import sys
import time
import base64

currentPath = os.path.dirname(os.path.abspath(__file__)) + "\\fileCache\\"
tempData = bytearray()

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("127.0.0.1", 1234))
s.listen(5)
conn, addr = s.accept()

while True:
    dataReceived = conn.recv(2048)
    if sys.getsizeof(dataReceived) > 17:
        tempData = tempData + dataReceived
    else:
        data = json.loads(tempData.decode("utf-8"))
        break
    time.sleep(1)

print(data)

myFile = base64.b64decode(data["file"])

with open(currentPath + data["filename"], "wb") as f:
    f.write(myFile)
    f.close()

3 个答案:

答案 0 :(得分：2)

正如我在评论中所说的那样，将二进制数据打包成字符串格式（如JSON）是浪费的 - 如果使用base64，则会使数据传输大小增加33％，这也使得JSON解码器变得困难正确解码JSON，因为它需要流过整个结构才能提取索引。

最好单独发送它们 - JSON作为JSON，然后将文件内容直接作为二进制发送。当然，您需要一种方法来区分这两者，最简单的方法是在发送JSON数据时将其长度作为前缀，以便服务器知道要读取多少字节来获取JSON，然后阅读其余内容作为文件内容。这将使它成为一种非常简单的协议，包形成如下：

[JSON LENGTH][JSON][FILE CONTENTS]

假设JSON永远不会超过4GB（如果是，那么解析它将会有更大的问题，这将是一场噩梦），it fixed {{{{{{it it it it it it it it it it it it it it it it it it it it it it it it it it it it （32位）作为无符号整数（如果你不希望JSON超过64KB，你甚至可以使用16位）所以整个策略在客户端可以工作：

  1. 创建有效负载
  2. 将其编码为JSON，然后使用UTF-8编码将其编码为JSON LENGTH
  3. 获取上述包的长度并将其作为流的前4个字节发送
  4. 发送JSON包
  5. 阅读并发送文件内容

    6. 在服务器端，您执行相同的过程

    1. 读取接收数据的前4个字节以获取JSON有效负载长度
    2. 读取下一个字节数以匹配此长度
    3. 使用UTF-8将它们解码为字符串，然后解码JSON以获取有效负载
    4. 读取其余的流数据并将其存储到文件

      5. 或者在代码中，客户端：

      bytes

      服务器：

      import json
import os
import socket
import struct

BUFFER_SIZE = 4096  # a uniform buffer size to use for our transfers

# pick up an absolute path from the script folder, not necessary tho
file_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "downloads", "cat.png"))

# let's first prepare the payload to send over
payload = {"id": 12, "filename": os.path.basename(file_path), "message": "So cute!"}
# now JSON encode it and then turn it onto a bytes stream by encoding it as UTF-8
json_data = json.dumps(payload).encode("utf-8")
# then connect to the server and send everything
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:  # create a socket
    print("Connecting...")
    s.connect(("127.0.0.1", 1234))  # connect to the server
    # first send the JSON payload length
    print("Sending `{filename}` with a message: {message}.".format(**payload))
    s.sendall(struct.pack(">I", len(json_data)))  # pack as BE 32-bit unsigned int
    # now send the JSON payload itself
    s.sendall(json_data)  # let Python deal with the buffer on its own for the JSON...
    # finally, open the file and 'stream' it to the socket
    with open(file_path, "rb") as f:
        chunk = f.read(BUFFER_SIZE)
        while chunk:
            s.send(chunk)
            chunk = f.read(BUFFER_SIZE)
    # alternatively, if you're using Python 3.5+ you can just use socket.sendfile() instead
    print("Sent.")

      注意：请记住，这是Python 3.x代码 - 对于Python 2.x，您必须自己处理上下文管理，而不是使用import json import os import socket import struct BUFFER_SIZE = 4096 # a uniform buffer size to use for our transfers target_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "fileCache")) with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: s.bind(("127.0.0.1", 1234)) # bind to the 1234 port on localhost s.listen(0) # allow only one connection so we don't have to deal with data separation while True: print("Waiting for a connection...") connection, address = s.accept() # wait for and accept the incoming connection print("Connection from `{}` accepted.".format(address)) # read the starting 32 bits and unpack them into an int to get the JSON length json_length = struct.unpack(">I", connection.recv(4))[0] # now read the JSON data of the given size and JSON decode it json_data = b"" # initiate an empty bytes structure while len(json_data) < json_length: chunk = connection.recv(min(BUFFER_SIZE, json_length - len(json_data))) if not chunk: # no data, possibly broken connection/bad protocol break # just exit for now, you should deal with this case in production json_data += chunk payload = json.loads(json_data.decode("utf-8")) # JSON decode the payload # now read the rest and store it into a file at the target path file_path = os.path.join(target_path, payload["filename"]) with open(file_path, "wb") as f: # open the target file for writing... chunk = connection.recv(BUFFER_SIZE) # and stream the socket data to it... while chunk: f.write(chunk) chunk = connection.recv(BUFFER_SIZE) # finally, lets print out that we received the data print("Received `{filename}` with a message: {message}".format(**payload)) 块来打开/关闭套接字。

      这就是它的全部内容。当然，在实际设置中，您需要处理断开连接，多个客户端等。但这是潜在的过程。

答案 1 :(得分：1)

你应该可以这样做：

data = base64.b64encode(myFile)
dataToSend = json.dumps({"id":"1283","filename":"cat.jpeg", "file":data})

然后通过套接字发送。当您在套接字的另一端收到数据时，只需执行以下操作：

jsonDict = json.loads(dataReceived)
data = base64.b64decode(jsonDict["file"])

更好的方法可能就是使用bson，https://github.com/py-bson/bson。

from gevent import monkey, socket
monkey.patch_all()

import bson
bson.patch_socket()
with open("cat.jpeg", "rb") as f:
    myFile = f.read()
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 12345))
s.sendobj({u"id": "1283", u"filename": "cat.jpeg", u"file": myFile})

答案 2 :(得分：1)

感谢大家的帮助，我终于使用base64完成了它。 我在堆栈溢出时找到了答案，我忘记了它的链接，但它就在这里。

在使用json.dumps之前，我必须像这样对文件进行编码和解码。

base64_bytes = b64encode(l)
myFile = base64_bytes.decode("utf-8")

这是一个有效的例子：

客户端：

import os
from base64 import b64encode
import json
import socket

currentPath = os.path.dirname(os.path.abspath(__file__)) + "\\downloads\\"

with open(currentPath + "cat.png", "rb") as f:
    l = f.read()

base64_bytes = b64encode(l)
myFile = base64_bytes.decode("utf-8")

data = {"id": "12", "filename": "cat.png", "message": "So cute!", "file": myFile}

dataToSend = json.dumps(data).encode("utf-8")

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 1234))
s.sendall(dataToSend)
s.close()

服务器：

import socket
import json
import os
import sys
import base64

currentPath = os.path.dirname(os.path.abspath(__file__)) + "\\fileCache\\"
tempData = bytearray()

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("127.0.0.1", 1234))
s.listen(5)
conn, addr = s.accept()

while True:
    dataReceived = conn.recv(4096)

    if sys.getsizeof(dataReceived) > 17:
        tempData = tempData + dataReceived
    else:
        data = json.loads(tempData.decode("utf-8"))
        break

myFile = base64.b64decode(data["file"])

with open(currentPath + data["filename"], "wb") as f:
    f.write(myFile)
    f.close()
相关问题
最新问题