为SSL套接字上的每个请求创建新进程会产生“TypeError:无法序列化套接字对象”,但对普通/非SSL套接字执行相同操作

时间:2018-06-18 13:25:16

标签: java python sockets ssl

我正在尝试在python服务器和java客户端中使用使用java keytool生成的密钥和证书。我创造了关键&密钥库,导出证书,将证书添加到信任库,将密钥库转换为标准pkcs格式,然后从pkcs中提取密钥和证书以在python服务器中使用。 (here的前三个步骤和here的最后三个步骤)。这个问题稍微跟进了this question,可以在that question中找到生成密钥和证书的详细步骤。

我的SSL服务器看起来像这样

server.py

import socket
import multiprocessing as mup
import ssl

def worker_ssl(data_socket, client_address):
    print("Inside worker")
    #some processing

def start_server_ssl():

    socketObj = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server_address = ("127.0.0.1", 6000)
    socketObj.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    socketObj.bind(server_address)
    socketObj.listen(10)

    ssl_socket = ssl.wrap_socket(socketObj,
                            server_side = True,
                            certfile="cert.pem",
                            keyfile="key.pem")        

    while True:
        print("Waiting For Connections .........\n")
        try:
            data_socket, client_address = ssl_socket.accept()

            process = mup.Process(target=worker_ssl, args=(data_socket, client_address))            
            process.daemon = True
            process.start()

        except socket.error as msg:
            print (" [ERROR] : %s " % msg)
            continue

    socketObj.close()
    ssl_socket.shutdown(socket.SHUT_RDWR)
    ssl_socket.close()

if __name__ == '__main__':
    start_server_ssl()

SSL客户端如下所示:

Client4Py.java

import java.io.BufferedWriter;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.nio.charset.StandardCharsets;
import java.security.KeyStore;

import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.SSLSocketFactory;
import javax.net.ssl.TrustManager;
import javax.net.ssl.TrustManagerFactory;

public class Client4Py {
    static KeyStore ks;
    static KeyManagerFactory kmf;
    static TrustManagerFactory tmf;
    static SSLContext sc;
    static TrustManager[] trustManagers;

    static {
        try {
            ks = KeyStore.getInstance("JKS");
            ks.load(new FileInputStream("D:\\javasslstores\\truststore.jks"), "passwd123".toCharArray());

            kmf = KeyManagerFactory.getInstance("SunX509");
            kmf.init(ks, "passwd123".toCharArray());

            tmf = TrustManagerFactory.getInstance("SunX509"); 
            tmf.init(ks);

            sc = SSLContext.getInstance("TLS"); 

            sc.init(null, tmf.getTrustManagers(), null);
        } catch (Exception e) {
            System.out.println(e.getMessage());
            System.out.println(e.getStackTrace());
        }
    }

    public static void main(String[] args) throws IOException {
        SSLSocketFactory ssf = sc.getSocketFactory();
        SSLSocket socket = (SSLSocket) ssf.createSocket("127.0.0.1", 6000);
        socket.startHandshake();

        PrintWriter out = new PrintWriter(new BufferedWriter(new OutputStreamWriter(socket.getOutputStream(),StandardCharsets.UTF_8)));
        //PrintWriter out = new PrintWriter(new BufferedWriter(new OutputStreamWriter(socket.getOutputStream())));

        out.println("<<startMessage>>");
        out.println("Message from Client4Py");
        out.println("<<endMessage>>");
        out.flush();

        if (out.checkError())
            System.out.println(
                "SSLSocketClient:  java.io.PrintWriter error");

        out.close();
        socket.close();
    }
}

首次运行服务器后服务器控制台上的输出,然后客户端如下:

1    Waiting For Connections .........
2    
3    Traceback (most recent call last):
4      File "D:\workspaces\workspace6\PythonServer\server.py", line 40, in <module>
5        start_server_ssl()
6      File "D:\workspaces\workspace6\PythonServer\server.py", line 29, in start_server_ssl
7        process.start()
8      File "D:\Programs\python\python-3.6.6-amd64\lib\multiprocessing\process.py", line 105, in start
9        self._popen = self._Popen(self)
10      File "D:\Programs\python\python-3.6.6-amd64\lib\multiprocessing\context.py", line 223, in _Popen
11        return _default_context.get_context().Process._Popen(process_obj)
12      File "D:\Programs\python\python-3.6.6-amd64\lib\multiprocessing\context.py", line 322, in _Popen
13        return Popen(process_obj)
14      File "D:\Programs\python\python-3.6.6-amd64\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
15        reduction.dump(process_obj, to_child)
16      File "D:\Programs\python\python-3.6.6-amd64\lib\multiprocessing\reduction.py", line 60, in dump
17        ForkingPickler(file, protocol).dump(obj)
18      File "D:\Programs\python\python-3.6.6-amd64\lib\socket.py", line 185, in __getstate__
19        raise TypeError("Cannot serialize socket object")
20    TypeError: Cannot serialize socket object
21    Traceback (most recent call last):
22      File "<string>", line 1, in <module>
23      File "D:\Programs\python\python-3.6.6-amd64\lib\multiprocessing\spawn.py", line 99, in spawn_main
24        new_handle = reduction.steal_handle(parent_pid, pipe_handle)
25      File "D:\Programs\python\python-3.6.6-amd64\lib\multiprocessing\reduction.py", line 87, in steal_handle
26        _winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
27    PermissionError: [WinError 5] Access is denied

您可以在第20行看到TypeError: Cannot serialize socket object

删除所有SSL内容后,代码开始工作

当我评论对wrap_socket()的来电时,将ssl_socket.accept()替换为socketObject.accept()并发表评论ssl_socket.shutdown()close(),代码就会开始运作。它根据需要从worker()创建新进程,并在控制台上结束打印Inside worker。以下是修改后的非SSL服务器:

import socket
import multiprocessing as mup
# import ssl

def worker_ssl(data_socket, client_address):
    print("Inside worker")
    #some processing

def start_server_ssl():

    socketObj = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server_address = ("127.0.0.1", 6000)
    socketObj.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    socketObj.bind(server_address)
    socketObj.listen(10)

#     ssl_socket = ssl.wrap_socket(socketObj,
#                             server_side = True,
#                             certfile="cert.pem",
#                             keyfile="key.pem")        

    while True:
        print("Waiting For Connections .........\n")
        try:
            data_socket, client_address = socketObj.accept()

            process = mup.Process(target=worker_ssl, args=(data_socket, client_address))            
            process.daemon = True
            process.start()

        except socket.error as msg:
            print (" [ERROR] : %s " % msg)
            continue

    socketObj.close()
#     ssl_socket.shutdown(socket.SHUT_RDWR)
#     ssl_socket.close()

if __name__ == '__main__':
    start_server_ssl()

2 个答案:

答案 0 :(得分:1)

我通过在子进程内使用SSLContext明确封装套接字来解决此问题。创建SSLContext多处理所需的一切都没有序列化的问题。

更新:以下是服务器sudo代码,它说明了上面的解释:

def client_connection_handler(client_socket, client_address, ssl_settings):
    context = ssl.SSLContext()
    context.load_cert_chain(certfile=ssl_settings['certfile'], keyfile=ssl_settings['keyfile'],)
    context.verify_mode = ssl_settings["verify_mode"]
    context.load_verify_locations(cafile=ssl_settings["cafile"])
    client_secured_socket = context.wrap_socket(client_socket, server_side=True)
    #
    # send and receive data
    #
    client_secured_socket.close()


def server():
    server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server_socket.bind((hostname, port))
    server_socket.listen(5)
    while True:
        # Waiting for client connection
        client_socket, client_address = server_socket.accept()
        concurrent_connections = len(multiprocessing.active_children())
        if max_concurrent_clients > concurrent_connections:
            p = multiprocessing.Process(target=client_connection_handler, args=(client_socket, client_address, ssl_settings))
            p.daemon = True
            p.start()
            continue
        # Max client connection has been reached
        client_socket.close()

答案 1 :(得分:0)

我不知道我是否理解确切的原因,但是我在这里说明。

查看以下堆栈跟踪行:

17        ForkingPickler(file, protocol).dump(obj)
18      File "D:\Programs\python\python-3.6.6-amd64\lib\socket.py", line 185, in __getstate__
19        raise TypeError("Cannot serialize socket object")
20    TypeError: Cannot serialize socket object

似乎multiprocessing模块的Process.start()方法将传递给它的参数序列化,以将它们传递给新进程。而且似乎SSLSocket对象无法序列化。但是,似乎Socket对象可以序列化。 This issue声明相同,这就是基于Socket的服务器正常工作的原因。但是,我不明白为什么会这样(SSLSocket无法序列化,但是Socket对象可以序列化)。我的意思是,有Socket个对象实现的任何方法,而SSLSocket没有实现。另请注意,错误发生在Socket类的socket.py, line 185, in __getstate__,而不是SSLSocket类的错误。

如果有人确认了上述原因(即SSLSocket对象不可序列化,而Socket对象可序列化),我想解释为什么会出现这种情况并提供解决方案SSLSocketmultiprocessing模块一起使用。

解决方法

我最终使用os.fork分叉了新流程。 (可以在here中找到os.fork的示例)。它仅在Linux上受支持。因此,我installed python on cygwin然后使用它。