Netty:TCP客户端服务器文件传输:异常TooLongFrameException:

时间:2018-11-05 00:59:45

标签: tcp netty file-transfer

我是netty的新手,我正在尝试设计以下解决方案,以通过TCP将文件从服务器传输到客户端:

1. Zero copy based file transfer in case of non-ssl based transfer (Using default region of the file)
2. ChunkedFile transfer in case of SSL based transfer.

客户端-服务器文件传输以这种方式工作:

1. The client sends the location of the file to be transfered
2. Based on the location (sent by the client) the server transfers the file to the client

文件内容可以是任何内容(字符串/ image / pdf等)和任何大小。

现在,我在服务器端收到了这个 TooLongFrameException:,尽管服务器只是在解码从客户端接收的路径,用于运行下面提到的代码(服务器/客户端)。

io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 65536: 215542494061 - discarded
    at io.netty.handler.codec.LengthFieldBasedFrameDecoder.fail(LengthFieldBasedFrameDecoder.java:522)
    at io.netty.handler.codec.LengthFieldBasedFrameDecoder.failIfNecessary(LengthFieldBasedFrameDecoder.java:500)

现在,我的问题是:

  1. 我是否误解了编码器和解码器的顺序及其配置?如果是这样,将其配置为从服务器接收文件的正确方法是什么?
  2. 我经历了一些相关的StackOverflow帖子SO Q1SO Q2SO Q3SO Q4。我了解了LengthFieldBasedDecoder,但是我不知道如何在服务器(编码端)上配置其相应的LengthFieldPrepender。甚至需要吗?

请为我指明正确的方向。

FileClient:

public final class FileClient {

    static final boolean SSL = System.getProperty("ssl") != null;
    static final int PORT = Integer.parseInt(System.getProperty("port", SSL ? "8992" : "8023"));
    static final String HOST = System.getProperty("host", "127.0.0.1");

    public static void main(String[] args) throws Exception {
        // Configure SSL.
        final SslContext sslCtx;
        if (SSL) {
            SelfSignedCertificate ssc = new SelfSignedCertificate();
            sslCtx = SslContextBuilder.forServer(ssc.certificate(), ssc.privateKey()).build();
        } else {
            sslCtx = null;
        }

        // Configure the client
        EventLoopGroup group = new NioEventLoopGroup();

        try {

            Bootstrap b = new Bootstrap();
            b.group(group)
            .channel(NioSocketChannel.class)
            .option(ChannelOption.SO_KEEPALIVE, true)
            .handler(new ChannelInitializer<SocketChannel>() {
                @Override
                public void initChannel(SocketChannel ch) throws Exception {
                    ChannelPipeline pipeline = ch.pipeline();
                    if (sslCtx != null) {
                        pipeline.addLast(sslCtx.newHandler(ch.alloc(), HOST, PORT));
                    }
                    pipeline.addLast("frameDecoder", new LengthFieldBasedFrameDecoder(64*1024, 0, 8));
                    pipeline.addLast("frameEncoder", new LengthFieldPrepender(4));
                    pipeline.addLast(new ObjectDecoder(ClassResolvers.cacheDisabled(null)));
                    pipeline.addLast(new ObjectEncoder());
                    pipeline.addLast( new FileClientHandler());                }
             });


            // Start the server.
            ChannelFuture f = b.connect(HOST,PORT).sync();

            // Wait until the server socket is closed.
            f.channel().closeFuture().sync();
        } finally {
            // Shut down all event loops to terminate all threads.
            group.shutdownGracefully();
        }
    }
}

FileClientHandler:

public class FileClientHandler extends ChannelInboundHandlerAdapter{

    @Override
    public void channelActive(ChannelHandlerContext ctx) {
        String filePath = "/Users/Home/Documents/Data.pdf";
        ctx.writeAndFlush(Unpooled.wrappedBuffer(filePath.getBytes()));
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
            System.out.println("File Client Handler Read method...");
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
        cause.printStackTrace();

    }
}

FileServer:

/**
 * Server that accept the path of a file and echo back its content.
 */
public final class FileServer {

    static final boolean SSL = System.getProperty("ssl") != null;
    static final int PORT = Integer.parseInt(System.getProperty("port", SSL ? "8992" : "8023"));

    public static void main(String[] args) throws Exception {
        // Configure SSL.
        final SslContext sslCtx;
        if (SSL) {
            SelfSignedCertificate ssc = new SelfSignedCertificate();
            sslCtx = SslContextBuilder.forServer(ssc.certificate(), ssc.privateKey()).build();
        } else {
            sslCtx = null;
        }

        // Configure the server.
        EventLoopGroup bossGroup = new NioEventLoopGroup(1);
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup).channel(NioServerSocketChannel.class)
                    .option(ChannelOption.SO_KEEPALIVE, true).handler(new LoggingHandler(LogLevel.INFO))
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        public void initChannel(SocketChannel ch) throws Exception {
                            ChannelPipeline pipeline = ch.pipeline();
                            if (sslCtx != null) {
                                pipeline.addLast(sslCtx.newHandler(ch.alloc()));
                            }
                            pipeline.addLast("frameDecoder",new LengthFieldBasedFrameDecoder(64*1024, 0, 8));
                            pipeline.addLast("frameEncoder", new LengthFieldPrepender(4));
                            pipeline.addLast(new ObjectDecoder(ClassResolvers.cacheDisabled(null)));
                            pipeline.addLast(new ObjectEncoder());

                            pipeline.addLast(new ChunkedWriteHandler());
                            pipeline.addLast(new FileServerHandler());
                        }
                    });

            // Start the server.
            ChannelFuture f = b.bind(PORT).sync();

            // Wait until the server socket is closed.
            f.channel().closeFuture().sync();
        } finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }
}

FileServerHandler:

public class FileServerHandler extends ChannelInboundHandlerAdapter {

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object obj) throws Exception {
        RandomAccessFile raf = null;
        long length = -1;
        try {
            ByteBuf buff = (ByteBuf)obj;

            byte[] bytes = new byte[buff.readableBytes()];
            buff.readBytes(bytes);

            String msg = new String(bytes);

            raf = new RandomAccessFile(msg, "r");
            length = raf.length();
        } catch (Exception e) {
            ctx.writeAndFlush("ERR: " + e.getClass().getSimpleName() + ": " + e.getMessage() + '\n');
            return;
        } finally {
            if (length < 0 && raf != null) {
                raf.close();
            }
        }

        if (ctx.pipeline().get(SslHandler.class) == null) {
            // SSL not enabled - can use zero-copy file transfer.
            ctx.writeAndFlush(new DefaultFileRegion(raf.getChannel(), 0, length));
        } else {
            // SSL enabled - cannot use zero-copy file transfer.
            ctx.writeAndFlush(new ChunkedFile(raf));
        }
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
        cause.printStackTrace();
        System.out.println("Exception server.....");
    }
}

我引用了Netty In Action和来自here的代码示例

1 个答案:

答案 0 :(得分:1)

您的服务器/客户端有很多问题。首先是SSL,对于客户端,您不需要为服务器初始化SslContext,而是可以执行以下操作:

sslCtx = SslContextBuilder.forClient().trustManager(InsecureTrustManagerFactory.INSTANCE).build();

在服务器端,您使用的SelfSignedCertificate本身并没有错,但要提醒您,它仅应用于调试目的,而不能用于生产。另外,不建议使用ChannelOption.SO_KEEPALIVE,因为keepalive间隔取决于操作系统。此外,您已将Object En-/Decoder添加到管道中,这种情况对您没有任何帮助,因此可以将其删除。

由于参数 list 不完整和错误,您还配置了LengthFieldBasedFrameDecoder错误。在netty docs中,您需要定义lengthFieldLengthinitialBytesToStrip的构造函数版本。除了不剥离长度字段外,您还定义了错误的lengthFieldLength,它应该与LengthFieldPrepender的{​​{1}}相同,为4个字节。总之,您可以使用如下构造函数:

lengthFieldLength

在您的两个处理程序中,在对new LengthFieldBasedFrameDecoder(64 * 1024, 0, 4, 0, 4) 进行编码/解码时都没有指定Charset,这可能会导致问题,因为如果未定义“ Charset”,则将使用系统默认值变化。您可以执行以下操作:

String

另外,如果没有将//to encode the String string.getBytes(StandardCharsets.UTF_8); //to decode the String new String(bytes, StandardCharsets.UTF_8); 添加到管道中,那么您尝试使用DefaultFileRegion,如果您不添加SslHandler会很好,因为它们需要内存副本,因此可以使用要发送的byte []的长度添加到length字段。此外,我建议使用LengthFieldHandler而不是ChunkedNioFile,因为它是无阻塞的,这总是一件好事。您可以这样:

ChunkedFile

关于如何{em>解码 new ChunkedNioFile(randomAccessFile.getChannel()) 的最后一件事,因为它被分成几块,您可以使用简单的OutputStream简单地将它们拼凑在一起。这是我的旧文件处理程序:

ChunkedFile