使用Armeria / Netty中断在两点之间传输数据

时间:2019-04-29 15:25:45

标签: java asynchronous streaming netty

我有一个应用程序,根据收到的路径,它应该在openstack-swift和s3之间流式传输文件。

当前我正在使用基于净值的Armeria框架

一段时间后传输中断

两个客户端的超时都已达到最大值,并且挂起的时间不一定总是相同。

  @Override
    protected HttpResponse doPost(ServiceRequestContext context, HttpRequest request) {
        final CompletableFuture<HttpResponse> responseFuture = new CompletableFuture<>();

        CompletableFuture<HttpResponse> requestMessageFuture = request.aggregate().thenApplyAsync((AggregatedHttpMessage message) -> {
            // Get the object key from the message
            JSONObject o = new JSONObject(message.content().toStringUtf8());
            String key = o.getString("key");
            String token = request.headers().get(AsciiString.of("x-auth-token"));
            CompletableFuture<URL> signedUrlFuture = S3Utils.getInstance().getS3SignedUrl(com.amazonaws.HttpMethod.PUT, key);
            // Lock and check for lock status
            //Lock lock = lockKey(key);
            Lock lock = null;

            log.info("dbg: before get");

            HttpHeaders getHeaders = HttpHeaders.of(
                    AsciiString.of("x-auth-token"), token,
                    AsciiString.of(":path"), "/v1/" + key,
                    AsciiString.of(":method"), "GET"
            );

            log.info(token + "  /v1/" + key);

            signedUrlFuture.whenComplete((signedUrl, throwable) -> swiftClient.execute(HttpRequest.of(getHeaders))
                    .subscribe(new Subscriber<HttpObject>() {
                        private HttpRequestWriter writer;
                        private Subscription subscription;
                        private long size = 0;
                        private long tSize = 0;

                        @Override
                        public void onSubscribe(Subscription subscription) {
                            this.subscription = subscription;
                            subscription.request(1);
                        }

                        @Override
                        public final void onNext(HttpObject o) {
                            log.info("dbg: on next");
                            if (o instanceof HttpHeaders) {
                                log.info("dbg: on next headers");
                                onHeaders((HttpHeaders) o);
                                subscription.request(1);
                            } else if (o instanceof HttpData) {
                                log.info("dbg: on next other");
                                log.info("size: " + size);
                                log.info("tran: " + tSize);

                                writer.write(o);

                                // We increase the transferred data counter.
                                tSize += ((HttpData) o).length();

                                writer.onDemand(() -> {
                                    log.info("dbg: on next demand");
                                    subscription.request(1);
                                });
                            }
                        }

                        private void onHeaders(HttpHeaders headers) {
                            log.info("dbg: on headers");

                            if (headers.status().equals(HttpStatus.OK)) {

                                size = headers.getLong(AsciiString.of("Content-Length"));

                                headers.set(AsciiString.of(":path"), signedUrl.getPath());
                                headers.set(AsciiString.of(":method"), "PUT");

                                log.info("dbg: headers: " + headers);

                                writer = HttpRequest.streaming(translateHeaders(headers, key));

                                s3HttpClient.execute(writer).closeFuture();
                            } else {
                                log.error("key " + key + " error, status code: " + headers.status());
                                writer.close();
                                subscription.cancel();
                            }
                        }

                        @Override
                        public void onError(Throwable throwable) {
                            log.error("error migrating, " + throwable);
                            writer.close(throwable);
                            responseFuture.complete(unlockingResponse(500, lock, key));
                        }

                        @Override
                        public void onComplete() {
                            log.info("completed, " + key);
                            writer.close();
                            responseFuture.complete(unlockingResponse(200, lock, key));
                        }
                    }));
            return HttpResponse.from(responseFuture);
        });
        return HttpResponse.from(requestMessageFuture);
    }

似乎有时会挂起,有时会在此行上引发ClosedPublisherException:

writer.write(o);

0 个答案:

没有答案