如何从外部源通过Spring MVC异步传输大文件?

时间:2016-10-09 09:25:22

标签: java spring-mvc servlets asynchronous resttemplate

致电所有Spring Framework专家,

情景

  

用户 A 向Spring Controller执行GET请求,而另一个请求   GET请求远程主机获取流内容的文件(缓冲区   字节复制)作为响应的初始用户 A

PS:Spring Controller就像一个代理

  • Spring WebMVC 4.3.3.RELEASE
  • Apache Tomcat 8.5.5

问题

  

似乎没有可用的Servlet请求线程或其他东西   否则被阻止......

第一位用户可以在下面列出的所有情况下启动文件下载。 不幸的是, Spring 停止调用控制器下载方法,直到第一次下载完成(但有时,它会在用户等待时间的XX秒后调用它)。 用

尝试方法
  • @Async在包含RestTemplate方法的服务上 - 在NullPointerException期间将byte copy operation作为响应输出缓冲区nullat org.apache.coyote.http11.Http11OutputBuffer$SocketOutputBuffer.doWrite(Http11OutputBuffer.java:561))抛出。服务 - download方法。
  • StreamingResponseBody也无法解决并发下载问题,即使在服务级别@Async上包含AsyncResult<StreamingResponseBody>返回时也是如此。服务 - downloadAsync方法。
  

也许有人知道在春天这样做的更好方法吗?

途径

  • RestTemplateFileCopyUtils.copy(downloadResponse.getBody(), userResponse.getOutputStream());内执行ResponseExtractor
  • HttpsURLConnection在控制器的返回FileCopyUtils.copy(downloadResponse.getBody(), userResponse.getOutputStream());
  • 中执行StreamingResponseBody

所有列出的方法都有效,它们将文件内容从一个服务器响应InputStream传输到用户的响应OutputStream。 但是,并发下载存在问题( Spring 会停止调用控制器的下载方法)。

来源

私人信息(网址,身份验证等)被替换,代码严格用于问题演示目的

分派器

通过扩展AbstractAnnotationConfigDispatcherServletInitializer

来构建

控制器

包含3个不同方法的端点。

    @RequestMapping(value = "/download/way1", method = RequestMethod.GET)
    public void requestDownloadPage(final HttpServletResponse downloadResponse, final HttpServletRequest downloadRequest) throws ExecutionException, InterruptedException, URISyntaxException {
         dwn.download(downloadResponse);
    }

    @RequestMapping(value = "/download/way2", method = RequestMethod.GET)
    public StreamingResponseBody requestDownloadPage2(final HttpServletResponse response) throws ExecutionException, InterruptedException, URISyntaxException {
       return dwn.downloadAsync(response).get();
    }

    @RequestMapping(value = "/download/way3", method = RequestMethod.GET)
    public StreamingResponseBody requestDownloadPage3(final HttpServletResponse response) throws ExecutionException, InterruptedException, URISyntaxException, IOException {
        return outputStream -> {
            URL url = new URL("https://some/url/path/veryBig.zip");
            HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
            connection.setDoOutput(false);
            connection.setDoInput(true);
            connection.setUseCaches(false);
            connection.setRequestProperty ("Authorization", "Basic " + Base64.getEncoder().encodeToString("username:password".getBytes(StandardCharsets.UTF_8)));
            connection.setRequestMethod("GET");

            response.addHeader(HttpHeaders.CONTENT_TYPE, connection.getHeaderField(HttpHeaders.CONTENT_TYPE));
            response.addHeader(HttpHeaders.CONTENT_LENGTH, connection.getHeaderField(HttpHeaders.CONTENT_LENGTH));
            String disposition = connection.getHeaderField(HttpHeaders.CONTENT_DISPOSITION);
            if (disposition != null && !disposition.isEmpty()) {
                response.addHeader(HttpHeaders.CONTENT_DISPOSITION, disposition);
            }

            try (InputStream inputStream = connection.getInputStream();) {
                FileCopyUtils.copy(inputStream, outputStream);

            } catch (Throwable any) {
                // failed
            }
        };
    }

服务

public void download(HttpServletResponse downloadResponse) {
    final RestTemplate restTemplate = new RestTemplate();
    RequestCallback requestCallback = new RequestCallback() {
        @Override
        public void doWithRequest(ClientHttpRequest request) throws IOException {
            request.getHeaders().set(HttpHeaders.AUTHORIZATION, "Basic " + Base64.getEncoder().encodeToString("username:password".getBytes(StandardCharsets.UTF_8)));
        }
    };

    ResponseExtractor<Void> responseExtractor = response -> {

        List<String> type = response.getHeaders().get(HttpHeaders.CONTENT_TYPE);
        List<String> length = response.getHeaders().get(HttpHeaders.CONTENT_LENGTH);
        List<String> disposition = response.getHeaders().get(HttpHeaders.CONTENT_DISPOSITION);

        downloadResponse.addHeader(HttpHeaders.CONTENT_TYPE, type.get(0));
        downloadResponse.addHeader(HttpHeaders.CONTENT_LENGTH, length.get(0));
        if (disposition != null && !disposition.isEmpty()) {
            downloadResponse.addHeader(HttpHeaders.CONTENT_DISPOSITION, disposition.get(0));
        }

        FileCopyUtils.copy(response.getBody(), downloadResponse.getOutputStream());

        return null;
    };


    restTemplate.execute("https://some/url/path/veryBig.zip",
            HttpMethod.GET,
            requestCallback,
            responseExtractor);
}

@Async
public Future<StreamingResponseBody> downloadAsync(HttpServletResponse response) {
    final RestTemplate restTemplate = new RestTemplate();
    RequestCallback requestCallback = new RequestCallback() {
        @Override
        public void doWithRequest(ClientHttpRequest request) throws IOException {
            request.getHeaders().set(HttpHeaders.AUTHORIZATION, "Basic " + Base64.getEncoder().encodeToString("username:password".getBytes(StandardCharsets.UTF_8)));
        }
    };

    ResponseExtractor<StreamingResponseBody> responseExtractor = responseOwnCloud -> {

        List<String> type = responseOwnCloud.getHeaders().get(HttpHeaders.CONTENT_TYPE);
        List<String> length = responseOwnCloud.getHeaders().get(HttpHeaders.CONTENT_LENGTH);
        List<String> disposition = responseOwnCloud.getHeaders().get(HttpHeaders.CONTENT_DISPOSITION);

        response.setHeader(HttpHeaders.CONTENT_TYPE, type.get(0));
        response.setHeader(HttpHeaders.CONTENT_LENGTH, length.get(0));
        if (disposition != null && !disposition.isEmpty()) {
            response.setHeader(HttpHeaders.CONTENT_DISPOSITION, disposition.get(0));
        }

        return outputStream -> {
            FileCopyUtils.copy(responseOwnCloud.getBody(), response.getOutputStream());
        };
    };


    return new AsyncResult<>(restTemplate.execute("https://some/url/path/veryBig.zip",
            HttpMethod.GET,
            requestCallback,
            responseExtractor));

}

Spring异步配置

@Configuration
@ComponentScan(basePackages = { "some.service"})
@EnableAsync
public class AsyncConfig implements AsyncConfigurer {
    @Override
    public Executor getAsyncExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4);
        executor.setMaxPoolSize(4);
        executor.setQueueCapacity(500);
        executor.setThreadNamePrefix("AsyncExec-");
        executor.initialize();
        return executor;
    }

    @Override
    public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
        return null;
    }
}

0 个答案:

没有答案