下载大文件会在生产env

时间:2018-03-19 14:18:50

标签: angular hibernate spring-boot jhipster

的Env jhipster,角5, 春季靴子

我的应用已上传和下载文件工具。文件作为二进制blob存储在db中。

我可以下载小文件。但是,35M的大文件会出现此错误。 (这些大文件由应用上传)

根据堆栈跟踪,java.util.Arrays.copyOf抛出错误。 hibernate调用这个数组fn。我假设35M的二进制数据(以blob为单位)试图放在可以容纳2M左右的数组中。

是否有解决大数据的工作或修复?我们可以告诉hibernate将数据块化为位吗?

通过堆栈跟踪扫描也给出了指向com.neemshade.sniper.security.jwt.JWTFilter.doFilter(JWTFilter.java:36)的指针 第36行是

public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain)
        throws IOException, ServletException {
        HttpServletRequest httpServletRequest = (HttpServletRequest) servletRequest;
        String jwt = resolveToken(httpServletRequest);
        if (StringUtils.hasText(jwt) && this.tokenProvider.validateToken(jwt)) {
            Authentication authentication = this.tokenProvider.getAuthentication(jwt);
            SecurityContextHolder.getContext().setAuthentication(authentication);
        }
        filterChain.doFilter(servletRequest, servletResponse);  <-- this code
    }

如果此过滤器是错误的起源,那么它是否可能是休眠问题?

我的代码说明。 afterDownload()调用服务器api来下载二进制文件。一旦数据可用,就会调用saveToLocal()来弹出&#34;另存为&#34;用于将二进制数据存储在文件中的对话框。

这是我的客户端代码:

afterDownload(event) {
      this.pending = true;

      this.extTaskService.downloadFiles(this.source, this.id, this.selectedTasks)
      .subscribe(
        (data) => {
          this.jhiAlertService.success('success! downloaded files. ');
          console.log(data);
          this.saveToLocal(data);
          this.uponCompletion(true);
          this.pending = false;
        },
        (err) => {
          this.jhiAlertService.error('error in download! ' + err.message, null, null);
          console.log(JSON.stringify(err));
          this.pending = false;
        },
        () => this.jhiAlertService.success('downloaded files', null, null)
      );
    }

  saveToLocal(response) {
    console.log('inside saveToLocal');
    const ieEDGE = navigator.userAgent.match(/Edge/g);
    const ie = navigator.userAgent.match(/.NET/g); // IE 11+
    const oldIE = navigator.userAgent.match(/MSIE/g);

    const blob = new Blob([response], { type: 'application/octet-stream'});
    const fileName = 'files.zip';

    if (ie || oldIE || ieEDGE) {
      console.log('got to ie');
      window.navigator.msSaveBlob(blob, fileName);
    } else {
      const reader = new FileReader();
      reader.onloadend = function() {
        console.log('onloadend');
        window.location.href = reader.result;
      };
      console.log('readAsDataURL');
      reader.readAsDataURL(blob);
    }
  }

客户服务:

downloadFiles(source: string, id: number, selectedTasks: Task[]): Observable<any> {
    const finalUrl: string = this.resourceUrl + 'download-files/' + source + '/' + id + '/' +
           (selectedIds == null || selectedIds === '' ? '0' : selectedIds);
    return this.http.get(finalUrl, { responseType: 'blob' });
  }

服务器端: extDownloaderService.downloadFiles()准备输出二进制数据。我将数据以小块的形式刷新到响应中。

@GetMapping(value="download-files/{source}/{id}/{selectedIds}", produces="application/zip")
    @ResponseBody
    public void downloadFiles(
            @PathVariable String source, @PathVariable(value = "id") Long id,
            @PathVariable(value = "selectedIds") String selectedIds, HttpServletResponse response) throws Exception {

        byte[] bytes = extDownloaderService.downloadFiles(source, id, selectedIds);

//      headers.add("Content-Type", "application/octet-stream");
        response.setHeader("Content-Type", "application/zip");
        response.setHeader("Content-Disposition", "attachment; filename=\"files.zip\"");

        OutputStream os = response.getOutputStream();

        try
        {
            ByteArrayInputStream byteIs = new ByteArrayInputStream(bytes);


            byte[] buf=new byte[8192];
            int bytesread = 0, bytesBuffered = 0;
            while( (bytesread = byteIs.read( buf )) > -1 ) {
                os.write( buf, 0, bytesread );
                bytesBuffered += bytesread;
                if (bytesBuffered > 1024 * 1024) { //flush after 1MB
                    bytesBuffered = 0;
                    os.flush();
                }
            }
        }
        finally {
            if (os != null) {
                os.flush();
            }
        }

        os.close();

//      return new ResponseEntity<>(bytes, headers, HttpStatus.OK);
       }

当使用大型fie运行应用程序时,我会得到此堆栈跟踪。

  

@ 400000005aaeaf751478b324引起:java.lang.OutOfMemoryError:Java   堆空间@ 400000005aaeaf751478baf4 at   java.util.Arrays.copyOf(java.base@9-internal/Arrays.java:3744)   @ 400000005aaeaf751478bedc at   java.lang.AbstractStringBuilder.ensureCapacityInternal(java.base@9-internal/AbstractStringBuilder.java:146)   @ 400000005aaeaf751478bedc at   java.lang.AbstractStringBuilder.append(java.base@9-internal/AbstractStringBuilder.java:510)   @ 400000005aaeaf751478c2c4 at   java.lang.StringBuilder.append(java.base@9-internal/StringBuilder.java:141)   @ 400000005aaeaf751478ca94 at   java.util.Arrays.toString(java.base@9-internal/Arrays.java:4958)   @ 400000005aaeaf751478ca94 at   org.hibernate.type.descriptor.java.PrimitiveByteArrayTypeDescriptor.extractLoggableRepresentation(PrimitiveByteArrayTypeDescriptor.java:63)   @ 400000005aaeaf751478ce7c at   org.hibernate.type.descriptor.java.PrimitiveByteArrayTypeDescriptor.extractLoggableRepresentation(PrimitiveByteArrayTypeDescriptor.java:26)   @ 400000005aaeaf751478de1c at   org.hibernate.type.AbstractStandardBasicType.toLoggableString(AbstractStandardBasicType.java:296)   @ 400000005aaeaf751478e204 at   org.hibernate.type.TypeHelper.toLoggableString(TypeHelper.java:439)   @ 400000005aaeaf751478e204 at   org.hibernate.cache.spi.entry.StandardCacheEntryImpl。(StandardCacheEntryImpl.java:60)   @ 400000005aaeaf751478e9d4 at   org.hibernate.persister.entity.AbstractEntityPersister $ StandardCacheEntryHelper.buildCacheEntry(AbstractEntityPersister.java:5307)   @ 400000005aaeaf751478edbc at   org.hibernate.persister.entity.AbstractEntityPersister.buildCacheEntry(AbstractEntityPersister.java:4307)   @ 400000005aaeaf751478edbc at   org.hibernate.engine.internal.TwoPhaseLoad.doInitializeEntity(TwoPhaseLoad.java:196)   @ 400000005aaeaf751478f58c at   org.hibernate.engine.internal.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:125)   @ 400000005aaeaf751478f58c at   org.hibernate.loader.plan.exec.process.internal.AbstractRowReader.performTwoPhaseLoad(AbstractRowReader.java:238)   @ 400000005aaeaf751478f974 at   org.hibernate.loader.plan.exec.process.internal.AbstractRowReader.finishUp(AbstractRowReader.java:209)   @ 400000005aaeaf751478fd5c at   org.hibernate.loader.plan.exec.process.internal.ResultSetProcessorImpl.extractResults(ResultSetProcessorImpl.java:133)   @ 400000005aaeaf7514790cfc at   org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeLoad(AbstractLoadPlanBasedLoader.java:122)   @ 400000005aaeaf7514790cfc at   org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeLoad(AbstractLoadPlanBasedLoader.java:86)   @ 400000005aaeaf75147910e4 at   org.hibernate.loader.entity.plan.AbstractLoadPlanBasedEntityLoader.load(AbstractLoadPlanBasedEntityLoader.java:167)   @ 400000005aaeaf75147918b4 at   org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:4083)   @ 400000005aaeaf7514791c9c at   org.hibernate.event.internal.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:508)   @ 400000005aaeaf7514791c9c at   org.hibernate.event.internal.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:478)   @ 400000005aaeaf751479246c at   org.hibernate.event.internal.DefaultLoadEventListener.load(DefaultLoadEventListener.java:219)   @ 400000005aaeaf751479246c at   org.hibernate.event.internal.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:278)   @ 400000005aaeaf7514792854 at   org.hibernate.event.internal.DefaultLoadEventListener.doOnLoad(DefaultLoadEventListener.java:121)   @ 400000005aaeaf75147937f4 at   org.hibernate.event.internal.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:89)   @ 400000005aaeaf7514793bdc at   org.hibernate.internal.SessionImpl.fireLoad(SessionImpl.java:1239)   @ 400000005aaeaf7514793bdc at   org.hibernate.internal.SessionImpl.internalLoad(SessionImpl.java:1122)   @ 400000005aaeaf7514793fc4 at   org.hibernate.type.EntityType.resolveIdentifier(EntityType.java:646)   @ 400000005aaeaf7514794f64 at   org.hibernate.type.EntityType.resolve(EntityType.java:431)   @ 400000005aaeaf7514794f64 at   org.hibernate.engine.internal.TwoPhaseLoad.doInitializeEntity(TwoPhaseLoad.java:165)

任何帮助表示感谢。

1 个答案:

答案 0 :(得分:1)

我不确定您是否尝试过,但是请尝试以下解决方案,

解决方案-1:允许上传文件的最大大小,该大小在配置文件中提及。如果未定义或尺寸较小,则增加它(根据您的要求进行设置)。

#Multipart
spring.http.multipart.enable=true
spring.servlet.multipart.max-request-size=50MB
spring.servlet.multipart.max-file-size=50MB

解决方案-2:如果您使用的是Spring Boot,那么它将为您提供一种将文件作为资源发送的便利。因此,请尝试按照以下方式操作,

Path path = Paths.get(<directory-path>).normalize();
Path filePath = path.resolve(fileName).normalize();
Resource resource = new UrlResource(filePath.toUri());
Return ResponseEntity.ok
        .contentType(MediaType.<MIME-Type>)
        .header(HttpHeaders.CONTENT_DISPOSITION, “attachment; filename=””+ resource.filename())
        .body(resource);

此方法的返回类型:ResponseEntity<Resource>

解决方案-3:在不能上传大型文件的服务器或系统上,服务器上存在权限或限制,因此请与系统联系支持团队(我对此不太清楚,因为我在0.6年前就这么做了)