GAE从URL将大文件上传到blobstore

时间:2013-03-07 22:29:46

标签: java google-app-engine

我想将30MB到2GB的大型视频文件上传到具有外部URL的blob商店,以便访问此文件。

到目前为止,我尝试了类似这样的东西,但它不适用于大文件,我认为是因为32MB限制:

    //InputStream is = new URL(getUrlThumbnailFullSize(wistiaResponse)).openStream();
    HttpURLConnection con = (HttpURLConnection) new URL("URL of the file here").openConnection();
    con.setConnectTimeout(0);
    con.connect();
    InputStream is = con.getInputStream();
    //create file service to upload the video to the blob store
    FileService fileService = FileServiceFactory.getFileService();
    //create the App Engine File; assign MIME type and name
    AppEngineFile file = fileService.createNewBlobFile("content type here", "file name here");
    //lock when uploading file
    boolean lock = true;
    //open channel
    FileWriteChannel writeChannel = fileService.openWriteChannel(file, lock);
    //upload file
    writeChannel.write(ByteBuffer.wrap(IOUtils.toByteArray(is)));
    writeChannel.closeFinally();
    //release the lock
    lock = false;
    //get blob key
    BlobKey blobKey = FileServiceFactory.getFileService().getBlobKey(file);

但我有以下堆栈跟踪:

  

严重:com.google.appengine.api.urlfetch.ResponseTooLargeException:   来自网址的回复   http://embed.wistia.com/deliveries/a0f8dc8ed210b53152af601eb0af40d76cd2929f.bin   太大了在   com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:57)     在   com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler $ Connection.fetchResponse(URLFetchServiceStreamHandler.java:417)     在   com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler $ Connection.getInputStream(URLFetchServiceStreamHandler.java:296)     在controller.CtrlWistia.download(CtrlWistia.java:149)at   controller.CtrlWistia.ajax(CtrlWistia.java:89)at   controller.CtrlAjax.main(CtrlAjax.java:66)at   vidaao.AjaxServlet.doPost(AjaxServlet.java:37)at   javax.servlet.http.HttpServlet.service(HttpServlet.java:637)at at   javax.servlet.http.HttpServlet.service(HttpServlet.java:717)at at   org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)     在   org.mortbay.jetty.servlet.ServletHandler $ CachedChain.doFilter(ServletHandler.java:1166)     在   org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)     在   org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)     在   org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)     在   org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)     在   org.mortbay.jetty.servlet.ServletHandler $ CachedChain.doFilter(ServletHandler.java:1157)     在   com.google.appengine.api.socket.dev.DevSocketFilter.doFilter(DevSocketFilter.java:74)     在   org.mortbay.jetty.servlet.ServletHandler $ CachedChain.doFilter(ServletHandler.java:1157)     在   com.google.appengine.tools.development.ResponseRewriterFilter.doFilter(ResponseRewriterFilter.java:123)     在   org.mortbay.jetty.servlet.ServletHandler $ CachedChain.doFilter(ServletHandler.java:1157)     在   com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:34)     在   org.mortbay.jetty.servlet.ServletHandler $ CachedChain.doFilter(ServletHandler.java:1157)     在   com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:61)     在   org.mortbay.jetty.servlet.ServletHandler $ CachedChain.doFilter(ServletHandler.java:1157)     在   com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)     在   org.mortbay.jetty.servlet.ServletHandler $ CachedChain.doFilter(ServletHandler.java:1157)     在   com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:125)     在   org.mortbay.jetty.servlet.ServletHandler $ CachedChain.doFilter(ServletHandler.java:1157)     在   com.google.appengine.tools.development.BackendServersFilter.doFilter(BackendServersFilter.java:97)     在   org.mortbay.jetty.servlet.ServletHandler $ CachedChain.doFilter(ServletHandler.java:1157)     在   org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)     在   org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)     在   org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)     在   org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)     在   org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)     在   com.google.appengine.tools.development.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:94)     在   org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)     在   com.google.appengine.tools.development.JettyContainerService $ ApiProxyHandler.handle(JettyContainerService.java:409)     在   org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)     在org.mortbay.jetty.Server.handle(Server.java:326)

1 个答案:

答案 0 :(得分:1)

请使用blobstoreService.createUploadUrl进行大文件上传。

[更新] 啊,我想我明白了。 目前,由于URLFetch服务具有32MB响应大小限制,因此无法实现。作为一种解决方法,您可以从客户端javascript中的URL下载文件,并使用blobstore服务创建的上传URL上传它吗?

[更新] 好的,你需要在服务器端实现这一点,这个问题我不清楚。所以请忘记createUploadUrl。

如果服务器支持Range标头,您可以多次请求较小的数据块并将它们存储为单独的blob存储对象,以及所需的元数据,即文件名,mimetype,blob键列表等。

此外,如果您想将它们合并为一个文件,您可以使用Google云端存储。使用Resumable upload option,您可以多次向Google云端存储发送大量数据。