通过appengine将文件上传到Google云端存储

时间:2016-07-26 13:17:37

标签: java google-app-engine google-cloud-storage google-cloud-endpoints

我正在尝试将文件上传到Google云端存储。我的Servlet代码是

public class UploadFile extends HttpServlet {

    private final String BUCKET = "XXXXXXXXX";
       private boolean isMultipart;
       private String filePath;
       private int maxFileSize = 5 * 1024 * 1024;
       private int maxMemSize = 5 * 1024 * 1024;
       private File file ;

       public void init( ){
          // Get the file location where it would be stored.
          filePath = 
                 getServletContext().getInitParameter("file-upload"); 
       }
       public void doPost(HttpServletRequest request, 
                   HttpServletResponse response)
                  throws ServletException, java.io.IOException {
          // Check that we have a file upload request
          isMultipart = ServletFileUpload.isMultipartContent(request);
          response.setContentType("text/html");
          java.io.PrintWriter out = response.getWriter( );
          if( !isMultipart ){
             out.println("<html>");
             out.println("<head>");
             out.println("<title>Servlet upload</title>");  
             out.println("</head>");
             out.println("<body>");
             out.println("<p>No file uploaded</p>"); 
             out.println("</body>");
             out.println("</html>");
             return;
          }
          DiskFileItemFactory factory = new DiskFileItemFactory();
          // maximum size that will be stored in memory
          factory.setSizeThreshold(maxMemSize);
          // Location to save data that is larger than maxMemSize.
          factory.setRepository(new File("/temp/image/"));

          // Create a new file upload handler
          ServletFileUpload upload = new ServletFileUpload(factory);
          // maximum file size to be uploaded.
          upload.setSizeMax( maxFileSize );

          try{ 
          // Parse the request to get file items.
          List fileItems = upload.parseRequest(request);

          // Process the uploaded file items
          Iterator i = fileItems.iterator();

          out.println("<html>");
          out.println("<head>");
          out.println("<title>Servlet upload</title>");  
          out.println("</head>");
          out.println("<body>");
          while ( i.hasNext () ) 
          {
             FileItem fi = (FileItem)i.next();
             if ( !fi.isFormField () )  
             {
                // Get the uploaded file parameters
                String fieldName = fi.getFieldName();
                String fileName = fi.getName();
                String contentType = fi.getContentType();
                boolean isInMemory = fi.isInMemory();
                long sizeInBytes = fi.getSize();
                // Write the file
                if( fileName.lastIndexOf("\\") >= 0 ){
                   file = new File( filePath + 
                   fileName.substring( fileName.lastIndexOf("\\"))) ;
                }else{
                   file = new File( filePath + 
                   fileName.substring(fileName.lastIndexOf("\\")+1)) ;
                }

               String path = Events.uploadFile ( fileName, "image/*", file, BUCKET );

                // fi.write( file ) ;
                out.println("Uploaded Filename: " + fileName + "<br>"+ " File Path:"+ path);
             }
          }
          out.println("</body>");
          out.println("</html>");
       }catch(Exception ex) {
           System.out.println(ex);
       }
       }
       public void doGet(HttpServletRequest request, 
                           HttpServletResponse response)
            throws ServletException, java.io.IOException {

            throw new ServletException("GET method used with " +
                    getClass( ).getName( )+": POST method required.");
       } 
    }

的web.xml

<servlet>
        <servlet-name>UploadFile</servlet-name>
        <servlet-class>XXXXXXXXXX.UploadFile</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>UploadFile</servlet-name>
        <url-pattern>/uploadManager/UploadFile</url-pattern>  //Based on your original URL
    </servlet-mapping>

fileUpload函数将文件保存到GCS

public static String uploadFile ( String name, String contentType, File file, String bucketName )
            throws IOException, GeneralSecurityException
    {

        InputStreamContent contentStream = new InputStreamContent ( contentType, new FileInputStream ( file ) );

        // Setting the length improves upload performance
        contentStream.setLength ( file.length () );

        StorageObject objectMetadata = new StorageObject ()
                // Set the destination object name
                .setName ( name )
                // Set the access control list to publicly read-only
                .setAcl ( Arrays.asList ( new ObjectAccessControl ().setEntity ( "allUsers" ).setRole ( "READER" ) ) );

        // Do the insert
        Storage client = StorageFactory.getService ();
        Storage.Objects.Insert insertRequest = client.objects ().insert ( bucketName, objectMetadata, contentStream );

        insertRequest.execute ();
        return "https://storage.cloud.google.com/" + BUCKET + "/" + file.getName ();
    }

但是当我尝试使用某些API测试客户端进行测试时,它会出错

org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found

进一步将其与Angular中的UI集成并在本地测试后,我面临这个问题

Cross-Origin Request Blocked Reason: CORS header 'Access-Control-Allow-Origin' missing

我尝试解决此问题,但未找到与google appengine相对应的解决方案。

最初我尝试通过此代码上传图片,但在不久的将来,相同的代码将用于将.pdf和.html文件上传到GCS。

供参考: 我正在使用Google Endpoints满足客户端的其他数据通信需求。 Client End是Angular中的webapp构建,但它将扩展到android和ios。

任何帮助将不胜感激。

谢谢

更新2016年1月8日

现在我收到服务器上的文件,但我不知道在将文件发送到Google云端存储之前我必须暂时保存该文件。在

中存储文件
war\WEB-INI\

我面临的例外是

 java.security.AccessControlException: access denied ("java.io.FilePermission" "\war\WEB-INI\profile.png" "read")

5 个答案:

答案 0 :(得分:4)

最后,我可以通过appengine将文件从客户端上传到Google云端存储。

我认为在执行这些步骤之前,您已准备好以下事项

  • 您服务帐户中的JSON文件。
  • 创建了默认存储桶。

第1步:制作这样的Servlet

package XXXXXXXXX;

import java.io.IOException;
import java.io.InputStream;
import java.security.GeneralSecurityException;
import java.util.Arrays;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.google.api.client.http.InputStreamContent;
import com.google.api.services.storage.Storage;
import com.google.api.services.storage.model.ObjectAccessControl;
import com.google.api.services.storage.model.StorageObject;

import XXXXXXXXXXXXX.StorageFactory;

//@author Umesh Chauhan

/**
 * Save File to GCS
 *
 * @param fileName        File Name with format
 * @header Content-Type   "*/*"
 * @return file path
 * @throws Exception      Any Error during upload
 */
public class UploadFile extends HttpServlet
{

    private static final long serialVersionUID = 1L;
    private final String BUCKET = "YOUR BUCKET NAME";
    private int maxFileSize = 6 * 1024 * 1024;

    @Override
    protected void doOptions ( HttpServletRequest req, HttpServletResponse resp ) throws ServletException, IOException
    {
        // pre-flight request processing
        resp.setHeader ( "Access-Control-Allow-Origin", "*" );
        resp.setHeader ( "Access-Control-Allow-Methods", "*" );
        resp.setHeader ( "Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept" );
    }

    @Override
    public void doPost ( HttpServletRequest request, HttpServletResponse response )
            throws ServletException, java.io.IOException
    {

        try
        {
            String path = uploadFile ( request.getParameter ( "fileName" ), request.getContentType (),
                    request.getInputStream (), BUCKET, request.getInputStream ().available () );
            // Sending Response
            response.setStatus ( HttpServletResponse.SC_OK );
            response.getWriter ().write ( path );
            response.getWriter ().flush ();
            response.getWriter ().close ();

        }
        catch ( GeneralSecurityException e )
        {
            e.printStackTrace ();
        }
    }

    public String uploadFile ( String name, String contentType, InputStream input, String bucketName,
            int contentLength ) throws IOException, GeneralSecurityException
    {

        InputStreamContent contentStream = new InputStreamContent ( contentType, input );

        if ( contentLength < maxFileSize )
        {

            // It is done Automatically.
            /*
             * // Setting the length improves upload performance
             * contentStream.setLength ( contentLength );
             */

            StorageObject objectMetadata = new StorageObject ()
                    // Set the destination object name
                    .setName ( name )
                    // Set the access control list to publicly read-only
                    .setAcl ( Arrays.asList (
                            new ObjectAccessControl ().setEntity ( "allUsers" ).setRole ( "READER" ) ) );

            // Do the insert
            Storage client = StorageFactory.getService ();

            Storage.Objects.Insert insertRequest = client.objects ()
                    .insert ( bucketName, objectMetadata, contentStream );

            insertRequest.execute ();

            return "https://storage.cloud.google.com/" + BUCKET + "/" + name;
        }
        else
        {
            throw new GeneralSecurityException ( "File size canot be more then 6 MB !" );
        }
    }

    public void doGet ( HttpServletRequest request, HttpServletResponse response )
            throws ServletException, java.io.IOException
    {
        throw new ServletException ( "GET method used with " + getClass ().getName () + ": POST method required." );
    }

}

第2步:您的存储工厂

package XXXXXXXXXXXX;

import java.io.IOException;
import java.net.URL;
import java.security.GeneralSecurityException;
import java.util.Collection;

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.storage.Storage;
import com.google.api.services.storage.StorageScopes;

//@author Umesh Chauhan 

public class StorageFactory
{

    private static Storage instance = null;

    public static synchronized Storage getService () throws IOException, GeneralSecurityException
    {
        if ( instance == null )
        {
            instance = buildService ();
        }
        return instance;
    }

    private static Storage buildService () throws IOException, GeneralSecurityException
    {

        HttpTransport transport = GoogleNetHttpTransport.newTrustedTransport ();
        JsonFactory jsonFactory = new JacksonFactory ();

        GoogleCredential credential = GoogleCredential.fromStream (
                new URL ( "HERE GOES THE URL FOR YOUR SERVICE ACCOUNT JSON - I USED GOOGLE DRIVE DIRECT DOWNLOAD LINK TO MY JSON FILE" )
                        .openStream () );

        // Depending on the environment that provides the default credentials
        // (for
        // example: Compute Engine, App Engine), the credentials may require us
        // to
        // specify the scopes we need explicitly. Check for this case, and
        // inject
        // the Cloud Storage scope if required.
        if ( credential.createScopedRequired () )
        {
            Collection<String> scopes = StorageScopes.all ();
            credential = credential.createScoped ( scopes );
        }

        return new Storage.Builder ( transport, jsonFactory, credential ).setApplicationName ( "YOUR PROJECT NAME" ).build ();
    }
}

第3步:更新web.xml

    <servlet>
        <servlet-name>UploadFile</servlet-name>
        <servlet-class>PACKAGE.UploadFile</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>UploadFile</servlet-name>
        <url-pattern>/uploadManager/UploadFile</url-pattern>  //Based on your original URL
    </servlet-mapping>

答案 1 :(得分:2)

以下是根据docs

上传到存储空间的正确方法
import com.google.appengine.tools.cloudstorage.*;

public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
  String buffer = "the data to be saved in GCS bla bla";
  GcsFileOptions instance = GcsFileOptions.getDefaultInstance();
  GcsFilename fileName = new GcsFilename("bucketName", "fileName");
  GcsOutputChannel outputChannel;
  gcsService.createOrReplace(fileName, instance, ByteBuffer.wrap(buffer.getBytes()));
}

您将找到完整的代码here

答案 2 :(得分:1)

首先,您需要在飞行前请求处理中解决CORS问题,您需要在后端执行:在Google App Engine上通过添加doOptions方法完成:

@Override
protected void doOptions(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException 
{ 
    resp.setHeader("Access-Control-Allow-Origin", "*");
    resp.setHeader("Access-Control-Allow-Methods", "*");
    resp.setHeader("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
}

然后您需要确保使用Content-Type: multipart/form-data这样的标头发送请求,否则您的文件将被错误编码。在angular2请求标头中,您的帖子请求中设置为第三个(可选)参数,如:

let headers = new Headers();
headers.append('content-type', 'multipart/form-data');
http.post(url, body, {
                  headers:headers
                })

答案 3 :(得分:1)

对不起,这并没有直接解决你的问题,但无论如何我想指出来。我建议使用您的一个正常端点生成一个临时上传URL,Angular客户端可以使用该URL直接将文件发送到云存储,而无需通过您的应用程序。一种方法是通过blobstore API,如here所述。您可以按照here所述,通过该API上传到云端存储(在同一页面的下方)。

这会减少您在服务器上所需的上传代码量,不受GAE请求的32MB限制,并且符合Google的建议(如上文文档链接所示)。

答案 4 :(得分:1)

使用com.google.cloud.storage

  • 在appengine上运行时 - 无需身份验证
  • 在本地运行gcloud auth application-default login命令时 见authentication

    import com.google.cloud.storage.*;
    
    
    public static void upload(String bucketName, String fileId, String content) throws IOException {
        Storage storage = StorageOptions.newBuilder().build().getService();
    
        BlobInfo fileInfo = BlobInfo.newBuilder(bucketName, fileId)
                .build();
    
        InputStream fileIS = IOUtils.toInputStream(content, "UTF-8");
        storage.create(fileInfo, fileIS);
    
    }