如何使用AzureBlobStorage Java SDK v10仅使用SAS创建容器并上载数据

时间:2019-02-18 14:26:43

标签: java azure-blob-storage

我从大学获得了SAS令牌。 SAS令牌如下所示:

https://myaccount.blob.core.windows.net/?sv=2015-04-05&st=2015-04-29T22%3A18%3A26Z&se=2015-04-30T02%3A23%3A26Z&sr=b&sp=rw&sip=168.1.5.60-168.1.5.70&spr=https&sig=Z%2FRHIX5Xcg0Mq2rqI3OlWTjEg2tYkboXr1P9ZUXDtkk%3D

他说我可以用它来创建容器并将数据推送到该容器中。因此,我开始借助一些示例进行编码。我想到了这个:

public ContainerURL getContainerURL() throws MalformedURLException {

    // From the Azure portal, get your Storage account's name and account key.
    String storgeResourceUri = "https://<account>.blob.core.windows.net";
    String sasToken = "https://myaccount.blob.core.windows.net/?sv=2015-04-05&st=2015-04-29T22%3A18%3A26Z&se=2015-04-30T02%3A23%3A26Z&sr=b&sp=rw&sip=168.1.5.60-168.1.5.70&spr=https&sig=Z%2FRHIX5Xcg0Mq2rqI3OlWTjEg2tYkboXr1P9ZUXDtkk%3D";

    StorageCredentialsSharedAccessSignature creds = new StorageCredentialsSharedAccessSignature(sasToken);


    // Create a ServiceURL objet that wraps the service URL and a request pipeline.
    ServiceURL serviceURL = new ServiceURL(new URL(storgeResourceUri), StorageURL.createPipeline(creds, new PipelineOptions()));

    // Now you can use the ServiceURL to perform various container and blob operations.

    // This example shows several common operations just to get you started.

    /*
    Create a URL that references a to-be-created container in your Azure Storage account. This returns a
    ContainerURL object that wraps the container's URL and a request pipeline (inherited from serviceURL).
    Note that container names require lowercase.
     */

    return serviceURL.createContainerURL("mapupdate-container");
}

public void uploadFile(ContainerURL containerURL, File sourceFile) throws IOException {

    String timeStamp = new SimpleDateFormat("yyyy.MM.dd.HH.mm.ss").format(new Date());

    File zippedFile = zipping(sourceFile, timeStamp);

    BlockBlobURL blobURL = containerURL.createBlockBlobURL(zippedFile.getName());

    AsynchronousFileChannel fileChannel = AsynchronousFileChannel.open(zippedFile.toPath());

    // Uploading a file to the blobURL using the high-level methods available in TransferManager class
    // Alternatively call the Upload/StageBlock low-level methods from BlockBlobURL type
    TransferManager.uploadFileToBlockBlob(fileChannel, blobURL, (int) zippedFile.length(), null)
            .subscribe(response -> {
                System.out.println("Completed upload request.");
                System.out.println(response.response().statusCode());
            });
}

它应该创建一个名称为“ mapudpater-container”的容器,然后上传一个压缩文件。之后,我想通过mqtt发送,另一个客户端可以通过链接对其进行下载。

我不确定如何使用SAS令牌访问Azure Blob存储帐户并创建容器并将文件上传到容器中。我应该为每个文件创建一个新容器吗?我如何获得链接以从中下载?我必须创建自己的SAS令牌才能下载它吗?

2 个答案:

答案 0 :(得分:1)

有两种与AzureBlobStorage进行交互并使用Java管理文件的方式,

  1. Azure Storage Blob-它使用基于REST的机制与Azure Storage Blob连接并管理文件。比较慢。

    <dependency>
     <groupId>com.microsoft.azure</groupId>
     <artifactId>azure-storage-blob</artifactId>
     <version>11.0.0</version>
    </dependency>
    
  2. Azure存储-Azure提供的Cloud API,它比基于REST的API更快

    <dependency>
        <groupId>com.microsoft.azure</groupId>
        <artifactId>azure-storage</artifactId>
        <version>8.6.4</version>
    </dependency>
    

以下是针对两种实现方式创建的实用程序:

  import java.io.ByteArrayOutputStream;
  import java.io.FileInputStream;
  import java.io.IOException;
  import java.net.URISyntaxException;
  import java.net.URL;
  import java.nio.ByteBuffer;
  import java.security.InvalidKeyException;
  import java.util.Calendar;
  import java.util.EnumSet;
  import java.util.GregorianCalendar;
  import java.util.Locale;
  import java.util.Objects;
  import java.util.TimeZone;
  
  import com.microsoft.azure.storage.CloudStorageAccount;
  import com.microsoft.azure.storage.StorageException;
  import com.microsoft.azure.storage.blob.BlockBlobURL;
  import com.microsoft.azure.storage.blob.CloudBlobClient;
  import com.microsoft.azure.storage.blob.CloudBlobContainer;
  import com.microsoft.azure.storage.blob.CloudBlockBlob;
  import com.microsoft.azure.storage.blob.ContainerURL;
  import com.microsoft.azure.storage.blob.DownloadResponse;
  import com.microsoft.azure.storage.blob.PipelineOptions;
  import com.microsoft.azure.storage.blob.ServiceURL;
  import com.microsoft.azure.storage.blob.SharedAccessBlobPermissions;
  import com.microsoft.azure.storage.blob.SharedAccessBlobPolicy;
  import com.microsoft.azure.storage.blob.SharedKeyCredentials;
  import com.microsoft.azure.storage.blob.StorageURL;
  import com.microsoft.azure.storage.blob.models.BlobDeleteResponse;
  import com.microsoft.azure.storage.blob.models.BlockBlobUploadResponse;
  import com.microsoft.azure.storage.core.Base64;
  import com.microsoft.rest.v2.http.HttpPipeline;
  import com.microsoft.rest.v2.util.FlowableUtil;
  
  import io.reactivex.Flowable;
  import io.reactivex.Single;

  public class AzureStorageUtils {

public static void main(String...args) throws Exception {
    FileInputStream fis = new FileInputStream("test.pdf");
    int available = fis.available();
    byte[] buffer = new byte[available];
    fis.read(buffer);
    fis.close();
    
    String accountName = "enter azure account name";
    String accountKey = "enter account key ";
    
    BlobFile file = new BlobFile();
    file.setFileType("image");
    file.setFileName("directory1/image.pdf");
    file.setFileSize(available);
    file.setContent(buffer);
    
    long startTime = System.currentTimeMillis();
    
    uploadFile(accountName, accountKey, file);
    //deleteFile(accountName, accountKey, file);
    
    long endTime = System.currentTimeMillis();
    System.out.println("Start Time: " + startTime);
    System.out.println("End Time: " + endTime);
    System.out.println("Time Taken: " + (endTime - startTime));
    
    
    String connectionString = "entery connection string for azure blob";

    startTime = System.currentTimeMillis();
    create(connectionString, "image", "directory1/image.pdf", buffer);
    endTime = System.currentTimeMillis();
    System.out.println("Start Time: " + startTime);
    System.out.println("End Time: " + endTime);
    System.out.println("Time Taken: " + (endTime - startTime));

    startTime = System.currentTimeMillis();
    byte[] temp = read(connectionString, "image", "directory1/image.pdf");
    System.out.println(Base64.encode(temp));
    endTime = System.currentTimeMillis();
    System.out.println("Start Time: " + startTime);
    System.out.println("End Time: " + endTime);
    System.out.println("Time Taken: " + (endTime - startTime));

    startTime = System.currentTimeMillis();
    String sasURL = sasURL(connectionString, "image", "directory1/image.pdf");
    System.out.println(sasURL);
    endTime = System.currentTimeMillis();
    System.out.println("Start Time: " + startTime);
    System.out.println("End Time: " + endTime);
    System.out.println("Time Taken: " + (endTime - startTime));
    
    startTime = System.currentTimeMillis();
    delete(connectionString, "image", "directory1/image.pdf");
    endTime = System.currentTimeMillis();
    System.out.println("Start Time: " + startTime);
    System.out.println("End Time: " + endTime);
    System.out.println("Time Taken: " + (endTime - startTime));
}


/**
* azure-storage
*/
private static CloudBlobClient createCloudBlobClient(String connectionString) throws InvalidKeyException, URISyntaxException {
    CloudStorageAccount account = CloudStorageAccount.parse(connectionString);
    return account.createCloudBlobClient();
}

private static CloudBlobContainer getBlobConatiner(String connectionString, String containerName) throws InvalidKeyException, URISyntaxException, StorageException {
    CloudBlobClient cloudBlobClient = createCloudBlobClient(connectionString);
    CloudBlobContainer cloudBlobContainer = cloudBlobClient.getContainerReference(containerName);
    cloudBlobContainer.createIfNotExists();
    return cloudBlobContainer;
}

public static CloudBlockBlob create(String connectionString, String containerName, String filePath, byte[] data) throws RuntimeException {
    try {
        CloudBlobContainer cloudBlobContainer = getBlobConatiner(connectionString, containerName);
        CloudBlockBlob cloudBlockBlob = cloudBlobContainer.getBlockBlobReference(filePath);
        cloudBlockBlob.uploadFromByteArray(data, 0, data.length);
        return cloudBlockBlob;
    }
    catch(InvalidKeyException | URISyntaxException | StorageException | IOException ex) {
        throw new RuntimeException();
    }
}

public static byte[] read(String connectionString, String containerName, String filePath) throws RuntimeException {
    try {
        CloudBlobContainer cloudBlobContainer = getBlobConatiner(connectionString, containerName);
        CloudBlockBlob cloudBlockBlob = cloudBlobContainer.getBlockBlobReference(filePath);
        ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
        cloudBlockBlob.download(outputStream);
        byte[] byteArray =  outputStream.toByteArray();
        outputStream.close();
        return byteArray;
    }
    catch(InvalidKeyException | URISyntaxException | StorageException | IOException ex) {
        throw new RuntimeException();
    }
}

public static boolean delete(String connectionString, String containerName, String filePath) throws RuntimeException {
    try {
        CloudBlobContainer cloudBlobContainer = getBlobConatiner(connectionString, containerName);
        CloudBlockBlob cloudBlockBlob = cloudBlobContainer.getBlockBlobReference(filePath);
        return cloudBlockBlob.deleteIfExists();
    }
    catch(InvalidKeyException | URISyntaxException | StorageException ex) {
        throw new RuntimeException();
    }
}

public static String sasURL(String connectionString, String containerName, String filePath) throws RuntimeException {
    try {
        CloudBlobContainer cloudBlobContainer = getBlobConatiner(connectionString, containerName);
        CloudBlockBlob cloudBlockBlob = cloudBlobContainer.getBlockBlobReference(filePath);
        SharedAccessBlobPolicy sasPolicy = new SharedAccessBlobPolicy();
        GregorianCalendar calendar = new GregorianCalendar(TimeZone.getTimeZone("UTC"));
        calendar.add(Calendar.HOUR, 10);
        sasPolicy.setSharedAccessExpiryTime(calendar.getTime());
        sasPolicy.setPermissions(EnumSet.of(SharedAccessBlobPermissions.READ, SharedAccessBlobPermissions.WRITE, SharedAccessBlobPermissions.LIST));
        String sas = cloudBlockBlob.generateSharedAccessSignature(sasPolicy,null);
        return cloudBlockBlob.getUri()+"?"+sas;
    }
    catch(InvalidKeyException | URISyntaxException | StorageException ex) {
        throw new RuntimeException();
    }
}



/**
* azure-storage-blob
*/
private static BlockBlobURL getBlobURL(String accountName, String accountKey, String containerName, String filename) throws Exception {
    ContainerURL containerURL = null;
    BlockBlobURL blobURL = null;
    try {
        SharedKeyCredentials credential = new SharedKeyCredentials(accountName, accountKey);
        HttpPipeline pipeline = StorageURL.createPipeline(credential, new PipelineOptions());
        URL url = new URL(String.format(Locale.ROOT, "https://%s.blob.core.windows.net", accountName));
        ServiceURL serviceURL = new ServiceURL(url, pipeline);
        containerURL = serviceURL.createContainerURL(containerName);
        blobURL = containerURL.createBlockBlobURL(filename);
    } catch (Exception e) {
        throw new Exception(e.getMessage());
    }
    return blobURL;
}   

public static boolean uploadFile(String accountName, String accountKey, BlobFile file) throws Exception {
    boolean isUploaded = false;
    if (Objects.isNull(file.getFileType())) {
        throw new Exception("FileType is missing");
    }
    if (Objects.isNull(file.getFileName())) {
        throw new Exception("FileName is missing");
    }
    if (Objects.isNull(file.getContent())) {
        throw new Exception("Content is missing");
    }
    if (file.getFileSize() == 0) {
        throw new Exception("File's size is missing");
    }

    BlockBlobURL blobURL = getBlobURL(accountName, accountKey, file.getFileType(), file.getFileName());
    Single<BlockBlobUploadResponse> blobResponse = blobURL.upload(Flowable.just(ByteBuffer.wrap(file.getContent())), file.getFileSize(), null, null, null, null);
    if (blobResponse.blockingGet().statusCode() == 201) {
        isUploaded = true;
    }
    return isUploaded;
}

public static BlobFile downloadFile(String accountName, String accountKey, BlobFile file) throws Exception {
    if (Objects.isNull(file.getFileType())) {
        throw new Exception("FileType is missing");
    }
    if (Objects.isNull(file.getFileName())) {
        throw new Exception("FileName is missing");
    }
    BlockBlobURL blobURL = getBlobURL(accountName, accountKey, file.getFileType(), file.getFileName());
    Single<DownloadResponse> blobResponse = blobURL.download(null, null, false, null);
    Flowable<ByteBuffer> fByteBuffer = blobResponse.blockingGet().body(null);
    Single<ByteBuffer> bf = FlowableUtil.collectBytesInBuffer(fByteBuffer);
    file.setContent(bf.blockingGet().array());
    return file;
}

public static boolean deleteFile(String accountName, String accountKey, BlobFile file) throws Exception {
    boolean flag = false;
    if (Objects.isNull(file.getFileType())) {
        throw new Exception("FileType is missing");
    }
    if (Objects.isNull(file.getFileName())) {
        throw new Exception("FileName is missing");
    }
    BlockBlobURL blobURL = getBlobURL(accountName, accountKey, file.getFileType(), file.getFileName());
    Single<BlobDeleteResponse> blobResponse = blobURL.delete(null, null, null);
    if (blobResponse.blockingGet().statusCode() == 201) {
        flag = true;
    }
    return flag;
}   

  }

以上解决方案均具有两种实现方式,我们还比较了进行相同活动(例如上传相同文件)所花费的时间。

首选方法是使用天蓝色存储依赖项。

答案 1 :(得分:0)

您可以使用SAS作为连接字符串的凭据。连接字符串格式为(仅包含换行符,以提高可读性):

BlobEndpoint=myBlobEndpoint;
QueueEndpoint=myQueueEndpoint;
TableEndpoint=myTableEndpoint;
FileEndpoint=myFileEndpoint;
SharedAccessSignature=sasToken

azure-storage-java

请参阅Create a connection string using a shared access signature for more information and examples.

有关更多信息,您还可以参考SO主题中提到的建议。

Uploading to Azure Blob Storage with Shared Access Signatures

在此使用方法指南中,您将学习how to use the client library for Java to upload, download, and list block blobs in a container in Azure Blob storage

以下是Java code examples,用于说明如何使用com.microsoft.azure.storage.blob.CloudBlockBlobclass.的upload()

其他信息: 本主题显示了shared access signatures with the REST API的示例用法。共享访问签名允许您提供对容器和Blob,表,队列或文件的访问权限。