Question

我正在尝试使用雪球将所有数据导入到s3，但我需要跟踪每个文件的一些元数据。我有一些c＃代码使用这个元数据将文件直接上传到存储桶，这很好。我以为我可以使用雪球s3适配器连接到我的雪球并从我的c＃代码上传数据，因为this link说：

您可以将S3适配器与现有的Amazon S3接口一起使用 AWS软件开发工具包，AWS CLI或您自己的自定义Amazon S3 REST 客户端。

我在http://localhost:8090上运行了适配器，所以我尝试通过将该端点作为ServiceUrl传入AmazonS3Config对象，将其设置为Amazon3Client的端点，但是当我尝试使用任一端点上传文件时PutObjectRequest或TransferUtility，我得到以下异常：

 Amazon.Runtime.AmazonServiceException: A WebException with status
 NameResolutionFailure was thrown. ---> System.Net.WebException: The
 remote name could not be resolved: 'BUCKET.localhost' at
 System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
 at System.Net.HttpWebRequest.GetRequestStream() at
 Amazon.Runtime.Internal.HttpRequest.GetRequestContent() at
 Amazon.Runtime.Internal.HttpHandler`1.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.RedirectHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.Unmarshaller.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.ErrorHandler.InvokeSync(IExecutionContext
 executionContext)
 --- End of inner exception stack trace --- at Amazon.Runtime.Internal.WebExceptionHandler.HandleException(IExecutionContext
 executionContext, WebException exception) at
 Amazon.Runtime.Internal.ExceptionHandler`1.Handle(IExecutionContext
 executionContext, Exception exception) at
 Amazon.Runtime.Internal.ErrorHandler.ProcessException(IExecutionContext
 executionContext, Exception exception) at
 Amazon.Runtime.Internal.ErrorHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.Signer.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.CredentialsRetriever.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.RetryHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.S3.Internal.AmazonS3KmsHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.EndpointResolver.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.S3.Internal.AmazonS3PostMarshallHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.Marshaller.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.S3.Internal.AmazonS3PreMarshallHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.MetricsHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.RuntimePipeline.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.AmazonServiceClient.Invoke(TRequest request,
 IMarshaller`2 marshaller, ResponseUnmarshaller unmarshaller) at
 Amazon.S3.AmazonS3Client.PutObject(PutObjectRequest request)

我尝试使用我的本地IP地址，以及127.0.0.1，但都得到了同样的错误。当我使用亚马逊cli将文档上传到雪球时，使用--endpoint来更改端点，它可以工作（使用localhost或我的本地ip），所以显然它只是关于.net sdk的东西。但我真的非常希望使用我现有的c＃代码，而不是编写一个命令行程序来实现这一点 - 从cli开始就不会那么简单了。

编辑：以下是无效的c＃代码：

var config = new AmazonS3Config
{
  ServiceURL = "http://localhost:8090"
};

var file = new FileInfo(FILENAME);

var pRequest = new PutObjectRequest
{
  BucketName = BUCKET,
  Key = S3FILENAME,
  StorageClass = S3StorageClass.StandardInfrequentAccess,
  FilePath = file.FullName
};

pRequest.Metadata.Add("x-amz-meta-local-date-modified", file.CreationTime.ToFileTimeUtc().ToString());

using (var client = new AmazonS3Client(config))
{
  client.PutObject(pRequest);
}

这是有效的命令行语句：

aws s3 cp FILENAME s3://BUCKET/S3FILENAME --endpoint http://localhost:8090

任何人都可以帮助我吗？

由于

Answer 1

问题在于，默认情况下，AWS开发工具包会将您的BucketName附加到ServiceURL上，因此您最终得到的URL无效：http://BUCKET.localhost:8090。

您需要使用ForcePathStyle参数，以防止将BucketName附加到ServiceURL：

var config = new AmazonS3Config
{
    ServiceURL = "http://localhost:8090",
    ForcePathStyle = true
};

使用c＃将数据传输到aws s3雪球上

1 个答案: