Question

我正在尝试使用Hadoop API连接到Amazon S3存储桶。我正在使用org.apache.hadoop.fs.FileSystem类来连接到存储桶并创建一个新文件。

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.*;
import java.net.URI;
public class FileMover {

public static void main(String [] args) {

    URI s3uri=URI.create(“s3a://mybucket”);
    Configuration conf = new Configuration();
    FileSystem s3fs=FileSystem.get(s3uri,conf);
    s3fs.create(new Path(“s3a://mybucket/MyFile”));
 }

}

但这会产生以下错误

org.apache.hadoop.fs.s3a.AWSS3IOException: getFileStatus on s3a://mybucket/Jesus/test: com.amazonaws.services.s3.model.AmazonS3Exception: 
The bucket is in this region: eu-west-3. 
Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: 3D3A4C0CE06CCF8B), S3 Extended Request ID: rDKHmonuOeVi2PhwTZ3ByLD3LGBZ+kbTvqwZ8/hfUAs8y9ei88HosVSuOqdGJa8VtjvOXZ8/DiE=: 
The bucket is in this region: eu-west-3. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: 3D3A4C0CE06CCF8B)

这是我在core-site.xml中设置的配置

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://quickstart.cloudera:8020</value>
  </property>



  <property>
    <name>fs.s3a.access.key</name>
    <value>XXXXXXXXXXXX</value>
  </property>

  <property>
    <name>fs.s3a.secret.key</name>
    <value>XXXXXXXXXXXX</value>
  </property>


  <!-- OOZIE proxy user setting -->
  <property>
    <name>hadoop.proxyuser.oozie.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.oozie.groups</name>
    <value>*</value>
  </property>

  <!-- HTTPFS proxy user setting -->
  <property>
    <name>hadoop.proxyuser.httpfs.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.httpfs.groups</name>
    <value>*</value>
  </property>

  <!-- Llama proxy user setting -->
  <property>
    <name>hadoop.proxyuser.llama.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.llama.groups</name>
    <value>*</value>
  </property>

  <!-- Hue proxy user setting -->
  <property>
    <name>hadoop.proxyuser.hue.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>*</value>
  </property>

</configuration>

我试图将core-site.xml中的端点设置为错误中请求的端点，但是它不起作用。

尝试使用Hadoop FileSystem连接到S3存储桶，但跳出一个错误，指出存储桶在此区域中

0 个答案: