删除Amazon S3上的文件夹,同时创建外部表

时间:2013-07-10 08:07:03

标签: hadoop amazon-s3 hive

我们的S3 Bucket上的行为非常不寻常,这种行为并不一致。所以,我们无法确定问题所在。现在来讨论这个问题我发了一个查询(创建外部表)。这导致删除我在外部表中指向的文件夹。这已经发生了3-4次。那么,请你解释一下这种行为。为方便起见,我附加了外部表查询和正在S3存储桶上执行操作的日志。

查询:

create table apr_2(date_local string, time_local string,s_computername string,c_ip string,s_ip string,s_port string,s_sitename string, referer string, localfile string, TimeTakenMS string, status string, w3status string, sc_substatus string, uri string, qs string, sc_bytes string, cs_bytes string, cs_username string, cs_User_Agent string, s_proxy string, c_protocol string, cs_version string, cs_method string, cs_Cookie string, cs_Host string, w3wpbytes string, RequestsPerSecond string, CPU_Utilization string, BeginRequest_UTC string, EndRequest_UTC string, time string, logdate string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' location 's3://logs/apr_2_com'

日志:

 REST.DELETE.OBJECT logs/apr_2_com/000002.tar.gz 
 REST.DELETE.OBJECT logs/apr_2_com/000001.tar.gz 

1 个答案:

答案 0 :(得分:0)

尝试使用此语法 -

    create external table if not exists apr_2(date_local string, time_local string,s_computername string,c_ip string,s_ip string,s_port string,s_sitename string, referer string, localfile string, TimeTakenMS string, status string, w3status string, sc_substatus string, uri string, qs string, sc_bytes string, cs_bytes string, cs_username string, cs_User_Agent string, s_proxy string, c_protocol string, cs_version string, cs_method string, cs_Cookie string, cs_Host string, w3wpbytes string, RequestsPerSecond string, CPU_Utilization string, BeginRequest_UTC string, EndRequest_UTC string, time string, logdate string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' location 's3://logs/apr_2_com'