Hive中将源重命名为目标路径时出错

时间:2019-11-21 17:53:57

标签: hive hdfs

运行配置单元查询时收到以下错误:

  session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem.
  2019-11-21T02:15:09,041  INFO [20b62f76-993c-4d40-a79b-cd2963472a45 main] 
  session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem.
  2019-11-21T02:15:09,335 ERROR [20b62f76-993c-4d40-a79b-cd2963472a45 main] exec.Task: Failed 
  with exception java.io.IOException: rename for src path: 
  gs://ed26ecsmth/blah/date=2019-11-19/.hive-staging_hive_2019-11-21_02-01-08_899_5236683505163401284-1/-ext-10000/000000_0 to dest
  path:gs://ed26ecsmth/blah/data_date=2019-11-19/000000_0 returned false

此错误消息之前的日志快照为:

  2019-11-21T02:14:47,215  INFO [Thread-439] exec.Task: 
  2019-11-21 02:14:47,215 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 172507.11 sec
  2019-11-21T02:15:03,736  INFO [Thread-439] exec.Task: 
  2019-11-21 02:15:03,736 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 173332.98 sec
  2019-11-21T02:15:04,757  INFO [Thread-439] exec.Task: 
  MapReduce Total cumulative CPU time: 2 days 0 hours 8 minutes 52 seconds 980 msec
  2019-11-21T02:15:04,784  INFO [Thread-439] exec.Task: 
  Ended Job = job_1571354426739_2144
  2019-11-21T02:15:05,051  INFO [Thread-439] exec.FileSinkOperator: Moving tmp dir: 
  gs://ed26ecsmth/blah/data_date=2019-11-19/.hive-staging_hive_2019-11-21_02-01-08_899_5236683505163401284-1/_tmp.-ext-10000 
  to: gs://ed26ecsmth/blah/data_date=2019-11-19/.hive- 
  staging_hive_2019-11-21_02-01-08_899_5236683505163401284-1/-ext-10000
  2019-11-21T02:15:08,761  INFO [20b62f76-993c-4d40-a79b-cd2963472a45 main] ql.Driver: 
  Starting task [Stage-0:MOVE] in serial mode
  2019-11-21T02:15:08,767  INFO [main] hive.metastore: 
  Closed a connection to metastore, current connections: 0
  2019-11-21T02:15:08,767  INFO [20b62f76-993c-4d40-a79b-cd2963472a45 main] exec.Task: 
  Loading data to table DB1.TB1 partition 
  (data_date=2019-11-19) from gs://ed26ecsmth/blah/data_date=2019-11-19/.hive- 
  staging_hive_2019-11-21_02-01-08_899_5236683505163401284-1/-ext-10000
  2019-11-21T02:15:08,768  INFO [main] hive.metastore: Trying to connect to metastore with 
  URI thrift://name-testing-m:9083
  2019-11-21T02:15:08,769  INFO [main] hive.metastore: Opened a connection to metastore, 
  current connections: 1
  2019-11-21T02:15:08,772  INFO [main] hive.metastore: Connected to metastore.
  2019-11-21T02:15:08,826  INFO [20b62f76-993c-4d40-a79b-cd2963472a45 main] exec.MoveTask: 
  Partition is: {data_date=2019-11-19}
  2019-11-21T02:15:08,952  INFO [20b62f76-993c-4d40-a79b-cd2963472a45 main] common.FileUtils: 
  Creating directory if it doesn't exist: gs://ed26ecsmth/blah/data_date=2019-11-19
  2019-11-21T02:15:09,041  INFO [20b62f76-993c-4d40-a79b-cd2963472a45 main]

您知道为什么会发生此错误或建议解决该错误吗?

0 个答案:

没有答案