Question

我有一个蜂巢格式为ORC的表。下面是create语句

  CREATE EXTERNAL TABLE IF NOT EXISTS Mydb.test(    
  transkey string,
  acckey string,
  name string,
  sex string,
  city string,
  gentime timestamp
  )
ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
LOCATION '/data/gau/work/hive/test';

现在我希望将2个额外的字段添加到此表中，因此我更改了表并添加了2个额外的字段。

更改表Mydb.test添加列（位置字符串，age int）。

因此，更改表后，我可以按预期将旧数据中新添加的列的字段值显示为NULL。例如，如果我触发此查询从Mydb.test限制5中选择transkey，gentime，location，age。

  +-----------------------------------+------------------------+-----------------+-----------------+--+
|          transkey                 |         gentime        |      location   | age              |
+-----------------------------------+------------------------+-----------------+-----------------+--+
| asxxxxx                           | 2019-07-xx 04:24:19.0  | india           | 25               |
| rtyxxxxxxx                        | 2019-07-04 04:24:19.0  | afg             | 45               |
| uuinxxxxxx                        | 2019-xx-04 04:24:20.0  | test            | 67               |
| ghjkxxxxxxx                       | 2017-xx-06 09:01:14.0  | NULL            | NULL             |
| uuuixxxxxxx                       | 2017-xx-06 09:01:xxx   | NULL            | NULL             |
| xxxxyyxxx                         | 2017-xx-06 09:01:1xxxx | NULL            | NULL             |

但是当我从旧数据中获取特定的transkey时（我可以通过查看记录的gentime知道数据是旧的还是新的）并触发查询就像从Mydb.test中选择年龄，位置一样，其中transkey ='xxxxyyxxx'（根据gentime xxxxyyxxx是将两个新添加的字段都设置为NULL的旧数据），错误如下：

java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row 
    at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row 
    at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:52)
    at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMa

pper.java:170)
    ... 8 more
Caused by: java.lang.NullPointerException
    at java.lang.System.arraycopy(Native Method)
    at org.apache.hadoop.io.Text.set(Text.java:225)
    at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$6.writeValue(VectorExpressionWriterFactory.java:686)
    at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterBytes.writeValue(VectorExpressionWriterFactory.java:272)

在触发上述查询后，理想情况下，我应该为这些列获取NULL值，但我却收到此错误。请帮忙。

P.S-所有这些查询对于具有这两个新列的新数据都可以正常工作

从Hive表中选择行时遇到“ org.apache.hadoop.hive.ql.metadata.HiveException：处理行时Hive运行时错误”的问题

0 个答案: