Question

我正在使用NIFI PutHiveStreaming处理器将记录写入HDFS，我一直在遇到这个问题。我无法做太多，因为我似乎已经遵守了所有的配置要求。来自某人的任何指针都已成功解决了这个问题？（Nifi - 1.4.0和Hive - 2.3.3）


2018-04-18 09:03:49,997 INFO [Timer-Driven Process Thread-5] hive.metastore Trying to connect to metastore with URI thrift://hive-metastore:9083
2018-04-18 09:03:49,999 INFO [Timer-Driven Process Thread-5] hive.metastore Connected to metastore.
2018-04-18 09:03:50,486 WARN [Timer-Driven Process Thread-5] o.a.h.h.m.RetryingMetaStoreClient MetaStoreClient lost connection. Attempting to reconnect.
org.apache.thrift.transport.TTransportException: null
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
2018-04-18 09:03:51,487 INFO [Timer-Driven Process Thread-5] hive.metastore Trying to connect to metastore with URI thrift://hive-metastore:9083
2018-04-18 09:03:51,491 INFO [Timer-Driven Process Thread-5] hive.metastore Connected to metastore.
2018-04-18 09:03:51,505 ERROR [Timer-Driven Process Thread-5] o.a.n.processors.hive.PutHiveStreaming PutHiveStreaming[id=1b8f6a4a-e456-3c2b-74be-bc9a0927a43b] Failed to create HiveWriter for endpoint: {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }: org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
    at org.apache.nifi.util.hive.HiveWriter.(HiveWriter.java:79)
    at org.apache.nifi.util.hive.HiveUtils.makeHiveWriter(HiveUtils.java:46)
    at org.apache.nifi.processors.hive.PutHiveStreaming.makeHiveWriter(PutHiveStreaming.java:968)

Caused by: org.apache.nifi.util.hive.HiveWriter$TxnBatchFailure: Failed acquiring Transaction Batch from EndPoint: {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
    at org.apache.nifi.util.hive.HiveWriter.nextTxnBatch(HiveWriter.java:264)
    at org.apache.nifi.util.hive.HiveWriter.(HiveWriter.java:73)
    ... 24 common frames omitted

Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable to acquire lock on {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
    at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:578)
    at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransaction(HiveEndPoint.java:547)
    at org.apache.nifi.util.hive.HiveWriter.nextTxnBatch(HiveWriter.java:261)
    ... 25 common frames omitted

Caused by: org.apache.thrift.transport.TTransportException: null
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)

我也在hive-site.xml中设置了并发和txn参数。

Apache NIFI PutHiveStreaming问题

0 个答案: