Flume Hive Sink错误

时间:2017-12-20 07:48:33

标签: hadoop hive hdfs flume

我正在为假脱机目录生成数据,并使用flume hive sink将其重定向到hive表。 Flume sink与hive Metastore相连,但之后我面临以下问题。

问题

  

:无法发送活动。 org.apache.flume.EventDeliveryException之后出现异常:Java.lang.ArrayIndexOutOfBoundsException:1

Flume.conf

flume-hive-ingest.sources = src1
flume-hive-ingest.channels = chan1
flume-hive-ingest.sinks = sink1

flume-hive-ingest.sources.src1.type = spooldir
flume-hive-ingest.sources.src1.channels = chan1
flume-hive-ingest.sources.src1.spoolDir = 
/vagrant/flume_log

flume-hive-ingest.channels.chan1.type = memory
flume-hive-ingest.channels.chan1.capacity = 1000
flume-hive-ingest.channels.chan1.transactionCapacity = 
1000

flume-hive-ingest.sinks.sink1.type = hive
flume-hive-ingest.sinks.sink1.channel = chan1

flume-hive-ingest.sinks.sink1.hive.metastore = 
thirft ://one.hdp:9083
flume-hive-ingest.sinks.sink1.hive.database = default
flume-hive-ingest.sinks.sink1.hive.table = stocks

flume-hive-ingest.sinks.sink1.serializer = delimited
flume-hive-ingest.sinks.sink1.serializer.delimiter = ,
flume-hive-ingest.sinks.sink1.serializer.fieldnames = 
date ,open,high,low,close,volume,adj_close

Hive脚本

DROP TABLE IF EXISTS stocks;
CREATE EXTERNAL TABLE stocks (
date STRING,
open DOUBLE,
high DOUBLE,
low DOUBLE,
close DOUBLE,
volume BIGINT,
adj_close DOUBLE)
STORED AS ORC
LOCATION '/ingest/stocks';

0 个答案:

没有答案