org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat无法强制转换为org.apache.hadoop.hive.ql.io.AcidOutputFormat

时间:2017-05-25 10:15:54

标签: mysql hive flume flume-ng

我正在尝试通过水槽来传输mysql数据。

然而,我收到此错误:

org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat cannot be cast to org.apache.hadoop.hive.ql.io.AcidOutputFormat

我的Hive表格具有ORC格式并且已被删除。

为什么会出现此错误?请问我能指点一下吗?

代理配置如下所示:

tier1.channels.ch1.type = memory 

tier1.sources.sqlSource.channels = ch1 

tier1.channels = ch1 

tier1.channels.ch1.type = memory
tier1.channels.ch1.capacity = 1000000
tier1.channels.ch1.transactionCapacity = 1500

tier1.sinks = Hive 



tier1.sources = sqlSource 

tier1.sources.sql-source.type = org.keedio.flume.source.SQLSource 

tier1.sources.sqlSource.hibernate.connection.url =  jdbc:mysql://IP/DB

tier1.sources.sql-source.connection.url = jdbc:mysql://IP/DB 



tier1.sources.sql-source.user = USER  

tier1.sources.sql-source.password = PASS

tier1.sources.sql-source.table = table 

tier1.sources.sql-source.database = db 



#tier1.sources.sql-source.columns.to.select = *  



tier1.sources.sql-source.incremental.column.name = entity_id  

tier1.sources.sql-source.incremental.value = 0  



tier1.sources.sql-source.run.query.delay=10000 



tier1.sources.sql-source.status.file.path = /var/lib/flume 

tier1.sources.sql-source.status.file.name = sql-source.status 



tier1.sinks.HDFS.channel = ch1 

tier1.sinks.HDFS.type = hdfs 

tier1.sinks.HDFS.hdfs.path = hdfs://IP:8020/flume/mysql3

tier1.sinks.HDFS.hdfs.file.Type = DataStream

tier1.sinks.Hive.type = hive
tier1.sinks.Hive.channel = ch1
tier1.sinks.Hive.hive.metastore = thrift://IPl:9083
tier1.sinks.Hive.hive.database = magentolive
tier1.sinks.Hive.hive.table = sales_flat_order_6
#tier1.sinks.Hive.hive.partition = state
tier1.sinks.Hive.round = true
tier1.sinks.Hive.roundValue = 10
tier1.sinks.Hive.roundUnit = minute
tier1.sinks.Hive.serializer = DELIMITED
tier1.sinks.Hive.serializer.delimiter = ","
tier1.sinks.Hive.serializer.serdeSeparator = ','
tier1.sinks.Hive.serializer.fieldnames=list_of_field_names_here
########
tier1.sources.sqlSource.hibernate.connection.url =  jdbc:mysql://IP/DB

tier1.sources.sqlSource.hibernate.connection.user = username 

tier1.sources.sqlSource.hibernate.connection.password = password 



tier1.sources.sqlSource.table = table_name

tier1.sources.sqlSource.type = org.keedio.flume.source.SQLSource 

tier1.sources.sqlSource.channels = ch1



# Columns to import to kafka (default * import entire row) 

#tier1.sources.sqlSource.columns.to.select = * 



# Query delay, each configured milisecond the query will be sent 

tier1.sources.sqlSource.run.query.delay=10000 



# Status file is used to save last readed row 

tier1.sources.sqlSource.status.file.path = /var/log/flume-ng/ 
tier1.sources.sqlSource.status.file.name = sqlSource.status

0 个答案:

没有答案