我正在尝试通过水槽来传输mysql数据。
然而,我收到此错误:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat cannot be cast to org.apache.hadoop.hive.ql.io.AcidOutputFormat
我的Hive表格具有ORC格式并且已被删除。
为什么会出现此错误?请问我能指点一下吗?
代理配置如下所示:
tier1.channels.ch1.type = memory
tier1.sources.sqlSource.channels = ch1
tier1.channels = ch1
tier1.channels.ch1.type = memory
tier1.channels.ch1.capacity = 1000000
tier1.channels.ch1.transactionCapacity = 1500
tier1.sinks = Hive
tier1.sources = sqlSource
tier1.sources.sql-source.type = org.keedio.flume.source.SQLSource
tier1.sources.sqlSource.hibernate.connection.url = jdbc:mysql://IP/DB
tier1.sources.sql-source.connection.url = jdbc:mysql://IP/DB
tier1.sources.sql-source.user = USER
tier1.sources.sql-source.password = PASS
tier1.sources.sql-source.table = table
tier1.sources.sql-source.database = db
#tier1.sources.sql-source.columns.to.select = *
tier1.sources.sql-source.incremental.column.name = entity_id
tier1.sources.sql-source.incremental.value = 0
tier1.sources.sql-source.run.query.delay=10000
tier1.sources.sql-source.status.file.path = /var/lib/flume
tier1.sources.sql-source.status.file.name = sql-source.status
tier1.sinks.HDFS.channel = ch1
tier1.sinks.HDFS.type = hdfs
tier1.sinks.HDFS.hdfs.path = hdfs://IP:8020/flume/mysql3
tier1.sinks.HDFS.hdfs.file.Type = DataStream
tier1.sinks.Hive.type = hive
tier1.sinks.Hive.channel = ch1
tier1.sinks.Hive.hive.metastore = thrift://IPl:9083
tier1.sinks.Hive.hive.database = magentolive
tier1.sinks.Hive.hive.table = sales_flat_order_6
#tier1.sinks.Hive.hive.partition = state
tier1.sinks.Hive.round = true
tier1.sinks.Hive.roundValue = 10
tier1.sinks.Hive.roundUnit = minute
tier1.sinks.Hive.serializer = DELIMITED
tier1.sinks.Hive.serializer.delimiter = ","
tier1.sinks.Hive.serializer.serdeSeparator = ','
tier1.sinks.Hive.serializer.fieldnames=list_of_field_names_here
########
tier1.sources.sqlSource.hibernate.connection.url = jdbc:mysql://IP/DB
tier1.sources.sqlSource.hibernate.connection.user = username
tier1.sources.sqlSource.hibernate.connection.password = password
tier1.sources.sqlSource.table = table_name
tier1.sources.sqlSource.type = org.keedio.flume.source.SQLSource
tier1.sources.sqlSource.channels = ch1
# Columns to import to kafka (default * import entire row)
#tier1.sources.sqlSource.columns.to.select = *
# Query delay, each configured milisecond the query will be sent
tier1.sources.sqlSource.run.query.delay=10000
# Status file is used to save last readed row
tier1.sources.sqlSource.status.file.path = /var/log/flume-ng/
tier1.sources.sqlSource.status.file.name = sqlSource.status