Question

我想将Kafka消息写入MySQL数据库。 this链接中有一个示例。在该示例中，apache flume用于消费消息并将其写入MySQL。我正在使用相同的代码，当我运行flume-ng agent时，event始终变为null

我的flume.conf.properties文件是：

agent.sources=kafkaSrc
agent.channels=channel1
agent.sinks=jdbcSink

agent.channels.channel1.type=org.apache.flume.channel.kafka.KafkaChannel
agent.channels.channel1.brokerList=localhost:9092
agent.channels.channel1.topic=kafkachannel
agent.channels.channel1.zookeeperConnect=localhost:2181
agent.channels.channel1.capacity=10000
agent.channels.channel1.transactionCapacity=1000
agent.channels.channel1.parseAsFlumeEvent=false


agent.sources.kafkaSrc.type = org.apache.flume.source.kafka.KafkaSource
agent.sources.kafkaSrc.channels = channel1
agent.sources.kafkaSrc.zookeeperConnect = localhost:2181
agent.sources.kafkaSrc.topic = kafka-mysql

agent.sinks.jdbcSink.type = com.stratio.ingestion.sink.jdbc.JDBCSink
agent.sinks.jdbcSink.connectionString = jdbc:mysql://127.0.0.1:3306/test?useSSL=false
agent.sinks.jdbcSink.username=root
agent.sinks.jdbcSink.password=pass
agent.sinks.jdbcSink.batchSize = 10
agent.sinks.jdbcSink.channel =channel1
agent.sinks.jdbcSink.sqlDialect=MYSQL
agent.sinks.jdbcSink.driver=com.mysql.jdbc.Driver
agent.sinks.jdbcSink.sql=INSERT INTO kafkamsg(msg) VALUES(${body:varchar})

我哪里错了？

感谢。

Answer 1

在我的例子中，flume听了kafka的ary.sort { |a, b| [b[:count], a[:char]] <=> [a[:count], b[:char]] } # ^ ^ # | | # +-------------------------+主题。但此代码适用于ary.sort { |a, b| [a[:count], b[:char]] <=> [b[:count], a[:char]] } # ^ ^ # | | # +-------------------------+主题。所以我们需要向kafka-mysql主题发送消息，我不知道为什么。

将Kafka消息流式传输到MySQL数据库

1 个答案: