我正在尝试使用org.apache.flume.source.MultiportSyslogTCPSource
设置Flume syslog源。设置和配置成功,但是当flume从syslog生成事件时出现以下错误。
[INFO ] [2019-03-28 13:22:27.217] [[channel=file-channel] - CheckpointBackUpThread] [org.apache.flume.channel.file.EventQueueBackingStoreFile] - Checkpoint backup completed.
[WARN ] [2019-03-28 13:22:31.853] [NioProcessor-2]
[org.apache.flume.source.MultiportSyslogTCPSource] - Event size larger than specified event size: 2500. Consider increasing the max event size.
[INFO ] [2019-03-28 13:22:35.686] [SinkRunner-PollingRunner-DefaultSinkProcessor] [org.apache.flume.sink.LoggerSink] - Event: { headers:{flume.syslog.status=Invalid} body: 31 33 33 62 32 20 74 3C 31 33 34 3E 4D 61 72 20 133b2 t<134>Mar }
这是我的配置,如下所示:
#source,channel and sink
testagent.sources = testlog
testagent.channels = file-channel
testagent.sinks = logger-sink
#source
testagent.sources.testlog.type = org.apache.flume.source.MultiportSyslogTCPSource
testagent.sources.testlog.ports = 9002
testagent.sources.testlog.host = 127.0.0.1
#sink
testagent.sinks.logger-sink.type = logger
#channel
testagent.channels.file-channel.type = file
testagent.channels.file-channel.dataDirs = /test/data/01/
testagent.channels.file-channel.checkpointDir = /test/data/01/checkpoint
testagent.channels.file-channel.useDualCheckpoints = true
testagent.channels.file-channel.backupCheckpointDir = /test/data/01/checkpoint-backup
testagent.channels.file-channel.transactionCapacity = 10000
testagent.channels.file-channel.checkpointInterval = 20000
testagent.channels.file-channel.maxFileSize = 1072692224
testagent.channels.file-channel.minimumRequiredSpace = 524288000
testagent.channels.file-channel.capacity = 1000000
testagent.channels.file-channel.keep-alive = 3
testagent.channels.file-channel.checkpointOnClose = true
testagent.sinks.logger-sink.channel = file-channel
testagent.sources.testlog.channels = file-channel
从我的测试应用程序生成的日志应该成功显示在水槽代理日志中,并且应该在每个日志语句(行)之外生成事件。
答案 0 :(得分:0)
在配置中更新事件的大小,如下所示:
testagent.sinks.logger-sink.maxBytesToLog = 256
事件的默认大小为16个字节。