我正尝试将我们的日志发送给两个提供商,分别用于Elasticsearch和Cloudwatch,并在记录转换中,我要添加要在Cloud Watch和Elastic Search上显示的文件,但我需要省略一些字段弹性搜索
<source>
@type tail
path /var/log/employee/
pos_file /var/log/tasks1.pos
tag foo.*
format none
time_format %Y-%m-%dT%H:%M:%S
time_key time
read_from_head true
</source>
<filter foo.**>
@type concat
key message
multiline_end_regexp /Task exited with return code [0-9]{1,3}/
</filter>
<filter foo.**>
@type record_transformer
enable_ruby true
#reform.example-test2-runme-1-a8c779c5-j4hs7_common-names-space_example-test2-runme-1-a8c779c5-601707f691957d29cc8779c7801ca353131fd96b926ac95a23b2487016e71511.log
<record>
env dev
employee ${tag_parts[4]}
age ${tag_parts[5]}
id ${tag_parts[6]}
salary ${tag_parts[7]}
benefits ${tag_parts[8]}
log_group tes1/test2/${tag_parts[4]}
log_stream ${tag_parts[5]}/${tag_parts[6]}/${tag_parts[7].gsub!(/[^a-z0-9\-T]/,'.')}-${tag_parts[8]}.log
</record>
</filter>
<match airflow.**>
@type copy
<store>
@type elasticsearch
host xxxxxxxx
port xxx
index_name testagain2
<buffer>
flush_interval 10s
</buffer>
</store>
<store>
@type cloudwatch_logs
log_group_name_key log_group
log_stream_name_key alog_stream
auto_create_stream true
</store>
</match>
我想要实现的是在将日志发送到弹性搜索时刷新log_group_name_key和log_stream_name_key进行弹性搜索