Question

我在/etc/logstash/conf.d下有4个文件。这4个文件用于读取tomcat的日志和apache的日志，用于监听端口3456上的log4j和读取CSV文件。

一切正常但是当Logstash解析我的CSV文件时，它会将CSV文件的相关数据放入tomcat的日志索引，apache的日志索引，log4j索引和CSV文件索引（就像我一样）要）。

我的CSV文件有配置文件：

input {
   file {
     path => "/exports/export4hadoop_FULL.csv"
     start_position => "beginning"
     sincedb_path => "/dev/null"
   }
}

filter {
    csv {
      columns => ["idTopic", "idContenu", "typeContenu", "titre", "date", "tag", "message", "typetag", "idGroupTmp"]
      separator => ";"
    }
    ruby {
      code => "
        b = event.get('idGroupTmp').split(',')
        arr = Array.new
        for c in b
            arr.push(c)
        end
        event.set('idGroups', arr)
      "
    }
}

output {
  elasticsearch {
    hosts => "bigdatam1:9200"
    index => "collabore_import"
  }
}

我在"collabore_index" ...

下真正指定了"elasticsearch"作为索引

Answer 1

Logstash会将配置文件合并为一个大文件。因此，根据您对其他类型日志的拥有情况，您可能不会＆＃34;分离＆＃34;正确的输出。

output {
  if ([type] == "csv") {
    elasticsearch {
      hosts => "bigdatam1:9200"
      index => "collabore_import"
    }
  }
}

并在输入中相应地设置type：

input {
   file {
     path => "/exports/export4hadoop_FULL.csv"
     start_position => "beginning"
     sincedb_path => "/dev/null"
     type => "csv"
   }
}

Logstash将所有内容放入所有Elasticsearch索引中

1 个答案: