我正在使用Python在本地ElasticSearch(localhost:9200)中添加条目
目前,我使用此方法:
def insertintoes(data):
"""
Insert data into ElasicSearch
:param data: dict
:return:
"""
timestamp = data.get('@timestamp')
logstashIndex = 'logstash-' + timestamp.strftime("%Y.%m.%d")
es = Elasticsearch()
if not es.indices.exists(logstashIndex):
# Setting mappings for index
mapping = '''
{
"mappings": {
"_default_": {
"_all": {
"enabled": true,
"norms": false
},
"dynamic_templates": [
{
"message_field": {
"path_match": "message",
"match_mapping_type": "string",
"mapping": {
"norms": false,
"type": "text"
}
}
},
{
"string_fields": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"fields": {
"keyword": {
"type": "keyword"
}
},
"norms": false,
"type": "text"
}
}
}
],
"properties": {
"@timestamp": {
"type": "date",
"include_in_all": true
},
"@version": {
"type": "keyword",
"include_in_all": true
}
}
}
}
}
'''
es.indices.create(logstashIndex, ignore=400, body=mapping)
es.index(index=logstashIndex, doc_type='system', timestamp=timestamp, body=data)
data
是一个dict
结构,其有效@timestamp
定义如下data['@timestamp'] = datetime.datetime.now()
问题是,即使我的数据中存在时间戳值,Kibana也不会在“发现”字段中显示该条目。 :(
以下是ElasicSearch中完整条目的示例:
{
"_index": "logstash-2017.06.25",
"_type": "system",
"_id": "AVzf3QX3iazKBndbIkg4",
"_score": 1,
"_source": {
"priority": 6,
"uid": 0,
"gid": 0,
"systemd_slice": "system.slice",
"cap_effective": "1fffffffff",
"exe": "/usr/bin/bash",
"hostname": "ns3003395",
"syslog_facility": 9,
"comm": "crond",
"systemd_cgroup": "/system.slice/cronie.service",
"systemd_unit": "cronie.service",
"syslog_identifier": "CROND",
"message": "(root) CMD (/usr/local/rtm/bin/rtm 14 > /dev/null 2> /dev/null)",
"systemd_invocation_id": "9228b6c72e6a4624a1806e4c59af8d04",
"syslog_pid": 26652,
"pid": 26652,
"@timestamp": "2017-06-25T17:27:01.734453"
}
}
正如您所看到的,有一个@timestamp
字段,但它似乎不是Kibana所期望的。
并且不知道如何使我的条目在Kibana中可见。
有什么想法吗?
答案 0 :(得分:3)
Elasticsearch并未将@timestamp识别为日期,而是将其视为字符串。如果您的数据['@ timestamp']是一个日期时间对象,您可以尝试将其转换为自动识别的ISO字符串,尝试:
timestamp = data.get('@timestamp').isoformat()
时间戳现在应该是一个字符串,但是采用ISO格式