Question

每当我尝试从文档中分配具有某些属性的document_id时，它都会在ElasticSearch上仅创建一个新文档，并将document_id作为我要使用的属性的名称。我想要的是实际检索到的文档数量，并将document_id设置为源数据库的主键。

output {
elasticsearch {
hosts => ["placeholder"]
index => "item"
document_id => "%{i.RecordId}"
doc_as_upsert => true
user => "placeholder"
password => "placeholder"
}
}

具体地说，创建一个看起来像这样的新文档；

"_index" : "item",
"_type" : "_doc",
"_id" : "%{i.RecordId}",
"_score" : 1.0,
"_source" : {

我尝试了几种不同的方法，例如将recordid复制到元数据字段中，但到目前为止没有任何效果。

非常感谢您的帮助

Answer 1

您可以使用元数据字段，如此处的文档所述：https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-fields.html

您必须在输出中分配如下内容：

output {
elasticsearch {
hosts => ["placeholder"]
index => "item"
document_id => "%{[@metadata][id]}"
doc_as_upsert => true
user => "placeholder"
password => "placeholder"
}
}

，您应该在过滤器中分配id的值，例如：

mutate {
    add_field => { "[@metadata][id]" => <your_id_var> }
}

或者，您可以将其放入过滤器中：

  fingerprint {                                                                              
     method => "SHA512"                                                                                    `        source => <YOUR SOURCE>                                                                
    target => "[@metadata][fingerprint]"                                          
  }

然后在输出中放入document_id

document_id => "%{[@metadata][fingerprint]}"

Answer 2

input {
  jdbc {
    jdbc_connection_string => "jdbc:sqlserver://placeholedr"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_driver_library => "C:\Users\Bruger\Documents\JBDC\sqljdbc_4.2\enu\jre8\sqljdbc42.jar"
    jdbc_user => "sa"
    jdbc_password => "placeholder"
    statement => "SELECT
                  i.RecordId,
                  i.FK_Catalog,
                  i.FK_Unit,
                  c.Name,
                  i.ItemNumber,
                  i.Ean,
                  i.ItemNumber2,
                  i.Description,
                  i.AltDescription,
                  i.IsStandard,
                  i.MarkedForDelete,
                  i.DateCreated,
                  i.DateUpdated,
                  i.ItemNumberNumerical,
                  i.ItemAlias,
                  i.FK_CatalogItemGroup,
                  f.UnitName,
                  i.NetPrice,
                  i.GrossPrice,
                  i.SellingPrice,
                  i.IsSellingPriceCalculated,
                  i.PriceMatrixItemId,
                  i.DiscountPercent,
                  i.DiscountAmount,
                  i.IsActive,
                  i.ItemOnStock,
                  i.AllowanceCharge,
                  w.Name as WholesalerName,
                  w.WholesalerID,
                  wp.Priority
                  from Item as i
                  JOIN Catalog as c on c.RecordId = i.FK_Catalog
                  join Wholesalers w on w.WholesalerID = c.FK_Wholesaler
                  join WholesalerPriority wp on wp.fk_wholesaler = w.WholesalerID
                  JOIN Units f on f.UnitsID = i.FK_Unit 
                  "
  }

}

filter
{
    mutate
        {
            add_field => {"[@metadata][id]" => "[recdordid]"     }      
        }
}



output { 
elasticsearch {
    hosts => [placeholder]
    index => "oegaard"
    doc_as_upsert => true
    document_id => "%{[@metadata][id]}"
    user => "placeholedr"
    password => "placeholder"
  }
}

使用外部属性作为document_id会返回属性的名称，即document_id

2 个答案: