通过传递自定义对象列表来创建弹性索引

时间:2017-09-05 06:04:00

标签: elasticsearch jest

通过传递自定义对象列表创建弹性搜索索引时遇到问题。以下是我根据某些条件填充的bean -

public class DataBean {
private String request_id;
private String clicks;
private String impressions;
private String conversion;

public String getRequest_id() {
    return request_id;
}

public void setRequest_id(String request_id) {
    this.request_id = request_id;
}

public String getClicks() {
    return clicks;
}

public void setClicks(String clicks) {
    this.clicks = clicks;
}

public String getImpressions() {
    return impressions;
}

public void setImpressions(String impressions) {
    this.impressions = impressions;
}

public String getConversion() {
    return conversion;
}

public void setConversion(String conversion) {
    this.conversion = conversion;
}

@Override
    public String toString() {
        return "DataBean [ clicks=" + clicks + ", impressions=" + impressions + ", conversion=" + conversion + ", request_id=" + request_id + "]";
    }
}

以下是代码的要点,它循环遍历记录并填充上面的bean并将其存储在用于创建索引的列表中 -

....
if (result.isSucceeded()) {
final List<DataBean> dataBeanList = new ArrayList<>();
final List<SearchResult.Hit<DataBean, Object>> hits = result.getHits(DataBean.class, Object.class);

for (SearchResult.Hit<DataBean, Object> hit : hits) {
    if (!dataBeanList.stream().anyMatch(
            dataBean -> Objects.equals(dataBean.getRequest_id(), hit.source.getRequest_id()))) {
        DataBean source = hit.source;
        dataBeanList.add(source);
    }
}

MetricAggregation aggregations = result.getAggregations();
TermsAggregation termsAggregation = aggregations.getTermsAggregation("request_id");
for (TermsAggregation.Entry entry : termsAggregation.getBuckets()) {
    for (DataBean dataBean : dataBeanList) {
        if (entry.getKey().equalsIgnoreCase(dataBean.getRequest_id())) {

            Double impression = entry.getSumAggregation("imp").getSum();
            Double click = entry.getSumAggregation("click").getSum();
            Double action = entry.getSumAggregation("action").getSum();
            dataBean.setImpressions(impression.toString());
            dataBean.setClicks(click.toString());
            dataBean.setConversion(action.toString());
            break;
        }
    }
}

Index index = new Index.Builder(new Gson().toJson(dataBeanList)).index(destinationIndex).type("hour")
        .build();

try {
    client.execute(index);
} catch (IOException ioException) {
    LOGGER.debug("Failed to execute the search request on indices");
    throw new JestClientExecutionException("Failed to execute the search request on indices - ",
            ioException);
}
}

以下是索引模板文件 -

{
  "order": 0,
  "template": "hour_*",
  "settings": {
  },
  "mappings": {
    "hour": {
      "properties": {
        "conversion": {
          "index": "not_analyzed",
          "store": true,
          "type": "integer"
        },
        "impressions": {
          "index": "not_analyzed",
          "store": true,
          "type": "integer"
        },
        "clicks": {
          "index": "not_analyzed",
          "store": true,
          "type": "integer"
        },
        "request_id": {
          "analyzer": "keyword",
          "index": "not_analyzed",
          "store": true,
          "type": "text"
        }
      }
    }
  },
  "aliases": {}
}

现在,当我尝试执行此代码时,它会出现以下错误 -

   [2017-09-04T06:10:29,365][INFO ][o.e.c.m.MetaDataCreateIndexService] [node-1] [hour_11_2017-09-04] creating index, cause [auto(bulk api)], templates [hour], shards [5]/[1], mappings [hour]
[2017-09-04T06:10:30,622][DEBUG][o.e.a.b.TransportShardBulkAction] [node-1] [hour_11_2017-09-04][1] failed to execute bulk item (index) BulkShardRequest [[hour_11_2017-09-04][1]] containing [index {[hour_11_2017-09-04][hour][AV5LgwJct35uj5JLu_X5], source[_na_]}]
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
        at org.elasticsearch.index.mapper.DocumentParser.wrapInMapperParsingException(DocumentParser.java:176) ~[elasticsearch-5.5.1.jar:5.5.1]
        at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:69) ~[elasticsearch-5.5.1.jar:5.5.1]
        at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:277) ~[elasticsearch-5.5.1.jar:5.5.1]
        at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:529) ~[elasticsearch-5.5.1.jar:5.5.1]
        at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:506) ~[elasticsearch-5.5.1.jar:5.5.1]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.prepareIndexOperationOnPrimary(TransportShardBulkAction.java:450) ~[elasticsearch-5.5.1.jar:5.5.1]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:458) ~[elasticsearch-5.5.1.jar:5.5.1]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:143) [elasticsearch-5.5.1.jar:5.5.1]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:113) [elasticsearch-5.5.1.jar:5.5.1]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:69) [elasticsearch-5.5.1.jar:5.5.1]

我尝试用地图替换对象列表,以确定列表是否有问题。我观察到地图有效但列表没有。所以问题在于对象列表,但我无法弄清楚如何解决这个问题。如果有人有任何见解,请告诉我。

0 个答案:

没有答案