按数字和日期字段排序时的Elasticsearch NumberFormatException

时间:2014-07-09 16:21:35

标签: elasticsearch

我正在尝试对两个字段进行排序,一个是日期字段,另一个是整数字段。我已经验证字段是否已正确映射,并且所有索引文档的值都具有正确的数据类型,但每次我尝试按数字字段排序时,我都会返回异常[java.lang.NumberFormatException:无效的移位prefixCoded字节中的值(编码值实际上是INT?)]。

奇怪的是它返回结果并且它们似乎正确排序所以我不完全确定它有什么影响,但我知道我不应该在响应中有任何错误。

这是一个映射:

{
   "blcdt.products": {
      "mappings": {
         "review_143758": {
            "properties": {
               "date_reviewed": {
                  "type": "date",
                  "format": "YYYY-MM-dd"
               },
               "description": {
                  "type": "string"
               },
               "manufacturer": {
                  "type": "string",
                  "index": "not_analyzed"
               },
               "model_number": {
                  "type": "string",
                  "boost": 2,
                  "index": "not_analyzed",
                  "norms": {
                     "enabled": true
                  }
               },
               "msrp": {
                  "type": "float"
               },
               "product": {
                  "type": "string"
               },
               "product_name": {
                  "type": "string",
                  "boost": 2,
                  "fields": {
                     "full": {
                        "type": "string",
                        "boost": 4,
                        "index": "not_analyzed",
                        "norms": {
                           "enabled": true
                        }
                     }
                  }
               },
               "product_type": {
                  "type": "string",
                  "index": "not_analyzed"
               },
               "review_category": {
                  "type": "string",
                  "index": "not_analyzed"
               },
               "review_title": {
                  "type": "string"
               },
               "review_type": {
                  "type": "string",
                  "index": "not_analyzed"
               },
               "review_type_priority": {
                  "type": "integer"
               },
               "score": {
                  "type": "float"
               },

            }
         }
      }
   }
}

这是一个抛出异常的查询:

POST /blcdt.products/review_143758/_search?pretty=true
{
   "query": {
      "filtered": {
         "filter": [
            {
               "term": {
                  "review_category": "Game Reviews"
               }
            }
         ]
      }
   },
   "from": 0,
   "size": 8,
   "sort": [
      {
         "review_type_priority": {
            "order": "desc"
         }
      },
      {
         "date_reviewed": {
            "order": "desc"
         }
      }
   ],
   "fields": [
      "review_title",
      "review_type",
      "score"
   ]
}

以下是ES的简短回复:

{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 1,
      "failed": 4,
      "failures": [
         {
            "index": "blcdt.products",
            "shard": 2,
            "status": 500,
            "reason": "QueryPhaseExecutionException[[blcdt.products][2]: query[filtered(ConstantScore(cache(review_category:Game Reviews)))->cache(_type:review_143758)],from[0],size[8],sort[<custom:\"review_type_priority\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@44ecbc5>!,<custom:\"date_reviewed\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@5123f4b8>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; "
         },
         {
            "index": "blcdt.products",
            "shard": 3,
            "status": 500,
            "reason": "QueryPhaseExecutionException[[blcdt.products][3]: query[filtered(ConstantScore(cache(review_category:Game Reviews)))->cache(_type:review_143758)],from[0],size[8],sort[<custom:\"review_type_priority\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@281d24cc>!,<custom:\"date_reviewed\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@115ae1a7>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; "
         },
         {
            "index": "blcdt.products",
            "shard": 4,
            "status": 500,
            "reason": "QueryPhaseExecutionException[[blcdt.products][4]: query[filtered(ConstantScore(cache(review_category:Game Reviews)))->cache(_type:review_143758)],from[0],size[8],sort[<custom:\"review_type_priority\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@41238b2a>!,<custom:\"date_reviewed\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@48593cf>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; "
         },
         {
            "index": "blcdt.products",
            "shard": 0,
            "status": 500,
            "reason": "QueryPhaseExecutionException[[blcdt.products][0]: query[filtered(ConstantScore(cache(review_category:Game Reviews)))->cache(_type:review_143758)],from[0],size[8],sort[<custom:\"review_type_priority\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@1cbd767c>!,<custom:\"date_reviewed\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@5aadd5a1>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; "
         }
      ]
   },
   "hits": {
      "total": 41,
      "max_score": null,
      "hits": [
         {
            "_index": "blcdt.products",
            "_type": "review_143758",
            "_id": "599606",
            "_score": null,
            "fields": {
               "score": [
                  8
               ],
               "review_type": [
                  "review"
               ],
               "review_title": [
                  "Entwined review"
               ]
            },
            "sort": [
               100,
               1403648392
            ]
         },
         ...
         {
            "_index": "blcdt.products",
            "_type": "review_143758",
            "_id": "452429",
            "_score": null,
            "fields": {
               "score": [
                  9
               ],
               "review_type": [
                  "review"
               ],
               "review_title": [
                  "Dark Souls II review"
               ]
            },
            "sort": [
               100,
               1394840094
            ]
         }
      ]
   }
}

2 个答案:

答案 0 :(得分:4)

我想出了如何解决问题,但我不知道为什么要修复它。通过将“review_type_priority”的数据类型从整数更改为long,它不再抛出异常,现在我得到了我应该得到的所有评论。任何人都想知道为什么这个领域必须很长?

答案 1 :(得分:1)

我遇到了类似的问题,我在某些分片上遇到了失败而在其他分片上失败了。事实证明,我删除了文档,删除了映射,为一个字段添加了更新数据类型的新映射,再次添加了文档,并且我的聚合未能分析我已更改映射的字段。

解决问题的方法是优化发生故障的索引:

POST logstash-2014.08/_optimize?max_num_segments=1&wait_for_merge=false

(在Sense中)

根据我发现的一些Lucene文档,即使删除了索引中的所有文档,也可以保留类型索引,并且优化索引会刷新类型索引。您可以在优化索引here上看到一些ES文档。