将MongoDB数据导出到ElasticSearch

时间:2014-09-03 11:05:59

标签: mongodb elasticsearch

我想将数据从MongoDB导出到ElasticSearch.Mongo River插件是我先选择转储集合然后恢复哪个适合我的选项。但我不想使用Mongo River插件,我正在使用elaster将数据从MongoDB导出到Elasticsearch。

  • 弹性搜索版本 - 1.3.2
  • Node.js版本 - v0.11.8-pre
  • MongoDB版本 - 2.4.x

当我执行:./bin/elaster时,它说:

{ 
  [Error: MapperParsingException[object mapping for [collection] tried to parse as object, but got EOF, has a concrete value been provided to it?]]
  message: 'MapperParsingException[object mapping for [collection] tried to parse as object, but got EOF, has a concrete value been provided to it?]'
}

我的Elaster配置是:

module.exports = {

  mongo: {
    connection: 'mongodb://127.0.0.1:27017/times'
  },

  elastic: {
    host: {
      host: '127.0.0.1'
    },
    requestTimeout: 5000
  },

  collections: [ 
    {
      name: "walldisplay",
      index: "walldisplay",
      type: "collection",
      fields: [
        "_id",
        "wat",
        "wct",
        "u",
        "i",
        "cd"
      ],
      mappings: {
        "collection": {
          "properties": {
            "wat":{
              'type':'string',
              'index': 'not_analyzed'
            },
            "wct":{
              'type':'string',
              'index': 'not_analyzed'
            },
            "u":{
              "type" : "object",
              "dynamic" : true,
              "properties":{
                "_id":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "n":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "st":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "id":{
                  'type':'string',
                  'index': 'not_analyzed'
                }
              },
              "index":"not_analyzed"
            },
            "i":{
              "type" : "nested",
              "include_in_parent" : true,
              "properties":{
                "_id":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "ti":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "st":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "n":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "cst":{
                  'type':'string',
                  'index': 'not_analyzed'
                }
              }
            },
            "cd":{
              'type':'long',
              'index': 'not_analyzed'
            },
          }
        }
      }
    }
  ]
};

另外,请在线检查样本文件

{
  "_id": ObjectId("5406a47970b17246b9a293e1"),
  "cd": 1409721465,
  "i": [
    {
      "_id": ObjectId("50f693d17deed44cf000007f"),
      "st": "seo-title",
      "ti": "title",
      "n": "categoryname",
      "cst": "category-seotitle",
      "r": null,
      "c": null
    },
    {
      "_id": ObjectId("50f693d17deed44cf000007f"),
      "st": "seo-title",
      "ti": "title",
      "n": "categoryname",
      "cst": "category-seotitle",
      "r": null,
      "c": null
    },
    {
      "_id": ObjectId("50f693d17deed44cf000007f"),
      "st": "seo-title",
      "ti": "title",
      "n": "categoryname",
      "cst": "category-seotitle",
      "r": null,
      "c": null
    }
  ],
  "u": {
    "_id": ObjectId("50ce4f79edaffd69e40ee010"),
    "n": "Richa Sen",
    "st": "richasen",
    "id": "d8mzxlp9ekn323l6jg5s8tly1"
  },
  "wat": 1,
  "wct": 1
}

2 个答案:

答案 0 :(得分:1)

由于您已将" collection"定义为" walldisplay"作为类型和索引,因此映射中的类型应为" collection"而不是" walldisplay"。

  

put mapping API允许注册特定的映射定义   特定类型

查看以下内容是否有效

 ..
     mappings:{
                "collection":{
                       "properties":{
                              ...
                           }
                     }
               }

答案 1 :(得分:0)

WCT字段被映射为数字类型,但是你的文档中有一个wct值作为字符串,这就是你得到这样的错误的原因所以试着把你的wct值从字符串改为整数。