Mongo DB Map在尝试处理超过100条记录时减少

时间:2016-01-06 09:00:54

标签: mongodb mapreduce mongodb-query

我是Mongo DB的新手,我使用的是Mongo DB 2.4.4。当我们尝试在Mapreduce中传递超过100条记录时,我们遇到以下错误:

com.mongodb.CommandFailureException: { "serverUsed" : "localhost/XXXX" , "errmsg" : "exception: TypeError: Cannot read property 'AAAA' of undefined\n    at _funcs2:1:296\n    at Array.forEach (native)\n    at _funcs2 (_funcs2:1:252) near 'ssionEvent.AAAA>0)

以下是正在使用的MapReduceCommand

MapReduceCommand(eventsCollection,map,reduce,null,MapReduceCommand.OutputType.INLINE, compoundquery);

我有相同的记录,但如果记录数超过100,则显示上述错误,如果小于100,则成功处理。 我们在应用程序中使用了以下mongo选项。

"maxWaitTime=300"
"connectionsPerHost=100"
"threadsAllowedToBlockForConnectionMultiplier=1500" 
"socketKeepAlive=true"
"connectTimeout=60000"
"socketTimeout=60000"
"autoConnectRetry=true"

有关Map,reduce和复合查询的详细信息如下:

String map = "function() {"+
" if (this.Doc1 !== undefined & this.Doc1.innerDoc1 !== undefined) { "+
"emit({field1:this.Doc1.innerDoc1.field1,field2:this.Doc1.field2,field3:this.Doc1.field3,field4:this.Doc1.field4,field5:this.Doc1.field5},this);}";
String reduce = "function(keyMappedId, valuesPrices){"+
 "var amtValue1=0;"+ 
 "var processedObj=new Object();"+
 "valuesPrices.forEach(function(doc){"+
 if(doc.Doc1 !== undefined && doc.Doc1.AmtValue !== undefined && doc.Doc1.AmtValue>0){"+ 
     "var amtNumber=new Number(doc.Doc1.AmtValue);"+ 
     "amtValue1=amtNumber;}});"+
"if(amtValue1>0 && valuesPrices[0].Doc1 !== undefined){"+
        "valuesPrices[0].Doc1.AmtValue=amtValue1; }"+
"processedObj=valuesPrices[0];"+ 
"return {aggregatedObject:processedObj};}";



DBObject compoundquery =  QueryBuilder.start("_id").in(idList).get();

其中idList将包含集合中文档的“_id”,我希望将其传递给map reduce

MapReduceCommand cmd=null;
cmd = new MapReduceCommand("MyCollection",map,reduce,null,MapReduceCommand.OutputType.INLINE, compoundquery);
            MapReduceOutput out = eventsCollection.mapReduce(cmd);

添加错误输出如下:103条记录

{ "aggregatedObject" : { "aggregatedObject" : { "_id" : "Order_REC16904427" , "Status" : "PROCESSING" , "Doc1" : { "field1" : "ABCS57829" , "field2" : "XYZ" , "AmtValue" : 9000.0 , "field3" : "AAAABCD" , "Doc2" : { "Serial_No" : "1" , "field4" : "Order" , "field5" : "ABC_1234"} , "Aggregation_Status" : "Submitted"}}}

获得311条记录

{ "aggregatedObject" : { "aggregatedObject" : { "aggregatedObject" : { "aggregatedObject" : { "_id" : "Order_REC16904427" , "Status" : "PROCESSING" , "Doc1" : { "field1" : "ABCS57829" , "field2" : "XYZ" , "AmtValue" : 9000.0 , "field3" : "AAAABCD" , "Doc2" : { "Serial_No" : "1" , "field4" : "Order" , "field5" : "ABC_1234"} , "Aggregation_Status" : "Submitted"}}}}}

1 个答案:

答案 0 :(得分:0)

基本上,MongoDB支持不超过100级的BSON文档嵌套。

This discussion遇到了同样的问题。我想这可能会有所帮助。)