无法通过mgo中的排序查询获得结果

时间:2018-04-17 15:12:19

标签: mongodb mgo

我试图了解我的用例有什么问题。我试图用这样的mgo排序查询来查询MongoDB:

conn := _Mongo.Connect() // custom mongo connection
defer conn.Close()
c := conn.DB(dbname).C(collname)
index := mgo.Index{
    Key:        skeys,
    Unique:     false,
    Background: true,
}
e := c.EnsureIndex(index)
if e != nil {
   panic(e)
}
err := c.Find(spec).Sort(skeys...).All(&out)

正如你所看到的,我确实在我传递的选择键(skeys)上调用了ensure index。但在大型查询的运行时,我收到以下错误:

Executor error: OperationFailed: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.

我检查过我的收藏确实有一个索引,但在我看来mgo不使用它。

谢谢, 瓦伦丁。

P.S。这里是mongo解释输出的问题:

Explain: bson.M{"queryPlanner":bson.M{"parsedQuery":bson.M{"$and":[]interface {}{bson.M{"das.record":bson.M{"$eq":1}}, bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}}}, "winningPlan":bson.M{"stage":"SORT", "sortPattern":bson.M{"dataset.name":1}, "inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", "inputStage":bson.M{"stage":"FETCH", "filter":bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}, "inputStage":bson.M{"isSparse":false, "isPartial":false, "indexVersion":2, "direction":"forward", "stage":"IXSCAN", "keyPattern":bson.M{"das.record":1}, "indexName":"das.record_1", "isMultiKey":false, "isUnique":false, "indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}}}}}, "rejectedPlans":[]interface {}{bson.M{"stage":"SORT", "sortPattern":bson.M{"dataset.name":1}, "inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", "inputStage":bson.M{"stage":"FETCH", "filter":bson.M{"das.record":bson.M{"$eq":1}}, "inputStage":bson.M{"keyPattern":bson.M{"qhash":1}, "indexName":"qhash_1", "isMultiKey":false, "isSparse":false, "stage":"IXSCAN", "isUnique":false, "isPartial":false, "indexVersion":2, "direction":"forward", "indexBounds":bson.M{"qhash":[]interface {}{"[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"}}}}}}, bson.M{"stage":"SORT", "sortPattern":bson.M{"dataset.name":1}, "inputStage":bson.M{"stage":"KEEP_MUTATIONS", "inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", "inputStage":bson.M{"stage":"FETCH", "inputStage":bson.M{"stage":"AND_SORTED", "inputStages":[]interface {}{bson.M{"keyPattern":bson.M{"qhash":1}, "isPartial":false, "indexVersion":2, "indexBounds":bson.M{"qhash":[]interface {}{"[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"}}, "direction":"forward", "stage":"IXSCAN", "indexName":"qhash_1", "isMultiKey":false, "isUnique":false, "isSparse":false}, bson.M{"stage":"IXSCAN", "isMultiKey":false, "isUnique":false, "isSparse":false, "isPartial":false, "direction":"forward", "indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}, "keyPattern":bson.M{"das.record":1}, "indexName":"das.record_1", "indexVersion":2}}}}}}}, bson.M{"sortPattern":bson.M{"dataset.name":1}, "inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", "inputStage":bson.M{"stage":"FETCH", "filter":bson.M{"$and":[]interface {}{bson.M{"das.record":bson.M{"$eq":1}}, bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}}}, "inputStage":bson.M{"direction":"forward", "keyPattern":bson.M{"dataset.name":1}, "indexName":"dataset.name_1", "isMultiKey":true, "isSparse":false, "indexBounds":bson.M{"dataset.name":[]interface {}{"[MinKey, MaxKey]"}}, "stage":"IXSCAN", "isUnique":false, "isPartial":false, "indexVersion":2}}}, "stage":"SORT"}}, "plannerVersion":1, "namespace":"das.cache", "indexFilterSet":false}, "executionStats":bson.M{"executionTimeMillis":317, "totalDocsExamined":30972, "executionSuccess":false, "errorMessage":"Exec error resulting in state FAILURE :: caused by :: errmsg: \"Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.\"", "totalKeysExamined":30972, "executionStages":bson.M{"isEOF":0, "sortPattern":bson.M{"dataset.name":1}, "needTime":30973, "saveState":497, "nReturned":0, "restoreState":497, "executionTimeMillisEstimate":150, "memLimit":33554432, "advanced":0, "needYield":0, "invalidates":0, "memUsage":33554660, "inputStage":bson.M{"advanced":30972, "needTime":1, "saveState":497, "restoreState":497, "isEOF":0, "invalidates":0, "inputStage":bson.M{"stage":"FETCH", "works":30972, "restoreState":497, "invalidates":0, "alreadyHasObj":0, "inputStage":bson.M{"dupsDropped":0, "seenInvalidated":0, "stage":"IXSCAN", "needTime":0, "isMultiKey":false, "indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}, "keysExamined":30972, "direction":"forward", "restoreState":497, "isEOF":0, "indexName":"das.record_1", "isSparse":false, "indexVersion":2, "seeks":1, "dupsTested":0, "executionTimeMillisEstimate":35, "advanced":30972, "needYield":0, "saveState":497, "keyPattern":bson.M{"das.record":1}, "nReturned":30972, "works":30972, "invalidates":0, "isUnique":false, "isPartial":false}, "saveState":497, "filter":bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}, "advanced":30972, "isEOF":0, "docsExamined":30972, "nReturned":30972, "executionTimeMillisEstimate":68, "needTime":0, "needYield":0}, "stage":"SORT_KEY_GENERATOR", "nReturned":30972, "executionTimeMillisEstimate":138, "works":30973, "needYield":0}, "stage":"SORT", "works":30974}, "allPlansExecution":[]interface {}{bson.M{"nReturned":0, "executionTimeMillisEstimate":69, "totalKeysExamined":10491, "totalDocsExamined":10491, "executionStages":bson.M{"stage":"SORT", "executionTimeMillisEstimate":69, "works":10492, "restoreState":335, "isEOF":0, "memUsage":11812527, "inputStage":bson.M{"advanced":10491, "needTime":1, "saveState":335, "restoreState":335, "isEOF":0, "stage":"SORT_KEY_GENERATOR", "executionTimeMillisEstimate":69, "works":10492, "invalidates":0, "inputStage":bson.M{"needTime":0, "invalidates":0, "isEOF":0, "docsExamined":10491, "alreadyHasObj":0, "stage":"FETCH", "filter":bson.M{"das.record":bson.M{"$eq":1}}, "advanced":10491, "needYield":0, "saveState":335, "inputStage":bson.M{"invalidates":0, "isMultiKey":false, "seeks":1, "dupsTested":0, "needTime":0, "isEOF":0, "works":10491, "needYield":0, "restoreState":335, "keyPattern":bson.M{"qhash":1}, "indexName":"qhash_1", "isSparse":false, "stage":"IXSCAN", "nReturned":10491, "seenInvalidated":0, "indexVersion":2, "indexBounds":bson.M{"qhash":[]interface {}{"[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"}}, "advanced":10491, "isPartial":false, "isUnique":false, "direction":"forward", "keysExamined":10491, "dupsDropped":0, "executionTimeMillisEstimate":10, "saveState":335}, "nReturned":10491, "executionTimeMillisEstimate":55, "works":10491, "restoreState":335}, "nReturned":10491, "needYield":0}, "needTime":10492, "memLimit":33554432, "nReturned":0, "advanced":0, "needYield":0, "saveState":335, "invalidates":0, "sortPattern":bson.M{"dataset.name":1}}}, bson.M{"nReturned":0, "executionTimeMillisEstimate":11, "totalKeysExamined":10491, "totalDocsExamined":5245, "executionStages":bson.M{"restoreState":335, "inputStage":bson.M{"saveState":335, "restoreState":335, "inputStage":bson.M{"nReturned":5245, "executionTimeMillisEstimate":11, "advanced":5245, "needYield":0, "restoreState":335, "invalidates":0, "inputStage":bson.M{"isEOF":0, "alreadyHasObj":0, "works":10491, "needTime":5246, "restoreState":335, "advanced":5245, "saveState":335, "nReturned":5245, "needYield":0, "invalidates":0, "inputStage":bson.M{"stage":"AND_SORTED", "executionTimeMillisEstimate":0, "works":10491, "saveState":335, "advanced":5245, "needTime":5246, "nReturned":5245, "needYield":0, "isEOF":0, "flagged":0, "restoreState":335, "invalidates":0, "failedAnd_0":0, "failedAnd_1":0, "inputStages":[]interface {}{bson.M{"stage":"IXSCAN", "nReturned":5246, "invalidates":0, "isUnique":false, "keysExamined":5246, "dupsDropped":0, "seeks":1, "dupsTested":0, "executionTimeMillisEstimate":0, "advanced":5246, "restoreState":335, "keyPattern":bson.M{"qhash":1}, "indexName":"qhash_1", "indexVersion":2, "indexBounds":bson.M{"qhash":[]interface {}{"[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"}}, "needTime":0, "needYield":0, "saveState":335, "isEOF":0, "isMultiKey":false, "direction":"forward", "works":5246, "isSparse":false, "isPartial":false, "seenInvalidated":0}, bson.M{"advanced":5245, "isEOF":0, "indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}, "seeks":1, "stage":"IXSCAN", "nReturned":5245, "needTime":0, "needYield":0, "isSparse":false, "indexVersion":2, "isPartial":false, "direction":"forward", "executionTimeMillisEstimate":0, "works":5245, "saveState":335, "invalidates":0, "keyPattern":bson.M{"das.record":1}, "isMultiKey":false, "keysExamined":5245, "restoreState":335, "indexName":"das.record_1", "isUnique":false, "dupsTested":0, "dupsDropped":0, "seenInvalidated":0}}}, "stage":"FETCH", "executionTimeMillisEstimate":11, "docsExamined":5245}, "stage":"SORT_KEY_GENERATOR", "works":10492, "needTime":5247, "saveState":335, "isEOF":0}, "stage":"KEEP_MUTATIONS", "nReturned":5245, "needYield":0, "needTime":5247, "isEOF":0, "invalidates":0, "executionTimeMillisEstimate":11, "works":10492, "advanced":5245}, "executionTimeMillisEstimate":11, "works":10492, "needYield":0, "invalidates":0, "memUsage":6102195, "stage":"SORT", "nReturned":0, "advanced":0, "needTime":10492, "saveState":335, "isEOF":0, "sortPattern":bson.M{"dataset.name":1}, "memLimit":33554432}}, bson.M{"nReturned":0, "executionTimeMillisEstimate":70, "totalKeysExamined":10491, "totalDocsExamined":10491, "executionStages":bson.M{"executionTimeMillisEstimate":70, "saveState":335, "sortPattern":bson.M{"dataset.name":1}, "works":10492, "inputStage":bson.M{"advanced":10485, "needYield":0, "executionTimeMillisEstimate":58, "works":10492, "needTime":7, "saveState":335, "restoreState":335, "isEOF":0, "invalidates":0, "inputStage":bson.M{"executionTimeMillisEstimate":36, "saveState":335, "docsExamined":10491, "nReturned":10485, "restoreState":335, "invalidates":0, "needYield":0, "filter":bson.M{"$and":[]interface {}{bson.M{"das.record":bson.M{"$eq":1}}, bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}}}, "advanced":10485, "alreadyHasObj":0, "stage":"FETCH", "needTime":6, "isEOF":0, "inputStage":bson.M{"keyPattern":bson.M{"dataset.name":1}, "isPartial":false, "indexVersion":2, "nReturned":10491, "executionTimeMillisEstimate":24, "advanced":10491, "saveState":335, "invalidates":0, "indexBounds":bson.M{"dataset.name":[]interface {}{"[MinKey, MaxKey]"}}, "dupsDropped":0, "seenInvalidated":0, "dupsTested":10491, "works":10491, "needTime":0, "isMultiKey":true, "isUnique":false, "keysExamined":10491, "needYield":0, "isEOF":0, "direction":"forward", "seeks":1, "stage":"IXSCAN", "restoreState":335, "indexName":"dataset.name_1", "isSparse":false}, "works":10491}, "stage":"SORT_KEY_GENERATOR", "nReturned":10485}, "stage":"SORT", "nReturned":0, "advanced":0, "memUsage":11787104, "needTime":10492, "needYield":0, "restoreState":335, "isEOF":0, "invalidates":0, "memLimit":33554432}}, bson.M{"executionTimeMillisEstimate":59, "totalKeysExamined":10491, "totalDocsExamined":10491, "executionStages":bson.M{"works":10492, "advanced":0, "needTime":10492, "isEOF":0, "invalidates":0, "sortPattern":bson.M{"dataset.name":1}, "restoreState":335, "memUsage":11812527, "inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", "works":10492, "advanced":10491, "inputStage":bson.M{"inputStage":bson.M{"isMultiKey":false, "isSparse":false, "indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}, "seeks":1, "advanced":10491, "needTime":0, "saveState":335, "indexName":"das.record_1", "dupsTested":0, "restoreState":335, "isPartial":false, "dupsDropped":0, "isUnique":false, "indexVersion":2, "direction":"forward", "keysExamined":10491, "stage":"IXSCAN", "nReturned":10491, "needYield":0, "isEOF":0, "seenInvalidated":0, "executionTimeMillisEstimate":11, "works":10491, "invalidates":0, "keyPattern":bson.M{"das.record":1}}, "stage":"FETCH", "works":10491, "advanced":10491, "isEOF":0, "invalidates":0, "alreadyHasObj":0, "filter":bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}, "nReturned":10491, "executionTimeMillisEstimate":21, "needTime":0, "needYield":0, "saveState":335, "restoreState":335, "docsExamined":10491}, "saveState":335, "restoreState":335, "isEOF":0, "invalidates":0, "nReturned":10491, "executionTimeMillisEstimate":47, "needTime":1, "needYield":0}, "stage":"SORT", "executionTimeMillisEstimate":59, "needYield":0, "nReturned":0, "saveState":335, "memLimit":33554432}, "nReturned":0}}, "errorCode":96, "nReturned":0}, "serverInfo":bson.M{"version":"3.6.3", "gitVersion":"9586e557d54ef70f9ca4b43c26892cd55257e1a5", "host":"vkair", "port":8230}}

特别是,它说:

"inputStage":bson.M{"direction":"forward", "keyPattern":bson.M{"dataset.name":1},
"indexName":"dataset.name_1", "isMultiKey":true, "isSparse":false, 
"indexBounds":bson.M{"dataset.name":[]interface {}{"[MinKey, MaxKey]"}},     
"stage":"IXSCAN", "isUnique":false, "isPartial":false, "indexVersion":2}}}, 
"stage":"SORT"}}, 
"plannerVersion":1, "namespace":"das.cache", "indexFilterSet":false}, 
"executionStats":bson.M{"executionTimeMillis":317, "totalDocsExamined":30972,
"executionSuccess":false,
"errorMessage":"Exec error resulting in state FAILURE :: caused by :: errmsg: \"Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.

P.S.S。获胜计划是(mgo的输出):

"winningPlan":bson.M{"stage":"SORT", 
"sortPattern":bson.M{"dataset.name":1}, 
"inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", 
"inputStage":bson.M{"stage":"FETCH", 
"filter":bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}, 
"inputStage":bson.M{"isSparse":false, 
"isPartial":false, "indexVersion":2, 
"direction":"forward", "stage":"IXSCAN", 
"keyPattern":bson.M{"das.record":1}, 
"indexName":"das.record_1",
"isMultiKey":false, "isUnique":false, 
"indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}}}}}

和,来自mongo shell的输出

                "winningPlan" : {
                    "stage" : "SORT",
                    "sortPattern" : {
                            "dataset.name" : 1
                    },
                    "inputStage" : {
                            "stage" : "SORT_KEY_GENERATOR",
                            "inputStage" : {
                                    "stage" : "FETCH",
                                    "filter" : {
                                            "qhash" : {
                                                    "$eq" : "7b32b92becab9b5de06fa8ac85011133"
                                            }
                                    },
                                    "inputStage" : {
                                            "stage" : "IXSCAN",
                                            "keyPattern" : {
                                                    "das.record" : 1
                                            },
                                            "indexName" : "das.record_1",
                                            "isMultiKey" : false,
                                            "isUnique" : false,
                                            "isSparse" : false,
                                            "isPartial" : false,
                                            "indexVersion" : 2,
                                            "direction" : "forward",
                                            "indexBounds" : {
                                                    "das.record" : [
                                                            "[1.0, 1.0]"
                                                    ]
                                            }
                                    }
                            }
                    }
            }

据我所知,mgo和mongo shell的获胜计划是一样的。如果我在mongo shell中执行相同的查询,我会得到以下结果:

> db.cache.find({"qhash":"7b32b92becab9b5de06fa8ac85011133", "das.record":1}).sort({"dataset.name":1})
Error: error: {
    "ok" : 0,
    "errmsg" : "Executor error during find command: OperationFailed: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.",
    "code" : 96,
    "codeName" : "OperationFailed"}

如果我计算聚合,它会快速返回结果:

db.cache.find({"qhash":"7b32b92becab9b5de06fa8ac85011133", "das.record":1}).sort({"dataset.name":1}).count()
69936

因此看起来mongo本身在获取结果时使用获胜计划(索引)存在问题。

UPDATE1 正如万建议我创建了新的复合指数

db.cache.createIndex({"qhash":1, "das.record":1, "dataset.name":1})

但它没有解决问题。如果我放置查询

db.cache.find({"qhash":"7b32b92becab9b5de06fa8ac85011133", "das.record":1}).sort({"dataset.name":1}).explain()

获胜计划仍然使用das.record索引,请参阅

{
    "queryPlanner" : {
            "plannerVersion" : 1,
            "namespace" : "das.cache",
            "indexFilterSet" : false,
            "parsedQuery" : {
                    "$and" : [
                            {
                                    "das.record" : {
                                            "$eq" : 1
                                    }
                            },
                            {
                                    "qhash" : {
                                            "$eq" : "7b32b92becab9b5de06fa8ac85011133"
                                    }
                            }
                    ]
            },
            "winningPlan" : {
                    "stage" : "SORT",
                    "sortPattern" : {
                            "dataset.name" : 1
                    },
                    "inputStage" : {
                            "stage" : "SORT_KEY_GENERATOR",
                            "inputStage" : {
                                    "stage" : "FETCH",
                                    "filter" : {
                                            "qhash" : {
                                                    "$eq" : "7b32b92becab9b5de06fa8ac85011133"
                                            }
                                    },
                                    "inputStage" : {
                                            "stage" : "IXSCAN",
                                            "keyPattern" : {
                                                    "das.record" : 1
                                            },
                                            "indexName" : "das.record_1",
                                            "isMultiKey" : false,
                                            "isUnique" : false,
                                            "isSparse" : false,
                                            "isPartial" : false,
                                            "indexVersion" : 2,
                                            "direction" : "forward",
                                            "indexBounds" : {
                                                    "das.record" : [
                                                            "[1.0, 1.0]"
                                                    ]
                                            }
                                    }
                            }
                    }
            },
            "rejectedPlans" : [
                    {
                            "stage" : "SORT",
                            "sortPattern" : {
                                    "dataset.name" : 1
                            },
                            "inputStage" : {
                                    "stage" : "SORT_KEY_GENERATOR",
                                    "inputStage" : {
                                            "stage" : "FETCH",
                                            "filter" : {
                                                    "das.record" : {
                                                            "$eq" : 1
                                                    }
                                            },
                                            "inputStage" : {
                                                    "stage" : "IXSCAN",
                                                    "keyPattern" : {
                                                            "qhash" : 1
                                                    },
                                                    "indexName" : "qhash_1",
                                                    "isMultiKey" : false,
                                                    "isUnique" : false,
                                                    "isSparse" : false,
                                                    "isPartial" : false,
                                                    "indexVersion" : 2,
                                                    "direction" : "forward",
                                                    "indexBounds" : {
                                                            "qhash" : [
                                                                    "[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"
                                                            ]
                                                    }
                                            }
                                    }
                            }
                    },
                    {
                            "stage" : "SORT",
                            "sortPattern" : {
                                    "dataset.name" : 1
                            },
                            "inputStage" : {
                                    "stage" : "SORT_KEY_GENERATOR",
                                    "inputStage" : {
                                            "stage" : "FETCH",
                                            "inputStage" : {
                                                    "stage" : "IXSCAN",
                                                    "keyPattern" : {
                                                            "qhash" : 1,
                                                            "das.record" : 1,
                                                            "dataset.name" : 1
                                                    },
                                                    "indexName" : "qhash_1_das.record_1_dataset.name_1",
                                                    "isMultiKey" : true,
                                                    "isUnique" : false,
                                                    "isSparse" : false,
                                                    "isPartial" : false,
                                                    "indexVersion" : 2,
                                                    "direction" : "forward",
                                                    "indexBounds" : {
                                                            "qhash" : [
                                                                    "[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"
                                                            ],
                                                            "das.record" : [
                                                                    "[1.0, 1.0]"
                                                            ],
                                                            "dataset.name" : [
                                                                    "[MinKey, MaxKey]"
                                                            ]
                                                    }
                                            }
                                    }
                            }
                    },
                    {
                            "stage" : "SORT",
                            "sortPattern" : {
                                    "dataset.name" : 1
                            },
                            "inputStage" : {
                                    "stage" : "SORT_KEY_GENERATOR",
                                    "inputStage" : {
                                            "stage" : "FETCH",
                                            "filter" : {
                                                    "$and" : [
                                                            {
                                                                    "das.record" : {
                                                                            "$eq" : 1
                                                                    }
                                                            },
                                                            {
                                                                    "qhash" : {
                                                                            "$eq" : "7b32b92becab9b5de06fa8ac85011133"
                                                                    }
                                                            }
                                                    ]
                                            },
                                            "inputStage" : {
                                                    "stage" : "IXSCAN",
                                                    "keyPattern" : {
                                                            "dataset.name" : 1
                                                    },
                                                    "indexName" : "dataset.name_1",
                                                    "isMultiKey" : true,
                                                    "isUnique" : false,
                                                    "isSparse" : false,
                                                    "isPartial" : false,
                                                    "indexVersion" : 2,
                                                    "direction" : "forward",
                                                    "indexBounds" : {
                                                            "dataset.name" : [
                                                                    "[MinKey, MaxKey]"
                                                            ]
                                                    }
                                            }
                                    }
                            }
                    }
            ]
    },
    "serverInfo" : {
            "host" : "vkair",
            "port" : 8230,
            "version" : "3.6.3",
            "gitVersion" : "9586e557d54ef70f9ca4b43c26892cd55257e1a5"
    },
    "ok" : 1
}

1 个答案:

答案 0 :(得分:1)

  

执行程序错误:OperationFailed:排序操作使用的RAM超过最大33554432字节。添加索引,或指定较小的限制。

在MongoDB中,排序操作可以通过根据索引中的排序检索文档来获取排序顺序。如果查询计划程序无法从索引获取排序顺序,它将在内存中对结果进行排序。使用索引以排序顺序检索结果比检索结果和在内存中排序更有效。此外,不使用索引的排序操作将在使用32兆字节的内存时中止。

根据您发布的explain的输出,这是因为您有das.record的索引(获胜计划,用于查询),但您需要根据{{进行排序1}}其中a)失去计划或b)没有这样的索引。

An index可以支持对索引键模式的非前缀子集进行排序操作。为此,查询必须在排序键之前的所有前缀键上包含相等条件。例如,如果您的查询是:

dataset.name

然后要使用索引进行过滤和排序,您应该有一个compound index

db.cache.find({"qhash":"7b32b92becab9b5de06fa8ac85011133", "das.record":1}).sort({"dataset.name":1})

查看有关手册Tutorial: sort and non-prefix subset of an index

的更多信息

您还可以从阅读中受益:MongoDB Indexing StrategiesBlog: Optimizing MongoDB Compound Indexes

  

据我所知,mgo和mongo shell的获胜计划是相同的

给定相同的查询,输出应该相同。我要求{"qhash":1, "das.record":1, "dataset.name":1 } shell输出的原因是删除mongo方面,即调试层中的代码。

更新

  

我创建了额外的索引{" v" :2," key" :{" qhash" :1," das.record" :1," dataset.name" :1},"名称" :" qhash_1_das.record_1_dataset.name_1"," ns" :" das.cache"但是MongoDB不使用它

根据您的解释输出,您可以发现Go部分下列出了新索引qhash_1_das.record_1_dataset.name_1。这种情况的可能原因很少,但在这种情况下可能的原因是rejectedPlan是一个数组或具有一些数组值。由dataset.name表示,isMultiKey:true的索引范围。

min, max上的multikey index可能导致意外的缓慢,因此被拒绝作为最快的查询计划。

如果您不打算在dataset.name中拥有数组值,请找到有问题的文档并更改/删除数组值。此外,您还需要重新创建索引dataset.name

如果qhash_1_das.record_1_dataset.name_1实际上包含数组,则基于多键的排序性能会降低。您可能必须删除其他索引,选择具有单个值的其他字段进行排序,操作拆分dataset.name的值,或重新设计dataset.name文档结构。