如何在多个范围的单个字段上有效查询Mongodb?

时间:2019-04-03 10:53:44

标签: mongodb geospatial compound-index mongodb-geospatial

我正在尝试使用MongoDB的B树索引来构建自定义地理空间索引,因为我发现Mongo的本机实现对我自己的情况有所限制。为了满足可使用复合索引有效搜索Mongo的地理空间查询,我需要按location.locIndexKey字段进行过滤,其中包括多个范围。

到目前为止,我能想到的唯一支持这种查询的解决方案是使用Mongo的$or运算符。但是,由于这是一个or查询,因此执行效果很差,Mongo必须一次又一次地检查索引上的相同键。为了克服这种效率低下的问题,我需要一种方法,使Mongo在该字段上使用多个索引范围,而不是对查询中的每个定义的边界使用or短语来复制查询。

这是我的查询:

db.users.find({
    "gender":2,
    "preferences.feed.gender":1,
    "age":{"$gte":18,"$lte":55},
    "feedPrefChangeDay":{"$gte":1553461200,"$lte":1554066000},
    "$or":[{"location.locIndexKey":{"$gte":NumberLong(1493233547543052300),"$lte":NumberLong(1493242343636074500)}},{"location.locIndexKey":{"$gte":NumberLong(1493242343636074500),"$lte":NumberLong(1493251139729096700)}},{"location.locIndexKey":{"$gte":NumberLong(1493287011295953000),"$lte":NumberLong(1493287148734906400)}}]
}).limit(20);

如您所见,为了表示字段location.locIndexKey上的多个范围,我不得不使用$or运算符。这是查询计划程序执行状态的简化版本:

{
    "executionSuccess" : true,
    "nReturned" : 0,
    "executionTimeMillis" : 17762,
    "totalKeysExamined" : 196192,
    "totalDocsExamined" : 0,
    "executionStages" : {
        "stage" : "LIMIT",
        "nReturned" : 0,
        "executionTimeMillisEstimate" : 351,
        "works" : 196193,
        "advanced" : 0,
        "needTime" : 196191,
        "needYield" : 0,
        "saveState" : 19944,
        "restoreState" : 19944,
        "isEOF" : 1,
        "invalidates" : 0,
        "limitAmount" : 20,
        "inputStage" : {
            "stage" : "FETCH",
            "nReturned" : 0,
            "executionTimeMillisEstimate" : 351,
            "works" : 196193,
            "advanced" : 0,
            "needTime" : 196191,
            "needYield" : 0,
            "saveState" : 19944,
            "restoreState" : 19944,
            "isEOF" : 1,
            "invalidates" : 0,
            "docsExamined" : 0,
            "alreadyHasObj" : 0,
            "inputStage" : {
                "stage" : "OR",
                "nReturned" : 0,
                "executionTimeMillisEstimate" : 351,
                "works" : 196192,
                "advanced" : 0,
                "needTime" : 196191,
                "needYield" : 0,
                "saveState" : 19944,
                "restoreState" : 19944,
                "isEOF" : 1,
                "invalidates" : 0,
                "dupsTested" : 0,
                "dupsDropped" : 0,
                "recordIdsForgotten" : 0,
                "inputStages" : [ 
                    {
                        "stage" : "IXSCAN",
                        "nReturned" : 0,
                        "executionTimeMillisEstimate" : 10,
                        "works" : 4534,
                        "advanced" : 0,
                        "needTime" : 4533,
                        "needYield" : 0,
                        "saveState" : 19944,
                        "restoreState" : 19944,
                        "isEOF" : 1,
                        "invalidates" : 0,
                        "keyPattern" : {
                            "gender" : 1.0,
                            "preferences.feed.gender" : 1.0,
                            "age" : 1.0,
                            "feedPrefChangeDay" : 1.0,
                            "location.locIndexKey" : 1.0
                        },
                        "indexName" : "gender_1_preferences.feed.gender_1_age_1_feedPrefChangeDay_1_location.locIndexKey_1",
                        "isMultiKey" : false,
                        "multiKeyPaths" : {
                            "gender" : [],
                            "preferences.feed.gender" : [],
                            "age" : [],
                            "feedPrefChangeDay" : [],
                            "location.locIndexKey" : []
                        },
                        "isUnique" : false,
                        "isSparse" : false,
                        "isPartial" : false,
                        "indexVersion" : 2,
                        "direction" : "forward",
                        "indexBounds" : {
                            "gender" : [ 
                                "[2.0, 2.0]"
                            ],
                            "preferences.feed.gender" : [ 
                                "[1.0, 1.0]"
                            ],
                            "age" : [ 
                                "[18.0, 55.0]"
                            ],
                            "feedPrefChangeDay" : [ 
                                "[1553461200.0, 1554066000.0]"
                            ],
                            "location.locIndexKey" : [ 
                                "[1493569998101151700, 1493572197124407300]"
                            ]
                        },
                        "keysExamined" : 4534,
                        "seeks" : 4534,
                        "dupsTested" : 0,
                        "dupsDropped" : 0,
                        "seenInvalidated" : 0
                    }, 
                    {
                        "stage" : "IXSCAN",
                        "nReturned" : 0,
                        "executionTimeMillisEstimate" : 0,
                        "works" : 4534,
                        "advanced" : 0,
                        "needTime" : 4533,
                        "needYield" : 0,
                        "saveState" : 19944,
                        "restoreState" : 19944,
                        "isEOF" : 1,
                        "invalidates" : 0,
                        "keyPattern" : {
                            "gender" : 1.0,
                            "preferences.feed.gender" : 1.0,
                            "age" : 1.0,
                            "feedPrefChangeDay" : 1.0,
                            "location.locIndexKey" : 1.0
                        },
                        "indexName" : "gender_1_preferences.feed.gender_1_age_1_feedPrefChangeDay_1_location.locIndexKey_1",
                        "isMultiKey" : false,
                        "multiKeyPaths" : {
                            "gender" : [],
                            "preferences.feed.gender" : [],
                            "age" : [],
                            "feedPrefChangeDay" : [],
                            "location.locIndexKey" : []
                        },
                        "isUnique" : false,
                        "isSparse" : false,
                        "isPartial" : false,
                        "indexVersion" : 2,
                        "direction" : "forward",
                        "indexBounds" : {
                            "gender" : [ 
                                "[2.0, 2.0]"
                            ],
                            "preferences.feed.gender" : [ 
                                "[1.0, 1.0]"
                            ],
                            "age" : [ 
                                "[18.0, 55.0]"
                            ],
                            "feedPrefChangeDay" : [ 
                                "[1553461200.0, 1554066000.0]"
                            ],
                            "location.locIndexKey" : [ 
                                "[1493587581697261600, 1493587590287196200]"
                            ]
                        },
                        "keysExamined" : 4534,
                        "seeks" : 4534,
                        "dupsTested" : 0,
                        "dupsDropped" : 0,
                        "seenInvalidated" : 0
                    }, 
                    {
                        "stage" : "IXSCAN",
                        "nReturned" : 0,
                        "executionTimeMillisEstimate" : 0,
                        "works" : 4534,
                        "advanced" : 0,
                        "needTime" : 4533,
                        "needYield" : 0,
                        "saveState" : 19944,
                        "restoreState" : 19944,
                        "isEOF" : 1,
                        "invalidates" : 0,
                        "keyPattern" : {
                            "gender" : 1.0,
                            "preferences.feed.gender" : 1.0,
                            "age" : 1.0,
                            "feedPrefChangeDay" : 1.0,
                            "location.locIndexKey" : 1.0
                        },
                        "indexName" : "gender_1_preferences.feed.gender_1_age_1_feedPrefChangeDay_1_location.locIndexKey_1",
                        "isMultiKey" : false,
                        "multiKeyPaths" : {
                            "gender" : [],
                            "preferences.feed.gender" : [],
                            "age" : [],
                            "feedPrefChangeDay" : [],
                            "location.locIndexKey" : []
                        },
                        "isUnique" : false,
                        "isSparse" : false,
                        "isPartial" : false,
                        "indexVersion" : 2,
                        "direction" : "forward",
                        "indexBounds" : {
                            "gender" : [ 
                                "[2.0, 2.0]"
                            ],
                            "preferences.feed.gender" : [ 
                                "[1.0, 1.0]"
                            ],
                            "age" : [ 
                                "[18.0, 55.0]"
                            ],
                            "feedPrefChangeDay" : [ 
                                "[1553461200.0, 1554066000.0]"
                            ],
                            "location.locIndexKey" : [ 
                                "[1493981215449940000, 1493990011542962200]"
                            ]
                        },
                        "keysExamined" : 4534,
                        "seeks" : 4534,
                        "dupsTested" : 0,
                        "dupsDropped" : 0,
                        "seenInvalidated" : 0
                    }

如果您在上方选中indexBounds,则会看到location.locIndexKey的每个范围都应用于单个查询,并与or组合在一起。但是,如果我选择使用Mongo的本地地理空间运算符$geoWithin运行相同的查询:

db.users.find({
    "gender" : 2.0,
    "preferences.feed.gender" : 1.0,
    "age" : {
        "$gte" : 18.0,
        "$lte" : 55.0
    },
    "feedPrefChangeDay" : {
        "$gte" : 1553461200.0,
        "$lte" : 1554066000.0
    },
    "location.loc" : {
        "$geoWithin" : {
            "$centerSphere" : [ 
                [ 
                    0.0, 
                    0.0
                ], 
                0.00784806152880239
            ]
        }
    }
}).limit(20);

我从查询计划者那里得到以下答复:

{
    "executionSuccess" : true,
    "nReturned" : 0,
    "executionTimeMillis" : 7,
    "totalKeysExamined" : 4506,
    "totalDocsExamined" : 0,
    "executionStages" : {
        "stage" : "LIMIT",
        "nReturned" : 0,
        "executionTimeMillisEstimate" : 0,
        "works" : 4506,
        "advanced" : 0,
        "needTime" : 4505,
        "needYield" : 0,
        "saveState" : 35,
        "restoreState" : 35,
        "isEOF" : 1,
        "invalidates" : 0,
        "limitAmount" : 20,
        "inputStage" : {
            "stage" : "FETCH",
            "filter" : {
                "location.loc" : {
                    "$geoWithin" : {
                        "$centerSphere" : [ 
                            [ 
                                0.0, 
                                0.0
                            ], 
                            0.00784806152880239
                        ]
                    }
                }
            },
            "nReturned" : 0,
            "executionTimeMillisEstimate" : 0,
            "works" : 4506,
            "advanced" : 0,
            "needTime" : 4505,
            "needYield" : 0,
            "saveState" : 35,
            "restoreState" : 35,
            "isEOF" : 1,
            "invalidates" : 0,
            "docsExamined" : 0,
            "alreadyHasObj" : 0,
            "inputStage" : {
                "stage" : "IXSCAN",
                "nReturned" : 0,
                "executionTimeMillisEstimate" : 0,
                "works" : 4506,
                "advanced" : 0,
                "needTime" : 4505,
                "needYield" : 0,
                "saveState" : 35,
                "restoreState" : 35,
                "isEOF" : 1,
                "invalidates" : 0,
                "keyPattern" : {
                    "gender" : 1.0,
                    "preferences.feed.gender" : 1.0,
                    "age" : 1.0,
                    "feedPrefChangeDay" : 1.0,
                    "location.loc" : "2dsphere"
                },
                "indexName" : "gender_1_preferences.feed.gender_1_age_1_feedPrefChangeDay_1_location.loc_2dsphere",
                "isMultiKey" : false,
                "multiKeyPaths" : {
                    "gender" : [],
                    "preferences.feed.gender" : [],
                    "age" : [],
                    "feedPrefChangeDay" : [],
                    "location.loc" : []
                },
                "isUnique" : false,
                "isSparse" : false,
                "isPartial" : false,
                "indexVersion" : 2,
                "direction" : "forward",
                "indexBounds" : {
                    "gender" : [ 
                        "[2.0, 2.0]"
                    ],
                    "preferences.feed.gender" : [ 
                        "[1.0, 1.0]"
                    ],
                    "age" : [ 
                        "[18.0, 55.0]"
                    ],
                    "feedPrefChangeDay" : [ 
                        "[1553461200.0, 1554066000.0]"
                    ],
                    "location.loc" : [ 
                        "[360287970189639680, 360287970189639680]", 
                        "[378302368699121664, 378302368699121664]", 
                        "[382805968326492160, 382805968326492160]", 
                        "[383931868233334784, 383931868233334784]", 
                        "[384213343210045440, 384213343210045440]", 
                        "[384230935396089856, 384230935396089856]", 
                        "[384235333442600960, 384235333442600960]", 
                        "[384236432954228736, 384236432954228736]", 
                        "[384236432954228737, 384236982710042623]", 
                        "[384266119768178688, 384266119768178688]", 
                        "[384266119768178689, 384274915861200895]", 
                        "[384274915861200897, 384283711954223103]", 
                        "[384283711954223104, 384283711954223104]", 
                        "[384283711954223105, 384318896326311935]", 
                        "[384318896326311937, 384354080698400767]", 
                        "[1080863910568919040, 1080863910568919040]", 
                        "[1134907106097364992, 1134907106097364992]", 
                        "[1148417904979476480, 1148417904979476480]", 
                        "[1151795604700004352, 1151795604700004352]", 
                        "[1152640029630136320, 1152640029630136320]", 
                        "[1152789563211513857, 1152798359304536063]", 
                        "[1152798359304536064, 1152798359304536064]", 
                        "[1152798359304536065, 1152807155397558271]", 
                        "[1152833543676624896, 1152833543676624896]", 
                        "[1152846737816158208, 1152846737816158208]", 
                        "[1152850036351041536, 1152850036351041536]", 
                        "[1152850586106855425, 1152851135862669311]", 
                        "[1152851135862669312, 1152851135862669312]", 
                        "[1152851135862669313, 1152859931955691519]", 
                        "[1152868728048713728, 1152868728048713728]", 
                        "[1152877524141735937, 1152886320234758143]", 
                        "[1152886320234758145, 1152921504606846975]", 
                        "[1152921504606846977, 1152956688978935807]", 
                        "[1152956688978935809, 1152991873351024639]", 
                        "[1152991873351024640, 1152991873351024640]", 
                        "[1152991873351024641, 1152992423106838527]", 
                        "[1152992972862652416, 1152992972862652416]", 
                        "[1152996271397535744, 1152996271397535744]", 
                        "[1153009465537069056, 1153009465537069056]", 
                        "[1153035853816135681, 1153044649909157887]", 
                        "[1153044649909157888, 1153044649909157888]", 
                        "[1153044649909157889, 1153053446002180095]", 
                        "[1153202979583557632, 1153202979583557632]", 
                        "[1154047404513689600, 1154047404513689600]", 
                        "[1157425104234217472, 1157425104234217472]", 
                        "[1170935903116328960, 1170935903116328960]", 
                        "[1224979098644774912, 1224979098644774912]", 
                        "[1921488928515293185, 1921524112887382015]", 
                        "[1921524112887382017, 1921559297259470847]", 
                        "[1921559297259470848, 1921559297259470848]", 
                        "[1921559297259470849, 1921594481631559679]", 
                        "[1921606026503651329, 1921606576259465215]", 
                        "[1921606576259465216, 1921606576259465216]", 
                        "[1921607675771092992, 1921607675771092992]", 
                        "[1921612073817604096, 1921612073817604096]", 
                        "[1921629666003648512, 1921629666003648512]", 
                        "[1921911140980359168, 1921911140980359168]", 
                        "[1923037040887201792, 1923037040887201792]", 
                        "[1927540640514572288, 1927540640514572288]", 
                        "[1945555039024054272, 1945555039024054272]"
                    ]
                },
                "keysExamined" : 4506,
                "seeks" : 4506,
                "dupsTested" : 0,
                "dupsDropped" : 0,
                "seenInvalidated" : 0
            }
        }
    }
}

如您所见,Mongo利用多个索引范围来实现此查询,并且效率更高。

我认为原始查询的效率低是因为Mongo的查询计划器不会检查$or表达式中的内容。我认为应该更聪明地理解表达式中只有一个具有多个范围的字段,并使用多个索引范围来构建查询。可悲的是,事实并非如此。

我的问题:有什么方法可以强制Mongo对查询使用多个索引范围,以使其与本地地理空间查询一样有效?

任何帮助将不胜感激。

谢谢!

0 个答案:

没有答案