MongoDB查询索引扫描未按预期过滤

时间:2020-06-04 05:00:35

标签: mongodb indexing

我有一个使用索引的查询,但是在获取期间查找了太多文档。

有问题的索引是:

{
  “v” : 2,
  “key” : {
    “vw” : -1,
    “if” : 1,
    “sa” : 1,
    “dd” : -1,
    “ca” : 1
  },
  “name” : “Viewed_By_Category”,
  “ns” : “redacted”,
  “background” : false
}

有问题的查询

db.stories.find({ 'if': {$ne: true}, 'sa': 2, 'dd': {$ne : null}, 'ca': 11}).skip(3990).limit(30).sort({'vw':-1}).explain('executionStats')*

这是解释输出:

{
        "queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "lushstories.stories",
                "indexFilterSet" : false,
                "parsedQuery" : {
                        "$and" : [
                                {
                                        "ca" : {
                                                "$eq" : 11
                                        }
                                },
                                {
                                        "sa" : {
                                                "$eq" : 2
                                        }
                                },
                                {
                                        "dd" : {
                                                "$not" : {
                                                        "$eq" : null
                                                }
                                        }
                                },
                                {
                                        "if" : {
                                                "$not" : {
                                                        "$eq" : true
                                                }
                                        }
                                }
                        ]
                },
                "winningPlan" : {
                        "stage" : "LIMIT",
                        "limitAmount" : 30,
                        "inputStage" : {
                                "stage" : "SKIP",
                                "skipAmount" : 0,
                                "inputStage" : {
                                        "stage" : "FETCH",
                                        "filter" : {
                                                "$and" : [
                                                        {
                                                                "ca" : {
                                                                        "$eq" : 11
                                                                }
                                                        },
                                                        {
                                                                "sa" : {
                                                                        "$eq" : 2
                                                                }
                                                        },
                                                        {
                                                                "dd" : {
                                                                        "$not" : {
                                                                                "$eq" : null
                                                                        }
                                                                }
                                                        },
                                                        {
                                                                "if" : {
                                                                        "$not" : {
                                                                                "$eq" : true
                                                                        }
                                                                }
                                                        }
                                                ]
                                        },
                                        "inputStage" : {
                                                "stage" : "IXSCAN",
                                                "keyPattern" : {
                                                        "vw" : -1,
                                                        "if" : 1,
                                                        "sa" : 1,
                                                        "dd" : -1,
                                                        "ca" : 1
                                                },
                                                "indexName" : "Viewed_By_Category",
                                                "isMultiKey" : false,
                                                "multiKeyPaths" : {
                                                        "vw" : [ ],
                                                        "if" : [ ],
                                                        "sa" : [ ],
                                                        "dd" : [ ],
                                                        "ca" : [ ]
                                                },
                                                "isUnique" : false,
                                                "isSparse" : false,
                                                "isPartial" : false,
                                                "indexVersion" : 2,
                                                "direction" : "forward",
                                                "indexBounds" : {
                                                        "vw" : [
                                                                "[MaxKey, MinKey]"
                                                        ],
                                                        "if" : [
                                                                "[MinKey, MaxKey]"
                                                        ],
                                                        "sa" : [
                                                                "[MinKey, MaxKey]"
                                                        ],
                                                        "dd" : [
                                                                "[MaxKey, MinKey]"
                                                        ],
                                                        "ca" : [
                                                                "[MinKey, MaxKey]"
                                                        ]
                                                }
                                        }
                                }
                        }
                },
                "rejectedPlans" : [
                        {
                                "stage" : "SKIP",
                                "skipAmount" : 3990,
                                "inputStage" : {
                                        "stage" : "SORT",
                                        "sortPattern" : {
                                                "vw" : -1
                                        },
                                        "limitAmount" : 4020,
                                        "inputStage" : {
                                                "stage" : "SORT_KEY_GENERATOR",
                                                "inputStage" : {
                                                        "stage" : "FETCH",
                                                        "filter" : {
                                                                "dd" : {
                                                                        "$not" : {
                                                                                "$eq" : null
                                                                        }
                                                                }
                                                        },
                                                        "inputStage" : {
                                                                "stage" : "IXSCAN",
                                                                "keyPattern" : {
                                                                        "dd" : -1,
                                                                        "if" : 1,
                                                                        "sa" : 1,
                                                                        "ca" : 1,
                                                                        "ha" : 1
                                                                },
                                                                "indexName" : "Story_Visible_With_Audio",
                                                                "isMultiKey" : false,
                                                                "multiKeyPaths" : {
                                                                        "dd" : [ ],
                                                                        "if" : [ ],
                                                                        "sa" : [ ],
                                                                        "ca" : [ ],
                                                                        "ha" : [ ]
                                                                },
                                                                "isUnique" : false,
                                                                "isSparse" : false,
                                                                "isPartial" : false,
                                                                "indexVersion" : 2,
                                                                "direction" : "forward",
                                                                "indexBounds" : {
                                                                        "dd" : [
                                                                                "[MaxKey, null)",
                                                                                "(null, MinKey]"
                                                                        ],
                                                                        "if" : [
                                                                                "[MinKey, true)",
                                                                                "(true, MaxKey]"
                                                                        ],
                                                                        "sa" : [
                                                                                "[2.0, 2.0]"
                                                                        ],
                                                                        "ca" : [
                                                                                "[11.0, 11.0]"
                                                                        ],
                                                                        "ha" : [
                                                                                "[MinKey, MaxKey]"
                                                                        ]
                                                                }
                                                        }
                                                }
                                        }
                                }
                        }
                ]
        },
        "executionStats" : {
                "executionSuccess" : true,
                "nReturned" : 30,
                "executionTimeMillis" : 5500,
                "totalKeysExamined" : 55743,
                "totalDocsExamined" : 55743,
                "executionStages" : {
                        "stage" : "LIMIT",
                        "nReturned" : 30,
                        "executionTimeMillisEstimate" : 5372,
                        "works" : 55744,
                        "advanced" : 30,
                        "needTime" : 55713,
                        "needYield" : 0,
                        "saveState" : 565,
                        "restoreState" : 565,
                        "isEOF" : 1,
                        "invalidates" : 0,
                        "limitAmount" : 30,
                        "inputStage" : {
                                "stage" : "SKIP",
                                "nReturned" : 30,
                                "executionTimeMillisEstimate" : 5372,
                                "works" : 55743,
                                "advanced" : 30,
                                "needTime" : 55713,
                                "needYield" : 0,
                                "saveState" : 565,
                                "restoreState" : 565,
                                "isEOF" : 0,
                                "invalidates" : 0,
                                "skipAmount" : 0,
                                "inputStage" : {
                                        "stage" : "FETCH",
                                        "filter" : {
                                                "$and" : [
                                                        {
                                                                "ca" : {
                                                                        "$eq" : 11
                                                                }
                                                        },
                                                        {
                                                                "sa" : {
                                                                        "$eq" : 2
                                                                }
                                                        },
                                                        {
                                                                "dd" : {
                                                                        "$not" : {
                                                                                "$eq" : null
                                                                        }
                                                                }
                                                        },
                                                        {
                                                                "if" : {
                                                                        "$not" : {
                                                                                "$eq" : true
                                                                        }
                                                                }
                                                        }
                                                ]
                                        },
                                        "nReturned" : 4020,
                                        "executionTimeMillisEstimate" : 5372,
                                        "works" : 55743,
                                        "advanced" : 4020,
                                        "needTime" : 51723,
                                        "needYield" : 0,
                                        "saveState" : 565,
                                        "restoreState" : 565,
                                        "isEOF" : 0,
                                        "invalidates" : 0,
                                        "docsExamined" : 55743,
                                        "alreadyHasObj" : 0,
                                        "inputStage" : {
                                                "stage" : "IXSCAN",
                                                "nReturned" : 55743,
                                                "executionTimeMillisEstimate" : 80,
                                                "works" : 55743,
                                                "advanced" : 55743,
                                                "needTime" : 0,
                                                "needYield" : 0,
                                                "saveState" : 565,
                                                "restoreState" : 565,
                                                "isEOF" : 0,
                                                "invalidates" : 0,
                                                "keyPattern" : {
                                                        "vw" : -1,
                                                        "if" : 1,
                                                        "sa" : 1,
                                                        "dd" : -1,
                                                        "ca" : 1
                                                },
                                                "indexName" : "Viewed_By_Category",
                                                "isMultiKey" : false,
                                                "multiKeyPaths" : {
                                                        "vw" : [ ],
                                                        "if" : [ ],
                                                        "sa" : [ ],
                                                        "dd" : [ ],
                                                        "ca" : [ ]
                                                },
                                                "isUnique" : false,
                                                "isSparse" : false,
                                                "isPartial" : false,
                                                "indexVersion" : 2,
                                                "direction" : "forward",
                                                "indexBounds" : {
                                                        "vw" : [
                                                                "[MaxKey, MinKey]"
                                                        ],
                                                        "if" : [
                                                                "[MinKey, MaxKey]"
                                                        ],
                                                        "sa" : [
                                                                "[MinKey, MaxKey]"
                                                        ],
                                                        "dd" : [
                                                                "[MaxKey, MinKey]"
                                                        ],
                                                        "ca" : [
                                                                "[MinKey, MaxKey]"
                                                        ]
                                                },
                                                "keysExamined" : 55743,
                                                "seeks" : 1,
                                                "dupsTested" : 0,
                                                "dupsDropped" : 0,
                                                "seenInvalidated" : 0
                                        }
                                }
                        }
                }
        },
        "serverInfo" : {
                "host" : "redacted",
                "port" : 27017,
                "version" : "4.0.9",
                "gitVersion" : "fc525e2d9b0e4bceff5c2201457e564362909765"
        },
        "ok" : 1
}

那么为什么IXSCAN扫描阶段不使用任何谓词进行过滤,而indexBounds都使用[MaxKey,MinKey]

这将返回被送入提取中的记录总数55743。

关于这些索引,我是否不了解?

谢谢

2 个答案:

答案 0 :(得分:2)

之所以发生这种情况,是因为您的查询没有为索引中的第一个字段指定任何匹配条件。

MongoDB不会单独存储索引的每个字段。它是一棵树,其中索引条目的值被串联到其中。为了使用{"vw" : -1, "if" : 1, "sa" : 1, "dd" : -1, "ca" : 1}上的索引来处理该查询,它必须检查vw的每个值。

此索引允许查询执行程序使用索引来满足排序,因此不需要内存中排序。如果您使用"allPlansExecution"重新运行该说明,则可以比较这比需要排序阶段的被拒绝计划要快多少。

查询的最高效索引遵循等值排序范围规则,这意味着应首先列出查询将与精确值匹配的字段,然后列出排序字段,然后列出所有带范围的字段或谓词不相等。

如果您在{"ca" : 1, "sa" : 1, "vw":-1, "dd" : -1, "if" : 1}上创建索引,则应该发现查询完成速度更快。

答案 1 :(得分:0)

Joe的回答很好地解释了这一点,只是在查询中添加了一个条件

 {$ne : null}

它仍然可以在Mongo(取决于版本)中触发过滤器操作。当前版本可以在索引扫描4.2.6中容纳它。