MongoDB $或查询更快

时间:2016-03-04 23:59:02

标签: mongodb mongodb-query

我有一个mongo实例,集合中包含16m文档。我正在编写一个查询来搜索其中一个(索引的)字段,我得到了一些奇怪的结果,我无法解释。

如果我直接执行查询:

find({ "$and" : [ { "ipAddr" : { "$regex" : "^01:172"}} , { "active" : true}]}).limit(100).sort({ "_id" : 1})

甚至添加无意义的$或查询:

find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1})

它返回 在71673ms中获取3条记录

但是,如果我使用$或反对自己,如:

find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}} , { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1})

它返回: 在4ms内获取3条记录

所以性能差异很大。通过检查查询上的explain(),我无法确定为什么存在这么大的性能差异。任何人都可以阐明我所缺少的东西或者mongo在这些方面做了哪些不同的做法?

单个$上解释()或者> 60000ms

find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1}).explain()
{
    "queryPlanner" : {
        "plannerVersion" : 1,
        "namespace" : "CLS-TEST.Leases",
        "indexFilterSet" : false,
        "parsedQuery" : {
            "$and" : [
                {
                    "active" : {
                        "$eq" : true
                    }
                },
                {
                    "ipAddr" : /^01:172/
                }
            ]
        },
        "winningPlan" : {
            "stage" : "SORT",
            "sortPattern" : {
                "_id" : 1
            },
            "limitAmount" : 100,
            "inputStage" : {
                "stage" : "SORT_KEY_GENERATOR",
                "inputStage" : {
                    "stage" : "FETCH",
                    "filter" : {
                        "active" : {
                            "$eq" : true
                        }
                    },
                    "inputStage" : {
                        "stage" : "IXSCAN",
                        "keyPattern" : {
                            "ipAddr" : 1
                        },
                        "indexName" : "ipAddr_1",
                        "isMultiKey" : false,
                        "isUnique" : false,
                        "isSparse" : false,
                        "isPartial" : false,
                        "indexVersion" : 1,
                        "direction" : "forward",
                        "indexBounds" : {
                            "ipAddr" : [
                                "[\"01:172\", \"01:173\")",
                                "[/^01:172/, /^01:172/]"
                            ]
                        }
                    }
                }
            }
        },
        "rejectedPlans" : [
            {
                "stage" : "SORT",
                "sortPattern" : {
                    "_id" : 1
                },
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage" : "FETCH",
                        "filter" : {
                            "ipAddr" : /^01:172/
                        },
                        "inputStage" : {
                            "stage" : "IXSCAN",
                            "keyPattern" : {
                                "active" : 1,
                                "sessionId" : 1,
                                "updateTime" : 1
                            },
                            "indexName" : "active_1_sessionId_1_updateTime_1",
                            "isMultiKey" : false,
                            "isUnique" : false,
                            "isSparse" : false,
                            "isPartial" : false,
                            "indexVersion" : 1,
                            "direction" : "forward",
                            "indexBounds" : {
                                "active" : [
                                    "[true, true]"
                                ],
                                "sessionId" : [
                                    "[MinKey, MaxKey]"
                                ],
                                "updateTime" : [
                                    "[MinKey, MaxKey]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage" : "SORT",
                "sortPattern" : {
                    "_id" : 1
                },
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage" : "FETCH",
                        "filter" : {
                            "ipAddr" : /^01:172/
                        },
                        "inputStage" : {
                            "stage" : "IXSCAN",
                            "keyPattern" : {
                                "active" : 1,
                                "clientId" : 1,
                                "startTime" : -1,
                                "_id" : -1
                            },
                            "indexName" : "active_1_clientId_1_startTime_-1__id_-1",
                            "isMultiKey" : false,
                            "isUnique" : false,
                            "isSparse" : false,
                            "isPartial" : false,
                            "indexVersion" : 1,
                            "direction" : "forward",
                            "indexBounds" : {
                                "active" : [
                                    "[true, true]"
                                ],
                                "clientId" : [
                                    "[MinKey, MaxKey]"
                                ],
                                "startTime" : [
                                    "[MaxKey, MinKey]"
                                ],
                                "_id" : [
                                    "[MaxKey, MinKey]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage" : "SORT",
                "sortPattern" : {
                    "_id" : 1
                },
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage" : "FETCH",
                        "inputStage" : {
                            "stage" : "IXSCAN",
                            "keyPattern" : {
                                "active" : 1,
                                "ipAddr" : 1,
                                "startTime" : -1,
                                "_id" : -1
                            },
                            "indexName" : "active_1_ipAddr_1_startTime_-1__id_-1",
                            "isMultiKey" : false,
                            "isUnique" : false,
                            "isSparse" : false,
                            "isPartial" : false,
                            "indexVersion" : 1,
                            "direction" : "forward",
                            "indexBounds" : {
                                "active" : [
                                    "[true, true]"
                                ],
                                "ipAddr" : [
                                    "[\"01:172\", \"01:173\")",
                                    "[/^01:172/, /^01:172/]"
                                ],
                                "startTime" : [
                                    "[MaxKey, MinKey]"
                                ],
                                "_id" : [
                                    "[MaxKey, MinKey]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage" : "SORT",
                "sortPattern" : {
                    "_id" : 1
                },
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage" : "FETCH",
                        "filter" : {
                            "ipAddr" : /^01:172/
                        },
                        "inputStage" : {
                            "stage" : "IXSCAN",
                            "keyPattern" : {
                                "active" : 1,
                                "macAddress" : 1,
                                "startTime" : -1,
                                "_id" : -1
                            },
                            "indexName" : "active_1_macAddress_1_startTime_-1__id_-1",
                            "isMultiKey" : false,
                            "isUnique" : false,
                            "isSparse" : false,
                            "isPartial" : false,
                            "indexVersion" : 1,
                            "direction" : "forward",
                            "indexBounds" : {
                                "active" : [
                                    "[true, true]"
                                ],
                                "macAddress" : [
                                    "[MinKey, MaxKey]"
                                ],
                                "startTime" : [
                                    "[MaxKey, MinKey]"
                                ],
                                "_id" : [
                                    "[MaxKey, MinKey]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage" : "SORT",
                "sortPattern" : {
                    "_id" : 1
                },
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage" : "FETCH",
                        "filter" : {
                            "ipAddr" : /^01:172/
                        },
                        "inputStage" : {
                            "stage" : "IXSCAN",
                            "keyPattern" : {
                                "active" : 1,
                                "remoteId" : 1,
                                "startTime" : -1,
                                "_id" : -1
                            },
                            "indexName" : "active_1_remoteId_1_startTime_-1__id_-1",
                            "isMultiKey" : false,
                            "isUnique" : false,
                            "isSparse" : false,
                            "isPartial" : false,
                            "indexVersion" : 1,
                            "direction" : "forward",
                            "indexBounds" : {
                                "active" : [
                                    "[true, true]"
                                ],
                                "remoteId" : [
                                    "[MinKey, MaxKey]"
                                ],
                                "startTime" : [
                                    "[MaxKey, MinKey]"
                                ],
                                "_id" : [
                                    "[MaxKey, MinKey]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage" : "LIMIT",
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "FETCH",
                    "filter" : {
                        "$and" : [
                            {
                                "active" : {
                                    "$eq" : true
                                }
                            },
                            {
                                "ipAddr" : /^01:172/
                            }
                        ]
                    },
                    "inputStage" : {
                        "stage" : "IXSCAN",
                        "keyPattern" : {
                            "_id" : 1
                        },
                        "indexName" : "_id_",
                        "isMultiKey" : false,
                        "isUnique" : true,
                        "isSparse" : false,
                        "isPartial" : false,
                        "indexVersion" : 1,
                        "direction" : "forward",
                        "indexBounds" : {
                            "_id" : [
                                "[MinKey, MaxKey]"
                            ]
                        }
                    }
                }
            }
        ]
    },
    "serverInfo" : {
        "host" : "",
        "port" : 27017,
        "version" : "3.2.3",
        "gitVersion" : "b326ba837cf6f49d65c2f85e1b70f6f31ece7937"
    },
    "ok" : 1
}

解释()对$或对自己的解释(< 50ms

find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}} , { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1}).explain()
{
    "queryPlanner" : {
        "plannerVersion" : 1,
        "namespace" : "CLS-TEST.Leases",
        "indexFilterSet" : false,
        "parsedQuery" : {
            "$and" : [
                {
                    "$or" : [
                        {
                            "ipAddr" : /^01:172/
                        },
                        {
                            "ipAddr" : /^01:172/
                        }
                    ]
                },
                {
                    "active" : {
                        "$eq" : true
                    }
                }
            ]
        },
        "winningPlan" : {
            "stage" : "SORT",
            "sortPattern" : {
                "_id" : 1
            },
            "limitAmount" : 100,
            "inputStage" : {
                "stage" : "SORT_KEY_GENERATOR",
                "inputStage" : {
                    "stage" : "FETCH",
                    "filter" : {
                        "active" : {
                            "$eq" : true
                        }
                    },
                    "inputStage" : {
                        "stage" : "IXSCAN",
                        "keyPattern" : {
                            "ipAddr" : 1
                        },
                        "indexName" : "ipAddr_1",
                        "isMultiKey" : false,
                        "isUnique" : false,
                        "isSparse" : false,
                        "isPartial" : false,
                        "indexVersion" : 1,
                        "direction" : "forward",
                        "indexBounds" : {
                            "ipAddr" : [
                                "[\"01:172\", \"01:173\")",
                                "[/^01:172/, /^01:172/]"
                            ]
                        }
                    }
                }
            }
        },
        "rejectedPlans" : [
            {
                "stage" : "SORT",
                "sortPattern" : {
                    "_id" : 1
                },
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage" : "FETCH",
                        "filter" : {
                            "$or" : [
                                {
                                    "ipAddr" : /^01:172/
                                },
                                {
                                    "ipAddr" : /^01:172/
                                }
                            ]
                        },
                        "inputStage" : {
                            "stage" : "IXSCAN",
                            "keyPattern" : {
                                "active" : 1,
                                "sessionId" : 1,
                                "updateTime" : 1
                            },
                            "indexName" : "active_1_sessionId_1_updateTime_1",
                            "isMultiKey" : false,
                            "isUnique" : false,
                            "isSparse" : false,
                            "isPartial" : false,
                            "indexVersion" : 1,
                            "direction" : "forward",
                            "indexBounds" : {
                                "active" : [
                                    "[true, true]"
                                ],
                                "sessionId" : [
                                    "[MinKey, MaxKey]"
                                ],
                                "updateTime" : [
                                    "[MinKey, MaxKey]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage" : "SORT",
                "sortPattern" : {
                    "_id" : 1
                },
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage" : "FETCH",
                        "filter" : {
                            "$or" : [
                                {
                                    "ipAddr" : /^01:172/
                                },
                                {
                                    "ipAddr" : /^01:172/
                                }
                            ]
                        },
                        "inputStage" : {
                            "stage" : "IXSCAN",
                            "keyPattern" : {
                                "active" : 1,
                                "clientId" : 1,
                                "startTime" : -1,
                                "_id" : -1
                            },
                            "indexName" : "active_1_clientId_1_startTime_-1__id_-1",
                            "isMultiKey" : false,
                            "isUnique" : false,
                            "isSparse" : false,
                            "isPartial" : false,
                            "indexVersion" : 1,
                            "direction" : "forward",
                            "indexBounds" : {
                                "active" : [
                                    "[true, true]"
                                ],
                                "clientId" : [
                                    "[MinKey, MaxKey]"
                                ],
                                "startTime" : [
                                    "[MaxKey, MinKey]"
                                ],
                                "_id" : [
                                    "[MaxKey, MinKey]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage" : "SORT",
                "sortPattern" : {
                    "_id" : 1
                },
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage" : "FETCH",
                        "filter" : {
                            "$or" : [
                                {
                                    "ipAddr" : /^01:172/
                                },
                                {
                                    "ipAddr" : /^01:172/
                                }
                            ]
                        },
                        "inputStage" : {
                            "stage" : "IXSCAN",
                            "keyPattern" : {
                                "active" : 1,
                                "ipAddr" : 1,
                                "startTime" : -1,
                                "_id" : -1
                            },
                            "indexName" : "active_1_ipAddr_1_startTime_-1__id_-1",
                            "isMultiKey" : false,
                            "isUnique" : false,
                            "isSparse" : false,
                            "isPartial" : false,
                            "indexVersion" : 1,
                            "direction" : "forward",
                            "indexBounds" : {
                                "active" : [
                                    "[true, true]"
                                ],
                                "ipAddr" : [
                                    "[MinKey, MaxKey]"
                                ],
                                "startTime" : [
                                    "[MaxKey, MinKey]"
                                ],
                                "_id" : [
                                    "[MaxKey, MinKey]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage" : "SORT",
                "sortPattern" : {
                    "_id" : 1
                },
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage" : "FETCH",
                        "filter" : {
                            "$or" : [
                                {
                                    "ipAddr" : /^01:172/
                                },
                                {
                                    "ipAddr" : /^01:172/
                                }
                            ]
                        },
                        "inputStage" : {
                            "stage" : "IXSCAN",
                            "keyPattern" : {
                                "active" : 1,
                                "macAddress" : 1,
                                "startTime" : -1,
                                "_id" : -1
                            },
                            "indexName" : "active_1_macAddress_1_startTime_-1__id_-1",
                            "isMultiKey" : false,
                            "isUnique" : false,
                            "isSparse" : false,
                            "isPartial" : false,
                            "indexVersion" : 1,
                            "direction" : "forward",
                            "indexBounds" : {
                                "active" : [
                                    "[true, true]"
                                ],
                                "macAddress" : [
                                    "[MinKey, MaxKey]"
                                ],
                                "startTime" : [
                                    "[MaxKey, MinKey]"
                                ],
                                "_id" : [
                                    "[MaxKey, MinKey]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage" : "SORT",
                "sortPattern" : {
                    "_id" : 1
                },
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage" : "FETCH",
                        "filter" : {
                            "$or" : [
                                {
                                    "ipAddr" : /^01:172/
                                },
                                {
                                    "ipAddr" : /^01:172/
                                }
                            ]
                        },
                        "inputStage" : {
                            "stage" : "IXSCAN",
                            "keyPattern" : {
                                "active" : 1,
                                "remoteId" : 1,
                                "startTime" : -1,
                                "_id" : -1
                            },
                            "indexName" : "active_1_remoteId_1_startTime_-1__id_-1",
                            "isMultiKey" : false,
                            "isUnique" : false,
                            "isSparse" : false,
                            "isPartial" : false,
                            "indexVersion" : 1,
                            "direction" : "forward",
                            "indexBounds" : {
                                "active" : [
                                    "[true, true]"
                                ],
                                "remoteId" : [
                                    "[MinKey, MaxKey]"
                                ],
                                "startTime" : [
                                    "[MaxKey, MinKey]"
                                ],
                                "_id" : [
                                    "[MaxKey, MinKey]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage" : "LIMIT",
                "limitAmount" : 100,
                "inputStage" : {
                    "stage" : "FETCH",
                    "filter" : {
                        "$and" : [
                            {
                                "$or" : [
                                    {
                                        "ipAddr" : /^01:172/
                                    },
                                    {
                                        "ipAddr" : /^01:172/
                                    }
                                ]
                            },
                            {
                                "active" : {
                                    "$eq" : true
                                }
                            }
                        ]
                    },
                    "inputStage" : {
                        "stage" : "IXSCAN",
                        "keyPattern" : {
                            "_id" : 1
                        },
                        "indexName" : "_id_",
                        "isMultiKey" : false,
                        "isUnique" : true,
                        "isSparse" : false,
                        "isPartial" : false,
                        "indexVersion" : 1,
                        "direction" : "forward",
                        "indexBounds" : {
                            "_id" : [
                                "[MinKey, MaxKey]"
                            ]
                        }
                    }
                }
            }
        ]
    },
    "serverInfo" : {
        "host" : "",
        "port" : 27017,
        "version" : "3.2.3",
        "gitVersion" : "b326ba837cf6f49d65c2f85e1b70f6f31ece7937"
    },
    "ok" : 1
}

1 个答案:

答案 0 :(得分:0)

您可能会注意到,没有任何索引选择包含"active""ipAddr"的组合,这将是此处定义的有用索引。

简而言之,“较慢”的查询只使用"ipAddr"的索引,因此需要做更多的工作才能“过滤掉”{ "active": true }条目。

显然,当其他索引选择正在使用带有这些边界的"active"键时,在正则表达式模式上传递给后续过滤器的结果较少。您似乎在这里有相当多的索引,并且它们都不是查询的最佳选择。

我会给你道具至少在任一查询上运行“解释”输出,但如果你仔细观察,你应该看到“慢”查询“错误地”选择"ipAddr"索引认为它是最佳的。它可能不是,但优化者考虑使用“锚定”正则表达式是合理的假设。

$or强制"index intersection",当$or中只有“一个”参数时,它很聪明,不能这样做。 “两个”参数会使这种情况发生,并且优化器通过查找前面有您的其他查询条件("active"值)的索引来进行另一次“猜测”。

这是有道理的,因为现在它正在运行“两个”查询,它将与结果“相交”,因此$or语句之外的任何条件都是最佳选择索引的逻辑选择。

由于这些返回的结果可能较小,因此“过滤”正则表达式匹配比查看所有正则表达式结果并过滤掉“活动”值更快。

因此,为此查询定义的“最佳”索引将是:

.createIndex({ "active": 1, "ipAddr": 1 })

然后来自任一查询的结果将是一致的,当然提供优化器不会被存在的其他索引混淆并选择那个。要强制选择索引,请使用.hint()