我有一个mongo实例,集合中包含16m文档。我正在编写一个查询来搜索其中一个(索引的)字段,我得到了一些奇怪的结果,我无法解释。
如果我直接执行查询:
find({ "$and" : [ { "ipAddr" : { "$regex" : "^01:172"}} , { "active" : true}]}).limit(100).sort({ "_id" : 1})
甚至添加无意义的$或查询:
find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1})
它返回 在71673ms中获取3条记录
但是,如果我使用$或反对自己,如:
find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}} , { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1})
它返回: 在4ms内获取3条记录
所以性能差异很大。通过检查查询上的explain(),我无法确定为什么存在这么大的性能差异。任何人都可以阐明我所缺少的东西或者mongo在这些方面做了哪些不同的做法?
单个$上解释()或者> 60000ms
find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1}).explain()
{
"queryPlanner" : {
"plannerVersion" : 1,
"namespace" : "CLS-TEST.Leases",
"indexFilterSet" : false,
"parsedQuery" : {
"$and" : [
{
"active" : {
"$eq" : true
}
},
{
"ipAddr" : /^01:172/
}
]
},
"winningPlan" : {
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"active" : {
"$eq" : true
}
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"ipAddr" : 1
},
"indexName" : "ipAddr_1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"ipAddr" : [
"[\"01:172\", \"01:173\")",
"[/^01:172/, /^01:172/]"
]
}
}
}
}
},
"rejectedPlans" : [
{
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"ipAddr" : /^01:172/
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"active" : 1,
"sessionId" : 1,
"updateTime" : 1
},
"indexName" : "active_1_sessionId_1_updateTime_1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"active" : [
"[true, true]"
],
"sessionId" : [
"[MinKey, MaxKey]"
],
"updateTime" : [
"[MinKey, MaxKey]"
]
}
}
}
}
},
{
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"ipAddr" : /^01:172/
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"active" : 1,
"clientId" : 1,
"startTime" : -1,
"_id" : -1
},
"indexName" : "active_1_clientId_1_startTime_-1__id_-1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"active" : [
"[true, true]"
],
"clientId" : [
"[MinKey, MaxKey]"
],
"startTime" : [
"[MaxKey, MinKey]"
],
"_id" : [
"[MaxKey, MinKey]"
]
}
}
}
}
},
{
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"active" : 1,
"ipAddr" : 1,
"startTime" : -1,
"_id" : -1
},
"indexName" : "active_1_ipAddr_1_startTime_-1__id_-1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"active" : [
"[true, true]"
],
"ipAddr" : [
"[\"01:172\", \"01:173\")",
"[/^01:172/, /^01:172/]"
],
"startTime" : [
"[MaxKey, MinKey]"
],
"_id" : [
"[MaxKey, MinKey]"
]
}
}
}
}
},
{
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"ipAddr" : /^01:172/
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"active" : 1,
"macAddress" : 1,
"startTime" : -1,
"_id" : -1
},
"indexName" : "active_1_macAddress_1_startTime_-1__id_-1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"active" : [
"[true, true]"
],
"macAddress" : [
"[MinKey, MaxKey]"
],
"startTime" : [
"[MaxKey, MinKey]"
],
"_id" : [
"[MaxKey, MinKey]"
]
}
}
}
}
},
{
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"ipAddr" : /^01:172/
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"active" : 1,
"remoteId" : 1,
"startTime" : -1,
"_id" : -1
},
"indexName" : "active_1_remoteId_1_startTime_-1__id_-1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"active" : [
"[true, true]"
],
"remoteId" : [
"[MinKey, MaxKey]"
],
"startTime" : [
"[MaxKey, MinKey]"
],
"_id" : [
"[MaxKey, MinKey]"
]
}
}
}
}
},
{
"stage" : "LIMIT",
"limitAmount" : 100,
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"$and" : [
{
"active" : {
"$eq" : true
}
},
{
"ipAddr" : /^01:172/
}
]
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"_id" : 1
},
"indexName" : "_id_",
"isMultiKey" : false,
"isUnique" : true,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"_id" : [
"[MinKey, MaxKey]"
]
}
}
}
}
]
},
"serverInfo" : {
"host" : "",
"port" : 27017,
"version" : "3.2.3",
"gitVersion" : "b326ba837cf6f49d65c2f85e1b70f6f31ece7937"
},
"ok" : 1
}
解释()对$或对自己的解释(< 50ms
find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}} , { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1}).explain()
{
"queryPlanner" : {
"plannerVersion" : 1,
"namespace" : "CLS-TEST.Leases",
"indexFilterSet" : false,
"parsedQuery" : {
"$and" : [
{
"$or" : [
{
"ipAddr" : /^01:172/
},
{
"ipAddr" : /^01:172/
}
]
},
{
"active" : {
"$eq" : true
}
}
]
},
"winningPlan" : {
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"active" : {
"$eq" : true
}
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"ipAddr" : 1
},
"indexName" : "ipAddr_1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"ipAddr" : [
"[\"01:172\", \"01:173\")",
"[/^01:172/, /^01:172/]"
]
}
}
}
}
},
"rejectedPlans" : [
{
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"$or" : [
{
"ipAddr" : /^01:172/
},
{
"ipAddr" : /^01:172/
}
]
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"active" : 1,
"sessionId" : 1,
"updateTime" : 1
},
"indexName" : "active_1_sessionId_1_updateTime_1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"active" : [
"[true, true]"
],
"sessionId" : [
"[MinKey, MaxKey]"
],
"updateTime" : [
"[MinKey, MaxKey]"
]
}
}
}
}
},
{
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"$or" : [
{
"ipAddr" : /^01:172/
},
{
"ipAddr" : /^01:172/
}
]
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"active" : 1,
"clientId" : 1,
"startTime" : -1,
"_id" : -1
},
"indexName" : "active_1_clientId_1_startTime_-1__id_-1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"active" : [
"[true, true]"
],
"clientId" : [
"[MinKey, MaxKey]"
],
"startTime" : [
"[MaxKey, MinKey]"
],
"_id" : [
"[MaxKey, MinKey]"
]
}
}
}
}
},
{
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"$or" : [
{
"ipAddr" : /^01:172/
},
{
"ipAddr" : /^01:172/
}
]
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"active" : 1,
"ipAddr" : 1,
"startTime" : -1,
"_id" : -1
},
"indexName" : "active_1_ipAddr_1_startTime_-1__id_-1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"active" : [
"[true, true]"
],
"ipAddr" : [
"[MinKey, MaxKey]"
],
"startTime" : [
"[MaxKey, MinKey]"
],
"_id" : [
"[MaxKey, MinKey]"
]
}
}
}
}
},
{
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"$or" : [
{
"ipAddr" : /^01:172/
},
{
"ipAddr" : /^01:172/
}
]
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"active" : 1,
"macAddress" : 1,
"startTime" : -1,
"_id" : -1
},
"indexName" : "active_1_macAddress_1_startTime_-1__id_-1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"active" : [
"[true, true]"
],
"macAddress" : [
"[MinKey, MaxKey]"
],
"startTime" : [
"[MaxKey, MinKey]"
],
"_id" : [
"[MaxKey, MinKey]"
]
}
}
}
}
},
{
"stage" : "SORT",
"sortPattern" : {
"_id" : 1
},
"limitAmount" : 100,
"inputStage" : {
"stage" : "SORT_KEY_GENERATOR",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"$or" : [
{
"ipAddr" : /^01:172/
},
{
"ipAddr" : /^01:172/
}
]
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"active" : 1,
"remoteId" : 1,
"startTime" : -1,
"_id" : -1
},
"indexName" : "active_1_remoteId_1_startTime_-1__id_-1",
"isMultiKey" : false,
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"active" : [
"[true, true]"
],
"remoteId" : [
"[MinKey, MaxKey]"
],
"startTime" : [
"[MaxKey, MinKey]"
],
"_id" : [
"[MaxKey, MinKey]"
]
}
}
}
}
},
{
"stage" : "LIMIT",
"limitAmount" : 100,
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"$and" : [
{
"$or" : [
{
"ipAddr" : /^01:172/
},
{
"ipAddr" : /^01:172/
}
]
},
{
"active" : {
"$eq" : true
}
}
]
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"_id" : 1
},
"indexName" : "_id_",
"isMultiKey" : false,
"isUnique" : true,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"direction" : "forward",
"indexBounds" : {
"_id" : [
"[MinKey, MaxKey]"
]
}
}
}
}
]
},
"serverInfo" : {
"host" : "",
"port" : 27017,
"version" : "3.2.3",
"gitVersion" : "b326ba837cf6f49d65c2f85e1b70f6f31ece7937"
},
"ok" : 1
}
答案 0 :(得分:0)
您可能会注意到,没有任何索引选择包含"active"
和"ipAddr"
的组合,这将是此处定义的有用索引。
简而言之,“较慢”的查询只使用"ipAddr"
的索引,因此需要做更多的工作才能“过滤掉”{ "active": true }
条目。
显然,当其他索引选择正在使用带有这些边界的"active"
键时,在正则表达式模式上传递给后续过滤器的结果较少。您似乎在这里有相当多的索引,并且它们都不是查询的最佳选择。
我会给你道具至少在任一查询上运行“解释”输出,但如果你仔细观察,你应该看到“慢”查询“错误地”选择"ipAddr"
索引认为它是最佳的。它可能不是,但优化者考虑使用“锚定”正则表达式是合理的假设。
$or
强制"index intersection",当$or
中只有“一个”参数时,它很聪明,不能这样做。 “两个”参数会使这种情况发生,并且优化器通过查找前面有您的其他查询条件("active"
值)的索引来进行另一次“猜测”。
这是有道理的,因为现在它正在运行“两个”查询,它将与结果“相交”,因此$or
语句之外的任何条件都是最佳选择索引的逻辑选择。
由于这些返回的结果可能较小,因此“过滤”正则表达式匹配比查看所有正则表达式结果并过滤掉“活动”值更快。
因此,为此查询定义的“最佳”索引将是:
.createIndex({ "active": 1, "ipAddr": 1 })
然后来自任一查询的结果将是一致的,当然提供优化器不会被存在的其他索引混淆并选择那个。要强制选择索引,请使用.hint()