为什么这个$ elemMatch查询不使用我的索引?

时间:2014-08-19 21:19:19

标签: mongodb mongodb-query mongodb-indexes

我的查询:

{
    "unique_contact_method.enrichments": {
        "$not": {
            "$elemMatch": {
                "created_by.name": "fullcontact"
            }
        }
    }
}

我的索引:

{
    v: 1,
    name: "unique_contact_method.enrichments.created_by.name_1",
    key: {
        "unique_contact_method.enrichments.created_by.name": 1
    },
    ns: "app27434806.unique_contact_methods",
    background: true,
    safe: true
}

.explain()结果:

enter image description here

为什么没有索引?

2 个答案:

答案 0 :(得分:1)

此处使用$not运算符是使索引使用无法实现的原因。文档中有一条声明“隐含”这个,如果不是完全清楚的话:

  

“请记住,$ not运算符只影响其他运算符,不能独立检查字段和文档。因此,使用$ not运算符进行逻辑析取,使用$ ne运算符直接测试字段的内容。”< / em>的

基本短语是“无法检查字段”,这意味着它实际上不会“测试”字段的值,因为可以使用索引来完成。一份简单的文件解释了这一点:

{ 
    "_id" : ObjectId("53f3e414deee3a78e47e57e2"), 
    "created" : [ { "name" : "Bill" }, { "name" : "Ted" } ]
}

当然在“created.name”上创建索引。

现在考虑以下查询并解释输出:

db.doctest.find({ "created": { "$elemMatch": { "name": "Bill" } } }).explain()

{
    "cursor" : "BtreeCursor created.name_1",
    "isMultiKey" : true,
    "n" : 1,
    "nscannedObjects" : 1,
    "nscanned" : 1,
    "nscannedObjectsAllPlans" : 1,
    "nscannedAllPlans" : 1,
    "scanAndOrder" : false,
    "indexOnly" : false,
    "nYields" : 0,
    "nChunkSkips" : 0,
    "millis" : 0,
    "indexBounds" : {
            "created.name" : [
                    [
                            "Bill",
                            "Bill"
                    ]
            ]
    },
    "server" : "ubuntu:27017",
    "filterSet" : false
}

只需选择索引并按预期显示索引边界。

请勿使用$not查看此内容,并且我将使用.hint()“强制”索引:

db.doctest.find({ "created": { "$not": { "$elemMatch": { "name": "Bill" } } } }).hint({ "created.name": 1 }).explain()
{
    "cursor" : "BtreeCursor created.name_1",
    "isMultiKey" : true,
    "n" : 0,
    "nscannedObjects" : 1,
    "nscanned" : 2,
    "nscannedObjectsAllPlans" : 1,
    "nscannedAllPlans" : 2,
    "scanAndOrder" : false,
    "indexOnly" : false,
    "nYields" : 0,
    "nChunkSkips" : 0,
    "millis" : 0,
    "indexBounds" : {
            "created.name" : [
                    [
                            {
                                    "$minElement" : 1
                            },
                            {
                                    "$maxElement" : 1
                            }
                    ]
            ]
    },
    "server" : "ubuntu:27017",
    "filterSet" : false
}

这里要看的重要部分是“indexBounds”。这解释了为什么没有提示就不会使用索引,因为简单地说,没有“边界”可供选择。 $not操作基本上说:

  

“查看条件测试的每个值,如果它是真的那么认为它是假的或基本上是相反的”

这里的最终评估是“Ted”不是“Bill”,因此条件成立,但没有办法使用索引“寻找”。

所以这里考虑的是你如何做同样的事情并使用索引?文档中的段落告诉您,为了考虑“字段”,您需要使用$ne运算符:

db.doctest.find({ "created": { "$elemMatch": { "name": { "$ne": "Bill" } } } }).explain()
{
    "cursor" : "BtreeCursor created.name_1",
    "isMultiKey" : true,
    "n" : 1,
    "nscannedObjects" : 1,
    "nscanned" : 2,
    "nscannedObjectsAllPlans" : 1,
    "nscannedAllPlans" : 2,
    "scanAndOrder" : false,
    "indexOnly" : false,
    "nYields" : 0,
    "nChunkSkips" : 0,
    "millis" : 0,
    "indexBounds" : {
            "created.name" : [
                    [
                            {
                                    "$minElement" : 1
                            },
                            "Bill"
                    ],
                    [
                            "Bill",
                            {
                                    "$maxElement" : 1
                            }
                    ]
            ]
    },
    "server" : "ubuntu:27017",
    "filterSet" : false
}

现在“indexBounds”显示索引用于基本上“过滤掉”提供的值。因此索引用于提取除“Bill”之外的任何其他值。

这里的结论是$not具有逻辑用途,但在许多情况下,您实际需要的是$ne。在必须应用$not的情况下,请考虑该字段值的索引不会用于进行比较。

答案 1 :(得分:1)

有时我发现即使操作符$not加入操作,索引也会自动用于查询。让我回忆一下 这个问题在很长一段时间里也困扰了我。我尝试了新的线索并找到了不同的东西。我想我终于找到了答案。如果发现其他不同的东西,欢迎大家在此发表评论。

在mongo shell上运行,V2.6.4

按如下方式初始化数据:

> db.a.drop();  
false

> db.a.insert({_id:1, a:[1,2,3], b:[{x:1, y:2}, {x:4, y:4}], c:1});
WriteResult({ "nInserted" : 1 })
> db.a.insert({_id:2, a:[4,2,3], b:[{x:1, y:2}, {x:4, y:4}], c:1});
WriteResult({ "nInserted" : 1 })

> db.a.ensureIndex({a:1}, {name:"a"});
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1
}
> db.a.ensureIndex({"b.x":1}, {name:"bx"});
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 2,
        "numIndexesAfter" : 3,
        "ok" : 1
}
> db.a.ensureIndex({c:1}, {name:"c"});
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 3,
        "numIndexesAfter" : 4,
        "ok" : 1
}
> db.a.getIndexes();
[
        {
                "v" : 1,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_",
                "ns" : "test.a"
        },
        {
                "v" : 1,
                "key" : {
                        "a" : 1
                },
                "name" : "a",
                "ns" : "test.a"
        },
        {
                "v" : 1,
                "key" : {
                        "b.x" : 1
                },
                "name" : "bx",
                "ns" : "test.a"
        },
        {
                "v" : 1,
                "key" : {
                        "c" : 1
                },
                "name" : "c",
                "ns" : "test.a"
        }
]

> db.a.find();
{ "_id" : 1, "a" : [ 1, 2, 3 ], "b" : [ { "x" : 1, "y" : 2 }, { "x" : 2, "y" : 3 } ], "c" : 1 }
{ "_id" : 2, "a" : [ 4, 2, 3 ], "b" : [ { "x" : 1, "y" : 2 }, { "x" : 4, "y" : 4 } ], "c" : 1 }

此块只是简单地证明即使$not加入查询操作,也会自动正确使用索引。

> db.a.find({c:{$not:{$gte:1}}}).explain();
{
        "cursor" : "BtreeCursor c",
        "isMultiKey" : false,
        "n" : 0,
        "nscannedObjects" : 0,
        "nscanned" : 1,
        "nscannedObjectsAllPlans" : 0,
        "nscannedAllPlans" : 1,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {
                "c" : [
                        [
                                {
                                        "$minElement" : 1
                                },
                                1
                        ],
                        [
                                Infinity,
                                {
                                        "$maxElement" : 1
                                }
                        ]
                ]
        },
        "server" : "Duke-PC:27017",
        "filterSet" : false
}

这是原始问题所提到的风格。索引已自动使用。

> db.a.find({b:{$elemMatch:{x:{$gte:1}}}}).explain();
{
        "cursor" : "BtreeCursor bx",            // attention on this line
        "isMultiKey" : true,
        "n" : 2,
        "nscannedObjects" : 2,
        "nscanned" : 4,
        "nscannedObjectsAllPlans" : 2,
        "nscannedAllPlans" : 4,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 9,
        "indexBounds" : {
                "b.x" : [
                        [
                                1,
                                Infinity
                        ]
                ]
        },
        "server" : "Duke-PC:27017",
        "filterSet" : false
}

$not之前使用运算符$elemMatch时索引不起作用。这是这个问题的核心。

> db.a.find({b:{$not:{$elemMatch:{x:{$gte:1}}}}}).explain();
{
        "cursor" : "BasicCursor",           // attention on this line
        "isMultiKey" : false,
        "n" : 0,
        "nscannedObjects" : 2,
        "nscanned" : 2,
        "nscannedObjectsAllPlans" : 2,
        "nscannedAllPlans" : 2,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "server" : "Duke-PC:27017",
        "filterSet" : false
}

这个块:找到一些方法来解释阵列上索引的机制。
共有两份文件,但nscanned: 6。这告诉我们索引是如何在数组类型上构建的。也就是说,索引节点位于数组的每个元素上,而不是数组本身。我想象字段a上的索引结构是这样的:
BTree: Node(value:1, entry:[entry({_id:1})]), Node(value:2, entry:[entry({_id:1}), entry({_id:2})]), ...
当然,这只是我想象的解释。 :)

> db.a.find({a:{$gte:1}}).explain();
{
        "cursor" : "BtreeCursor a",
        "isMultiKey" : true,
        "n" : 2,
        "nscannedObjects" : 2,
        "nscanned" : 6,                 // attention on this line
        "nscannedObjectsAllPlans" : 2,
        "nscannedAllPlans" : 6,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {
                "a" : [
                        [
                                1,
                                Infinity
                        ]
                ]
        },
        "server" : "Duke-PC:27017",
        "filterSet" : false
}

当使用operator $ not时,相关索引已自动采用。字段&#34; indexBounds&#34;告诉我们$not如何处理查询。

> db.a.find({a:{$not:{$gte:2}}},{_id:0,a:1}).explain();
{
        "cursor" : "BtreeCursor a",
        "isMultiKey" : true,
        "n" : 0,
        "nscannedObjects" : 1,          // attention on this field
        "nscanned" : 2,                 // attention on this field
        "nscannedObjectsAllPlans" : 1,
        "nscannedAllPlans" : 2,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {               // attention on this field
                "a" : [
                        [
                                {
                                        "$minElement" : 1
                                },
                                2
                        ],
                        [
                                Infinity,
                                {
                                        "$maxElement" : 1
                                }
                        ]
                ]
        },
        "server" : "Duke-PC:27017",
        "filterSet" : false
}

插入具有相同字段名称a但不包含数组的新文档。

> db.a.insert({a:1});
WriteResult({ "nInserted" : 1 })
> db.a.find();
{ "_id" : 1, "a" : [ 1, 2, 3 ], "b" : [ { "x" : 1, "y" : 2 }, { "x" : 2, "y" : 3 } ], "c" : 1 }
{ "_id" : 2, "a" : [ 4, 2, 3 ], "b" : [ { "x" : 1, "y" : 2 }, { "x" : 4, "y" : 4 } ], "c" : 1 }
{ "_id" : ObjectId("541e4fcbb65042180c128280"), "a" : 1 }

请阅读此块与上述内容进行比较。

> db.a.find({a:{$not:{$gte:2}}},{_id:0,a:1}).explain();
{
        "cursor" : "BtreeCursor a",
        "isMultiKey" : true,        // This tells engine there are repeated array elements on index.
        "n" : 1,
        "nscannedObjects" : 2,      // The third document should only access the index to fetch data 
                                    // since it has enough information.
                                    // But here engine still read from the collection. My unstanding is the engine 
                                    // can not distinguish whether this index field is an array element or not, 
                                    // so it has to access the collection to find more information.
        "nscanned" : 3,
        "nscannedObjectsAllPlans" : 2,
        "nscannedAllPlans" : 3,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 25,
        "indexBounds" : {
                "a" : [
                        [
                                {
                                        "$minElement" : 1
                                },
                                2
                        ],
                        [
                                Infinity,
                                {
                                        "$maxElement" : 1
                                }
                        ]
                ]
        },
        "server" : "Duke-PC:27017",
        "filterSet" : false
}

结论:

  1. elemMatch非常特别:
    • $elemMatch明确告诉该字段&#34; b&#34;是一个数组。
    • 根据此运算符的查询定义,可以立即返回与查询匹配的任何元素true。但只有完成扫描数组的所有元素而没有找到任何令人满意的元素,才能返回false
    • But index structure (think about my imagination above) on array can not support this kind of operation because engine can not determine which nodes on index are exactly from a certain array, if only by index. This is the most important point to explain this question.
  2. 其他运营商没有这个限制来自他们自己的查询定义,例如$ gte,$ lt,...,因为只有一个匹配可以判断它是否匹配,这可以通过索引直接满足。

  3. 最后,有一种方法可以解决原始问题,但并不完美,因为必须提供整个元素。
    数组字段的索引,而不是元素。

    > db.a.ensureIndex({b:1});
    {
            "createdCollectionAutomatically" : false,
            "numIndexesBefore" : 4,
            "numIndexesAfter" : 5,
            "ok" : 1
    }
    > db.a.find({b:{$ne:{x:2, y:3}}}).explain();
    {
            "cursor" : "BtreeCursor b_1",
            "isMultiKey" : true,
            "n" : 1,
            "nscannedObjects" : 2,
            "nscanned" : 4,
            "nscannedObjectsAllPlans" : 2,
            "nscannedAllPlans" : 4,
            "scanAndOrder" : false,
            "indexOnly" : false,
            "nYields" : 0,
            "nChunkSkips" : 0,
            "millis" : 33,
            "indexBounds" : {
                    "b" : [
                            [
                                    {
                                            "$minElement" : 1
                                    },
                                    {
                                            "x" : 2,
                                            "y" : 3
                                    }
                            ],
                            [
                                    {
                                            "x" : 2,
                                            "y" : 3
                                    },
                                    {
                                            "$maxElement" : 1
                                    }
                            ]
                    ]
            },
            "server" : "Duke-PC:27017",
            "filterSet" : false
    }