limit()和sort()命令pymongo和mongodb

时间:2016-03-27 18:25:07

标签: python mongodb pymongo

尽管阅读人们的答案说明排序是先完成的,但证据显示了一些不同的限制,在排序之前完成了限制。有没有办法永远强制排序?

views = mongo.db.view_logging.find().sort([('count', 1)]).limit(10)

无论我使用.sort().limit()还是.limit().sort(),限制都优先。我想知道这是否与pymongo ...

有关

5 个答案:

答案 0 :(得分:21)

根据documentation,无论哪一项在您的命令链中排在第一位,sort()始终会在limit()之前应用。

您还可以研究查询的.explain()结果并查看执行阶段 - 您会发现排序输入阶段会检查所有已过滤的(在您的情况下是集合中的所有文档),然后是限制适用。

让我们来看一个例子。

想象一下,foo数据库的test集合有6个文档:

>>> col = db.foo.test
>>> for doc in col.find():
...     print(doc)
{'time': '2016-03-28 12:12:00', '_id': ObjectId('56f9716ce4b05e6b92be87f2'), 'value': 90}
{'time': '2016-03-28 12:13:00', '_id': ObjectId('56f971a3e4b05e6b92be87fc'), 'value': 82}
{'time': '2016-03-28 12:14:00', '_id': ObjectId('56f971afe4b05e6b92be87fd'), 'value': 75}
{'time': '2016-03-28 12:15:00', '_id': ObjectId('56f971b7e4b05e6b92be87ff'), 'value': 72}
{'time': '2016-03-28 12:16:00', '_id': ObjectId('56f971c0e4b05e6b92be8803'), 'value': 81}
{'time': '2016-03-28 12:17:00', '_id': ObjectId('56f971c8e4b05e6b92be8806'), 'value': 90}

现在,让我们执行不同顺序sort()limit()的查询,并检查结果和解释计划。

排序然后限制:

>>> from pprint import pprint
>>> cursor = col.find().sort([('time', 1)]).limit(3)  
>>> sort_limit_plan = cursor.explain()
>>> pprint(sort_limit_plan)
{u'executionStats': {u'allPlansExecution': [],
                     u'executionStages': {u'advanced': 3,
                                          u'executionTimeMillisEstimate': 0,
                                          u'inputStage': {u'advanced': 6,
                                                          u'direction': u'forward',
                                                          u'docsExamined': 6,
                                                          u'executionTimeMillisEstimate': 0,
                                                          u'filter': {u'$and': []},
                                                          u'invalidates': 0,
                                                          u'isEOF': 1,
                                                          u'nReturned': 6,
                                                          u'needFetch': 0,
                                                          u'needTime': 1,
                                                          u'restoreState': 0,
                                                          u'saveState': 0,
                                                          u'stage': u'COLLSCAN',
                                                          u'works': 8},
                                          u'invalidates': 0,
                                          u'isEOF': 1,
                                          u'limitAmount': 3,
                                          u'memLimit': 33554432,
                                          u'memUsage': 213,
                                          u'nReturned': 3,
                                          u'needFetch': 0,
                                          u'needTime': 8,
                                          u'restoreState': 0,
                                          u'saveState': 0,
                                          u'sortPattern': {u'time': 1},
                                          u'stage': u'SORT',
                                          u'works': 13},
                     u'executionSuccess': True,
                     u'executionTimeMillis': 0,
                     u'nReturned': 3,
                     u'totalDocsExamined': 6,
                     u'totalKeysExamined': 0},
 u'queryPlanner': {u'indexFilterSet': False,
                   u'namespace': u'foo.test',
                   u'parsedQuery': {u'$and': []},
                   u'plannerVersion': 1,
                   u'rejectedPlans': [],
                   u'winningPlan': {u'inputStage': {u'direction': u'forward',
                                                    u'filter': {u'$and': []},
                                                    u'stage': u'COLLSCAN'},
                                    u'limitAmount': 3,
                                    u'sortPattern': {u'time': 1},
                                    u'stage': u'SORT'}},
 u'serverInfo': {u'gitVersion': u'6ce7cbe8c6b899552dadd907604559806aa2e9bd',
                 u'host': u'h008742.mongolab.com',
                 u'port': 53439,
                 u'version': u'3.0.7'}}

限制然后排序:

>>> cursor = col.find().limit(3).sort([('time', 1)])
>>> limit_sort_plan = cursor.explain()
>>> pprint(limit_sort_plan)
{u'executionStats': {u'allPlansExecution': [],
                     u'executionStages': {u'advanced': 3,
                                          u'executionTimeMillisEstimate': 0,
                                          u'inputStage': {u'advanced': 6,
                                                          u'direction': u'forward',
                                                          u'docsExamined': 6,
                                                          u'executionTimeMillisEstimate': 0,
                                                          u'filter': {u'$and': []},
                                                          u'invalidates': 0,
                                                          u'isEOF': 1,
                                                          u'nReturned': 6,
                                                          u'needFetch': 0,
                                                          u'needTime': 1,
                                                          u'restoreState': 0,
                                                          u'saveState': 0,
                                                          u'stage': u'COLLSCAN',
                                                          u'works': 8},
                                          u'invalidates': 0,
                                          u'isEOF': 1,
                                          u'limitAmount': 3,
                                          u'memLimit': 33554432,
                                          u'memUsage': 213,
                                          u'nReturned': 3,
                                          u'needFetch': 0,
                                          u'needTime': 8,
                                          u'restoreState': 0,
                                          u'saveState': 0,
                                          u'sortPattern': {u'time': 1},
                                          u'stage': u'SORT',
                                          u'works': 13},
                     u'executionSuccess': True,
                     u'executionTimeMillis': 0,
                     u'nReturned': 3,
                     u'totalDocsExamined': 6,
                     u'totalKeysExamined': 0},
 u'queryPlanner': {u'indexFilterSet': False,
                   u'namespace': u'foo.test',
                   u'parsedQuery': {u'$and': []},
                   u'plannerVersion': 1,
                   u'rejectedPlans': [],
                   u'winningPlan': {u'inputStage': {u'direction': u'forward',
                                                    u'filter': {u'$and': []},
                                                    u'stage': u'COLLSCAN'},
                                    u'limitAmount': 3,
                                    u'sortPattern': {u'time': 1},
                                    u'stage': u'SORT'}},
 u'serverInfo': {u'gitVersion': u'6ce7cbe8c6b899552dadd907604559806aa2e9bd',
                 u'host': u'h008742.mongolab.com',
                 u'port': 53439,
                 u'version': u'3.0.7'}}

如您所见,在这两种情况下,首先应用排序并影响所有6个文档,然后限制将结果限制为3。

而且,执行计划完全相同

>>> from copy import deepcopy  # just in case
>>> cursor = col.find().sort([('time', 1)]).limit(3)
>>> sort_limit_plan = deepcopy(cursor.explain())
>>> cursor = col.find().limit(3).sort([('time', 1)])
>>> limit_sort_plan = deepcopy(cursor.explain())
>>> sort_limit_plan == limit_sort_plan
True

另见:

答案 1 :(得分:3)

逻辑上它应该是管道中的第一个,但是MongoDB总是在限制之前排序。

在我的测试中,排序操作确实优先,无论它是在跳过之前还是之后。但是,对我来说这似乎是一种非常奇怪的行为。

我的样本数据集是:

[
    {
        "_id" : ObjectId("56f845fea524b4d098e0ef81"), 
        "number" : 48.98052410874508
    }, 
    {
        "_id" : ObjectId("56f845fea524b4d098e0ef82"), 
        "number" : 50.98747461471063
    }, 
    {
        "_id" : ObjectId("56f845fea524b4d098e0ef83"), 
        "number" : 81.32911244349772
    }, 
    {
        "_id" : ObjectId("56f845fea524b4d098e0ef84"), 
        "number" : 87.95549919039071
    }, 
    {
        "_id" : ObjectId("56f845fea524b4d098e0ef85"), 
        "number" : 81.63582683594402
    }, 
    {
        "_id" : ObjectId("56f845fea524b4d098e0ef86"), 
        "number" : 43.25696270026136
    }, 
    {
        "_id" : ObjectId("56f845fea524b4d098e0ef87"), 
        "number" : 88.22046335409453
    }, 
    {
        "_id" : ObjectId("56f845fea524b4d098e0ef88"), 
        "number" : 64.00556739160076
    }, 
    {
        "_id" : ObjectId("56f845fea524b4d098e0ef89"), 
        "number" : 16.09353150244296
    }, 
    {
        "_id" : ObjectId("56f845fea524b4d098e0ef8a"), 
        "number" : 17.46667776660574
    }
]

Python测试代码:

import pymongo

client = pymongo.MongoClient("mongodb://localhost:27017")
database = client.get_database("test")
collection = database.get_collection("collection")

print("----------------[limit -> sort]--------------------------")
result = collection.find().limit(5).sort([("number", pymongo.ASCENDING)])
for r in result:
    print(r)

print("----------------[sort -> limit]--------------------------")
result = collection.find().sort([("number", pymongo.ASCENDING)]).limit(5)
for r in result:
    print(r)

结果:

----------------[limit -> sort]--------------------------
{u'_id': ObjectId('56f845fea524b4d098e0ef89'), u'number': 16.09353150244296}
{u'_id': ObjectId('56f845fea524b4d098e0ef8a'), u'number': 17.46667776660574}
{u'_id': ObjectId('56f845fea524b4d098e0ef86'), u'number': 43.25696270026136}
{u'_id': ObjectId('56f845fea524b4d098e0ef81'), u'number': 48.98052410874508}
{u'_id': ObjectId('56f845fea524b4d098e0ef82'), u'number': 50.98747461471063}
----------------[sort -> limit]--------------------------
{u'_id': ObjectId('56f845fea524b4d098e0ef89'), u'number': 16.09353150244296}
{u'_id': ObjectId('56f845fea524b4d098e0ef8a'), u'number': 17.46667776660574}
{u'_id': ObjectId('56f845fea524b4d098e0ef86'), u'number': 43.25696270026136}
{u'_id': ObjectId('56f845fea524b4d098e0ef81'), u'number': 48.98052410874508}
{u'_id': ObjectId('56f845fea524b4d098e0ef82'), u'number': 50.98747461471063}

答案 2 :(得分:3)

我怀疑,你在sort参数中传递了错误的键。类似“$ key_name”而不仅仅是“key_name”

参考How do you tell Mongo to sort a collection before limiting the results? solution for same problem as yours

答案 3 :(得分:3)

mongodb文档指出skip()方法控制结果集的起点,然后是sort(),并以limit()方法结束。

这与您的代码顺序无关。原因是mongo获取了查询的所有方法,然后按照确切的顺序对skip-sort-limit方法进行排序,然后运行查询。

答案 4 :(得分:0)

已接受的答案对我不起作用,但这确实适用:

last5 = db.collection.find(   {'key': "YOURKEY"},   sort=[( '_id', pymongo.DESCENDING )] ).limit(5)

在find参数的外部限制和内部排序。