尝试使用mongo
命令行工具在MongoDB集合上运行聚合会失败,并在几个小时后出现网络错误。
time mongo 127.0.0.1/db --eval 'db.coll.aggregate([ {$group: { _id: {someField: "$someField", otherField: "$otherField"}, uniqueIDs: {$addToSet: "$_id"}, count: {$sum: 1} } }, {$match: { count: {"$gt": 1} } } ], { allowDiskUse:true, cursor:{} }).toArray()' > res.txt
它可以在我的机器上复制,并且总是在900分钟左右失败:
MongoDB shell version v4.0.2
connecting to: mongodb://127.0.0.1:27017/db
MongoDB server version: 4.0.2
2018-12-21T03:31:48.907+0100 E QUERY [js] Error: error doing query: failed: network error while attempting to run command 'aggregate' on host '127.0.0.1:27017' :
DB.prototype.runCommand@src/mongo/shell/db.js:168:1
DB.prototype.runReadCommand@src/mongo/shell/db.js:139:16
DB.prototype._runAggregate/doAgg<@src/mongo/shell/db.js:239:60
DB.prototype._runAggregate@src/mongo/shell/db.js:242:21
DBCollection.prototype.aggregate@src/mongo/shell/collection.js:1056:12
@(shell eval):1:1
2018-12-21T03:31:49.093+0100 I NETWORK [js] trying reconnect to 127.0.0.1:27017 failed
2018-12-21T03:31:49.132+0100 I NETWORK [js] reconnect 127.0.0.1:27017 failed failed
2018-12-21T03:31:49.132+0100 I QUERY [js] Failed to end session { id: UUID("476dd3d7-12a6-48a9-a327-b273f6997711") } due to SocketException: socket exception [CONNECT_ERROR] server [couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused]
real 901m53.846s
user 4m47.900s
sys 2m18.340s
该馆藏有十亿多份文档。汇总设计时是否没有考虑到这么多文档?足够的磁盘空间用于聚合。