我有一些来自先前mongodb实例的数据文件。我要启动mongodb服务器并加载这些数据文件。
$ ll
total 4708880
drwxrwxr-x 3 rshi rshi 12288 Sep 11 08:03 .
drwxr-xr-x 51 rshi rshi 4096 Sep 11 07:51 ..
-rw------- 1 rshi rshi 67108864 Sep 11 07:44 okex.0
-rw------- 1 rshi rshi 134217728 Sep 11 07:44 okex.1
-rw------- 1 rshi rshi 268435456 Sep 11 07:44 okex.2
-rw------- 1 rshi rshi 536870912 Sep 11 07:44 okex.3
-rw------- 1 rshi rshi 1073741824 Sep 11 07:44 okex.4
-rw------- 1 rshi rshi 2146435072 Sep 11 07:44 okex.5
-rw------- 1 rshi rshi 2146435072 Sep 11 07:45 okex.6
-rw------- 1 rshi rshi 16777216 Sep 11 07:45 okex.ns
$
但是,mongod
抱怨索引“ $ exist”无效(应该是“ $ exists”),并拒绝加载数据。这样可以防止我读取数据并将其潜在地导入到正确索引的数据库中。
$ mongod --dbpath .
2018-09-11T08:03:05.264-0700 I CONTROL [initandlisten] MongoDB starting : pid=28583 port=27017 dbpath=. 64-bit host=rshi-7600
2018-09-11T08:03:05.264-0700 I CONTROL [initandlisten] db version v3.6.3
2018-09-11T08:03:05.264-0700 I CONTROL [initandlisten] git version: 9586e557d54ef70f9ca4b43c26892cd55257e1a5
2018-09-11T08:03:05.264-0700 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.1.0g 2 Nov 2017
2018-09-11T08:03:05.264-0700 I CONTROL [initandlisten] allocator: tcmalloc
2018-09-11T08:03:05.264-0700 I CONTROL [initandlisten] modules: none
2018-09-11T08:03:05.264-0700 I CONTROL [initandlisten] build environment:
2018-09-11T08:03:05.264-0700 I CONTROL [initandlisten] distarch: x86_64
2018-09-11T08:03:05.264-0700 I CONTROL [initandlisten] target_arch: x86_64
2018-09-11T08:03:05.265-0700 I CONTROL [initandlisten] options: { storage: { dbPath: "." } }
2018-09-11T08:03:05.265-0700 W - [initandlisten] Detected unclean shutdown - ./mongod.lock is not empty.
2018-09-11T08:03:05.265-0700 I - [initandlisten] Detected data files in . created by the 'mmapv1' storage engine, so setting the active storage engine to 'mmapv1'.
2018-09-11T08:03:05.274-0700 I JOURNAL [initandlisten] journal dir=./journal
2018-09-11T08:03:05.274-0700 I JOURNAL [initandlisten] recover begin
2018-09-11T08:03:05.274-0700 I JOURNAL [initandlisten] info no lsn file in journal/ directory
2018-09-11T08:03:05.274-0700 I JOURNAL [initandlisten] recover lsn: 0
2018-09-11T08:03:05.274-0700 I JOURNAL [initandlisten] recover ./journal/j._0
2018-09-11T08:03:05.281-0700 I JOURNAL [initandlisten] recover applying initial journal section with sequence number 31087
2018-09-11T08:03:05.283-0700 I STORAGE [initandlisten] recover create file ./local.ns 16MB
2018-09-11T08:03:05.421-0700 I STORAGE [initandlisten] recover create file ./local.0 64MB
2018-09-11T08:03:05.932-0700 I JOURNAL [initandlisten] recover cleaning up
2018-09-11T08:03:05.932-0700 I JOURNAL [initandlisten] removeJournalFiles
2018-09-11T08:03:05.932-0700 I JOURNAL [initandlisten] old journal file will be removed: ./journal/j._0
2018-09-11T08:03:06.080-0700 I JOURNAL [initandlisten] recover done
2018-09-11T08:03:06.080-0700 I JOURNAL [initandlisten] preallocating a journal file ./journal/prealloc.0
2018-09-11T08:03:14.521-0700 I JOURNAL [durability] Durability thread started
2018-09-11T08:03:14.521-0700 I JOURNAL [journal writer] Journal writer thread started
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten]
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten]
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten] ** WARNING: This server is bound to localhost.
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten] ** Remote systems will be unable to connect to this server.
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten] ** Start the server with --bind_ip <address> to specify which IP
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten] ** addresses it should serve responses from, or with --bind_ip_all to
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten] ** bind to all interfaces. If this behavior is desired, start the
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten] ** server with --bind_ip 127.0.0.1 to disable this warning.
2018-09-11T08:03:14.544-0700 I CONTROL [initandlisten]
2018-09-11T08:03:14.547-0700 F INDEX [initandlisten] Found an invalid index { v: 1, key: { stimestamp: 1 }, name: "stimestamp", ns: "okex.ltc_btc_full", partialFilterExpression: { stimestamp: { $exist: true } } } on the okex.ltc_btc_full collection: BadValue: unknown operator: $exist
2018-09-11T08:03:14.547-0700 F - [initandlisten] Fatal Assertion 28782 at src/mongo/db/catalog/index_catalog_impl.cpp 174
2018-09-11T08:03:14.548-0700 F - [initandlisten]
***aborting after fassert() failure
$
问题:有什么方法可以加载数据?摆脱索引是可以的。
可能的解决方法如下,但我不知道该怎么做(谷歌搜索,但没有明显结果):
mongod
命令行中使用某些参数忽略(断开的)索引。mongod
之前,使用工具直接修改数据文件以删除索引。mongod
之前使用工具修复索引。侧面说明:我记得我是通过pymongo通过运行以下内容创建此索引的。我倾向于认为这是mongodb和pymongo的错误,它们在运行时接受了无效的索引,但是在重新启动时无法对其进行处理。但这不是这里的重点。重点是“我们如何加载(破碎的)数据”。
coll_full.create_index([('sequence', pymongo.ASCENDING)], name='sequence', partialFilterExpression={'sequence': {'$exist': True}})
编辑:
--noIndexBuildRetry
对我不起作用。
$ mongod --dbpath . --noIndexBuildRetry
2018-09-11T11:40:42.516-0700 I CONTROL [initandlisten] MongoDB starting : pid=29657 port=27017 dbpath=. 64-bit host=rshi-7600
2018-09-11T11:40:42.517-0700 I CONTROL [initandlisten] db version v3.6.3
2018-09-11T11:40:42.517-0700 I CONTROL [initandlisten] git version: 9586e557d54ef70f9ca4b43c26892cd55257e1a5
2018-09-11T11:40:42.517-0700 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.1.0g 2 Nov 2017
2018-09-11T11:40:42.517-0700 I CONTROL [initandlisten] allocator: tcmalloc
2018-09-11T11:40:42.517-0700 I CONTROL [initandlisten] modules: none
2018-09-11T11:40:42.517-0700 I CONTROL [initandlisten] build environment:
2018-09-11T11:40:42.517-0700 I CONTROL [initandlisten] distarch: x86_64
2018-09-11T11:40:42.517-0700 I CONTROL [initandlisten] target_arch: x86_64
2018-09-11T11:40:42.517-0700 I CONTROL [initandlisten] options: { storage: { dbPath: ".", indexBuildRetry: false } }
2018-09-11T11:40:42.517-0700 W - [initandlisten] Detected unclean shutdown - ./mongod.lock is not empty.
2018-09-11T11:40:42.517-0700 I - [initandlisten] Detected data files in . created by the 'mmapv1' storage engine, so setting the active storage engine to 'mmapv1'.
2018-09-11T11:40:42.526-0700 I JOURNAL [initandlisten] journal dir=./journal
2018-09-11T11:40:42.526-0700 I JOURNAL [initandlisten] recover begin
2018-09-11T11:40:42.526-0700 I JOURNAL [initandlisten] info no lsn file in journal/ directory
2018-09-11T11:40:42.526-0700 I JOURNAL [initandlisten] recover lsn: 0
2018-09-11T11:40:42.526-0700 I JOURNAL [initandlisten] recover ./journal/j._0
2018-09-11T11:40:42.527-0700 I JOURNAL [initandlisten] recover cleaning up
2018-09-11T11:40:42.527-0700 I JOURNAL [initandlisten] removeJournalFiles
2018-09-11T11:40:42.527-0700 I JOURNAL [initandlisten] old journal file will be removed: ./journal/j._0
2018-09-11T11:40:42.669-0700 I JOURNAL [initandlisten] recover done
2018-09-11T11:40:42.669-0700 I JOURNAL [initandlisten] preallocating a journal file ./journal/prealloc.0
2018-09-11T11:40:51.108-0700 I JOURNAL [durability] Durability thread started
2018-09-11T11:40:51.108-0700 I JOURNAL [journal writer] Journal writer thread started
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten]
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten]
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten] ** WARNING: This server is bound to localhost.
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten] ** Remote systems will be unable to connect to this server.
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten] ** Start the server with --bind_ip <address> to specify which IP
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten] ** addresses it should serve responses from, or with --bind_ip_all to
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten] ** bind to all interfaces. If this behavior is desired, start the
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten] ** server with --bind_ip 127.0.0.1 to disable this warning.
2018-09-11T11:40:51.133-0700 I CONTROL [initandlisten]
2018-09-11T11:40:51.136-0700 F INDEX [initandlisten] Found an invalid index { v: 1, key: { stimestamp: 1 }, name: "stimestamp", ns: "okex.ltc_btc_full", partialFilterExpression: { stimestamp: { $exist: true } } } on the okex.ltc_btc_full collection: BadValue: unknown operator: $exist
2018-09-11T11:40:51.136-0700 F - [initandlisten] Fatal Assertion 28782 at src/mongo/db/catalog/index_catalog_impl.cpp 174
2018-09-11T11:40:51.136-0700 F - [initandlisten]
***aborting after fassert() failure
$