Question

我正在运行一个由3个成员组成的传统MongoDB副本集（数据中心A中的member1，数据中心B中的member2和member3）。 member1是当前的PRIMARY，我通过rs.add()添加成员2和3。他们正在执行初始同步并很快成为SECONDARY。一切都很好，两个成员的复制延迟是0秒，直到凌晨2点。

现在：每天早上2点，两个成员都会进入RECOVERING状态并完全停止复制，这导致我在早上时间查看rs.printSlaveReplicationInfo()时的复制延迟时间。凌晨2点左右，我没有大量的插入或维护任务。

我在PRIMARY上获得以下日志条目：

2015-10-09T01:59:38.914+0200 [initandlisten] connection accepted from 192.168.227.209:59905 #11954 (37 connections now open)
2015-10-09T01:59:55.751+0200 [conn11111] warning: Collection dropped or state deleted during yield of CollectionScan
2015-10-09T01:59:55.869+0200 [conn11111] warning: Collection dropped or state deleted during yield of CollectionScan
2015-10-09T01:59:55.870+0200 [conn11111] getmore local.oplog.rs cursorid:1155433944036 ntoreturn:0 keyUpdates:0 numYields:1 locks(micros) r:32168 nreturned:0 reslen:20 134ms
2015-10-09T01:59:55.872+0200 [conn11111] end connection 192.168.227.209:58972 (36 connections now open)

而且，更有趣的是，我在两个SECONDARY上都获得了以下日志条目：

2015-10-09T01:59:55.873+0200 [rsBackgroundSync] repl: old cursor isDead, will initiate a new one
2015-10-09T01:59:55.873+0200 [rsBackgroundSync] replSet syncing to: member1:27017
2015-10-09T01:59:56.065+0200 [rsBackgroundSync] replSet error RS102 too stale to catch up, at least from member1:27017
2015-10-09T01:59:56.066+0200 [rsBackgroundSync] replSet our last optime : Oct  9 01:59:23 5617035b:17f
2015-10-09T01:59:56.066+0200 [rsBackgroundSync] replSet oldest at member1:27017 : Oct  9 01:59:23 5617035b:1af
2015-10-09T01:59:56.066+0200 [rsBackgroundSync] replSet See http://dochub.mongodb.org/core/resyncingaverystalereplicasetmember
2015-10-09T01:59:56.066+0200 [rsBackgroundSync] replSet error RS102 too stale to catch up
2015-10-09T01:59:56.066+0200 [rsBackgroundSync] replSet RECOVERING

这也很引人注目 - 每天凌晨凌晨2点左右，oplog开始“重置”：

configured oplog size:   990MB
log length start to end: 19485secs (5.41hrs)
oplog first event time:  Fri Oct 09 2015 02:00:33 GMT+0200 (CEST)
oplog last event time:   Fri Oct 09 2015 07:25:18 GMT+0200 (CEST)
now:                     Fri Oct 09 2015 07:25:26 GMT+0200 (CEST)

我不确定这是否与问题有某种关联。我也想知道这么小的延迟（Oct 9 01:59:23 5617035b:17f＆lt; - ＆gt; Oct 9 01:59:23 5617035b:1af）会让会员变得陈旧。

这也可能是服务器（VM主机）时间问题还是完全不同？（为什么第一个oplog事件每晚都被“重置”而不是“转移”到现在的时间戳减去24小时？）我可以做些什么来调查和避免？

Answer 1

提高oplog大小应该解决这个问题（根据我们的评论）。

针对遇到此问题的其他人的一些参考资料

Workloads that Might Require a Larger Oplog Size
错误：replSet错误RS102过于陈旧无法赶上link1＆amp; link2

MongoDB SECONDARY在夜间成为RECOVERING

1 个答案: