我有一个mongoDB 3.4实例,该实例吞噬了EC2机器上的所有RAM,并想知道如何最好地对其进行处理。
这是一个单个实例,它运行一些流量非常低但已积累了一些数据的应用程序。 出于预算原因,我需要保持该实例RAM的大小。
由于该OOM,mongo最近在该计算机上的其他进程崩溃了,我的计算机上只有几MB可用的ram。
$ free -h
total used free shared buffers cached
Mem: 3.9G 3.6G 263M 416K 667M 446M
-/+ buffers/cache: 2.5G 1.3G
Swap: 0B 0B 0B
我可以使用哪些方法?我已经审查了以下内容,但希望对这种情况的任何评论是最好的方法。
限制db.conf中的wiredTiger高速缓存大小,或者
配置选项--wiredTigerEngineConfigString="cache_size=200M"
不知道适合什么尺寸...
我将一些收藏更改为较小且有上限,但我认为我需要运行
db.repairDatabase()
真正看到任何好处
查看每个集合的索引大小,然后删除索引...这似乎很繁琐,必须在每个集合的基础上完成。是否有一些实用程序可以检查多个数据库和集合中的索引?
使用cgroups限制内存 https://www.techpaste.com/2016/04/limit-mongodb-memory-usage/
人们对于这些方法中的任何一种都有什么结果?
到目前为止的某些诊断
> db.serverStatus().wiredTiger.cache
{
"application threads page read from disk to cache count" : 414211,
"application threads page read from disk to cache time (usecs)" : 636568228,
"application threads page write from cache to disk count" : 458323,
"application threads page write from cache to disk time (usecs)" : 9120699,
"bytes belonging to page images in the cache" : 1107448266,
"bytes belonging to the lookaside table in the cache" : 182,
"bytes currently in the cache" : 1224100958,
"bytes not belonging to page images in the cache" : 116652691,
"bytes read into cache" : 12456110599,
"bytes written from cache" : 8781586219,
"checkpoint blocked page eviction" : 3,
"eviction calls to get a page" : 440255,
"eviction calls to get a page found queue empty" : 64661,
"eviction calls to get a page found queue empty after locking" : 3988,
"eviction currently operating in aggressive mode" : 0,
"eviction empty score" : 0,
"eviction passes of a file" : 39052,
"eviction server candidate queue empty when topping up" : 2703,
"eviction server candidate queue not empty when topping up" : 2067,
"eviction server evicting pages" : 0,
"eviction server slept, because we did not make progress with eviction" : 58773,
"eviction server unable to reach eviction goal" : 0,
"eviction state" : 32,
"eviction walk target pages histogram - 0-9" : 5172,
"eviction walk target pages histogram - 10-31" : 31097,
"eviction walk target pages histogram - 128 and higher" : 0,
"eviction walk target pages histogram - 32-63" : 2131,
"eviction walk target pages histogram - 64-128" : 652,
"eviction walks abandoned" : 19448,
"eviction walks gave up because they restarted their walk twice" : 9071,
"eviction walks gave up because they saw too many pages and found no candidates" : 58,
"eviction walks gave up because they saw too many pages and found too few candidates" : 4,
"eviction walks reached end of tree" : 37987,
"eviction walks started from root of tree" : 28845,
"eviction walks started from saved location in tree" : 10207,
"eviction worker thread active" : 4,
"eviction worker thread created" : 0,
"eviction worker thread evicting pages" : 371587,
"eviction worker thread removed" : 0,
"eviction worker thread stable number" : 0,
"failed eviction of pages that exceeded the in-memory maximum count" : 4,
"failed eviction of pages that exceeded the in-memory maximum time (usecs)" : 17,
"files with active eviction walks" : 0,
"files with new eviction walks started" : 28916,
"force re-tuning of eviction workers once in a while" : 0,
"hazard pointer blocked page eviction" : 606,
"hazard pointer check calls" : 372300,
"hazard pointer check entries walked" : 17002211,
"hazard pointer maximum array length" : 9,
"in-memory page passed criteria to be split" : 332,
"in-memory page splits" : 162,
"internal pages evicted" : 114,
"internal pages split during eviction" : 0,
"leaf pages split during eviction" : 434,
"lookaside score" : 0,
"lookaside table entries" : 0,
"lookaside table insert calls" : 0,
"lookaside table remove calls" : 0,
"maximum bytes configured" : 1535115264,
"maximum page size at eviction" : 0,
"modified pages evicted" : 10058,
"modified pages evicted by application threads" : 0,
"overflow pages read into cache" : 0,
"page split during eviction deepened the tree" : 0,
"page written requiring lookaside records" : 0,
"pages currently held in the cache" : 36933,
"pages evicted because they exceeded the in-memory maximum count" : 163,
"pages evicted because they exceeded the in-memory maximum time (usecs)" : 8083,
"pages evicted because they had chains of deleted items count" : 54,
"pages evicted because they had chains of deleted items time (usecs)" : 43123,
"pages evicted by application threads" : 0,
"pages queued for eviction" : 461533,
"pages queued for urgent eviction" : 802,
"pages queued for urgent eviction during walk" : 11,
"pages read into cache" : 415223,
"pages read into cache after truncate" : 53,
"pages read into cache after truncate in prepare state" : 0,
"pages read into cache requiring lookaside entries" : 0,
"pages read into cache requiring lookaside for checkpoint" : 0,
"pages read into cache skipping older lookaside entries" : 0,
"pages read into cache with skipped lookaside entries needed later" : 0,
"pages read into cache with skipped lookaside entries needed later by checkpoint" : 0,
"pages requested from the cache" : 80308407,
"pages seen by eviction walk" : 581486,
"pages selected for eviction unable to be evicted" : 625,
"pages walked for eviction" : 4385068,
"pages written from cache" : 481133,
"pages written requiring in-memory restoration" : 5,
"percentage overhead" : 8,
"tracked bytes belonging to internal pages in the cache" : 4786604,
"tracked bytes belonging to leaf pages in the cache" : 1219314354,
"tracked dirty bytes in the cache" : 0,
"tracked dirty pages in the cache" : 0,
"unmodified pages evicted" : 361519
}
此有用的线程提供了一些提示 Is there any option to limit mongodb memory usage?
> mem.tcmalloc.formattedString
------------------------------------------------
MALLOC: 1708093144 ( 1629.0 MiB) Bytes in use by application
MALLOC: + 36327424 ( 34.6 MiB) Bytes in page heap freelist
MALLOC: + 41100536 ( 39.2 MiB) Bytes in central cache freelist
MALLOC: + 3532640 ( 3.4 MiB) Bytes in transfer cache freelist
MALLOC: + 29304016 ( 27.9 MiB) Bytes in thread cache freelists
MALLOC: + 9396480 ( 9.0 MiB) Bytes in malloc metadata
MALLOC: ------------
MALLOC: = 1827754240 ( 1743.1 MiB) Actual memory used (physical + swap)
MALLOC: + 506347520 ( 482.9 MiB) Bytes released to OS (aka unmapped)
MALLOC: ------------
MALLOC: = 2334101760 ( 2226.0 MiB) Virtual address space used
MALLOC:
MALLOC: 70088 Spans in use
MALLOC: 50 Thread heaps in use
MALLOC: 4096 Tcmalloc page size
------------------------------------------------