mongodb如何减少RAM使用

时间:2019-06-23 07:38:30

标签: mongodb indexing ram database-administration

我有一个mongoDB 3.4实例,该实例吞噬了EC2机器上的所有RAM,并想知道如何最好地对其进行处理。

这是一个单个实例,它运行一些流量非常低但已积累了一些数据的应用程序。 出于预算原因,我需要保持该实例RAM的大小。

由于该OOM,mongo最近在该计算机上的其他进程崩溃了,我的计算机上只有几MB可用的ram。

$ free -h
             total       used       free     shared    buffers     cached
Mem:          3.9G       3.6G       263M       416K       667M       446M
-/+ buffers/cache:       2.5G       1.3G
Swap:           0B         0B         0B

我可以使用哪些方法?我已经审查了以下内容,但希望对这种情况的任何评论是最好的方法。

  • 限制db.conf中的wiredTiger高速缓存大小,或者 配置选项--wiredTigerEngineConfigString="cache_size=200M" 不知道适合什么尺寸...

  • 我将一些收藏更改为较小且有上限,但我认为我需要运行 db.repairDatabase()真正看到任何好处

  • 查看每个集合的索引大小,然后删除索引...这似乎很繁琐,必须在每个集合的基础上完成。是否有一些实用程序可以检查多个数据库和集合中的索引?

  • 使用cgroups限制内存 https://www.techpaste.com/2016/04/limit-mongodb-memory-usage/

人们对于这些方法中的任何一种都有什么结果?

到目前为止的某些诊断

> db.serverStatus().wiredTiger.cache
{
    "application threads page read from disk to cache count" : 414211,
    "application threads page read from disk to cache time (usecs)" : 636568228,
    "application threads page write from cache to disk count" : 458323,
    "application threads page write from cache to disk time (usecs)" : 9120699,
    "bytes belonging to page images in the cache" : 1107448266,
    "bytes belonging to the lookaside table in the cache" : 182,
    "bytes currently in the cache" : 1224100958,
    "bytes not belonging to page images in the cache" : 116652691,
    "bytes read into cache" : 12456110599,
    "bytes written from cache" : 8781586219,
    "checkpoint blocked page eviction" : 3,
    "eviction calls to get a page" : 440255,
    "eviction calls to get a page found queue empty" : 64661,
    "eviction calls to get a page found queue empty after locking" : 3988,
    "eviction currently operating in aggressive mode" : 0,
    "eviction empty score" : 0,
    "eviction passes of a file" : 39052,
    "eviction server candidate queue empty when topping up" : 2703,
    "eviction server candidate queue not empty when topping up" : 2067,
    "eviction server evicting pages" : 0,
    "eviction server slept, because we did not make progress with eviction" : 58773,
    "eviction server unable to reach eviction goal" : 0,
    "eviction state" : 32,
    "eviction walk target pages histogram - 0-9" : 5172,
    "eviction walk target pages histogram - 10-31" : 31097,
    "eviction walk target pages histogram - 128 and higher" : 0,
    "eviction walk target pages histogram - 32-63" : 2131,
    "eviction walk target pages histogram - 64-128" : 652,
    "eviction walks abandoned" : 19448,
    "eviction walks gave up because they restarted their walk twice" : 9071,
    "eviction walks gave up because they saw too many pages and found no candidates" : 58,
    "eviction walks gave up because they saw too many pages and found too few candidates" : 4,
    "eviction walks reached end of tree" : 37987,
    "eviction walks started from root of tree" : 28845,
    "eviction walks started from saved location in tree" : 10207,
    "eviction worker thread active" : 4,
    "eviction worker thread created" : 0,
    "eviction worker thread evicting pages" : 371587,
    "eviction worker thread removed" : 0,
    "eviction worker thread stable number" : 0,
    "failed eviction of pages that exceeded the in-memory maximum count" : 4,
    "failed eviction of pages that exceeded the in-memory maximum time (usecs)" : 17,
    "files with active eviction walks" : 0,
    "files with new eviction walks started" : 28916,
    "force re-tuning of eviction workers once in a while" : 0,
    "hazard pointer blocked page eviction" : 606,
    "hazard pointer check calls" : 372300,
    "hazard pointer check entries walked" : 17002211,
    "hazard pointer maximum array length" : 9,
    "in-memory page passed criteria to be split" : 332,
    "in-memory page splits" : 162,
    "internal pages evicted" : 114,
    "internal pages split during eviction" : 0,
    "leaf pages split during eviction" : 434,
    "lookaside score" : 0,
    "lookaside table entries" : 0,
    "lookaside table insert calls" : 0,
    "lookaside table remove calls" : 0,
    "maximum bytes configured" : 1535115264,
    "maximum page size at eviction" : 0,
    "modified pages evicted" : 10058,
    "modified pages evicted by application threads" : 0,
    "overflow pages read into cache" : 0,
    "page split during eviction deepened the tree" : 0,
    "page written requiring lookaside records" : 0,
    "pages currently held in the cache" : 36933,
    "pages evicted because they exceeded the in-memory maximum count" : 163,
    "pages evicted because they exceeded the in-memory maximum time (usecs)" : 8083,
    "pages evicted because they had chains of deleted items count" : 54,
    "pages evicted because they had chains of deleted items time (usecs)" : 43123,
    "pages evicted by application threads" : 0,
    "pages queued for eviction" : 461533,
    "pages queued for urgent eviction" : 802,
    "pages queued for urgent eviction during walk" : 11,
    "pages read into cache" : 415223,
    "pages read into cache after truncate" : 53,
    "pages read into cache after truncate in prepare state" : 0,
    "pages read into cache requiring lookaside entries" : 0,
    "pages read into cache requiring lookaside for checkpoint" : 0,
    "pages read into cache skipping older lookaside entries" : 0,
    "pages read into cache with skipped lookaside entries needed later" : 0,
    "pages read into cache with skipped lookaside entries needed later by checkpoint" : 0,
    "pages requested from the cache" : 80308407,
    "pages seen by eviction walk" : 581486,
    "pages selected for eviction unable to be evicted" : 625,
    "pages walked for eviction" : 4385068,
    "pages written from cache" : 481133,
    "pages written requiring in-memory restoration" : 5,
    "percentage overhead" : 8,
    "tracked bytes belonging to internal pages in the cache" : 4786604,
    "tracked bytes belonging to leaf pages in the cache" : 1219314354,
    "tracked dirty bytes in the cache" : 0,
    "tracked dirty pages in the cache" : 0,
    "unmodified pages evicted" : 361519
}

此有用的线程提供了一些提示 Is there any option to limit mongodb memory usage?

> mem.tcmalloc.formattedString
------------------------------------------------
MALLOC:     1708093144 ( 1629.0 MiB) Bytes in use by application
MALLOC: +     36327424 (   34.6 MiB) Bytes in page heap freelist
MALLOC: +     41100536 (   39.2 MiB) Bytes in central cache freelist
MALLOC: +      3532640 (    3.4 MiB) Bytes in transfer cache freelist
MALLOC: +     29304016 (   27.9 MiB) Bytes in thread cache freelists
MALLOC: +      9396480 (    9.0 MiB) Bytes in malloc metadata
MALLOC:   ------------
MALLOC: =   1827754240 ( 1743.1 MiB) Actual memory used (physical + swap)
MALLOC: +    506347520 (  482.9 MiB) Bytes released to OS (aka unmapped)
MALLOC:   ------------
MALLOC: =   2334101760 ( 2226.0 MiB) Virtual address space used
MALLOC:
MALLOC:          70088              Spans in use
MALLOC:             50              Thread heaps in use
MALLOC:           4096              Tcmalloc page size
------------------------------------------------

0 个答案:

没有答案