在我们的系统中,吞吐量大时会调用oom-killer,我们认为大多数内存应由内核驱动程序消耗,但我们找不到专门的使用者,非常感谢任何人都可以为您提供一些建议它。
以下是dmesg的详细日志
[14839.077171] passkey-agent invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
[14839.077187] CPU: 0 PID: 3443 Comm: passkey-agent Tainted: G O 4.1.35-rt41 #1
[14839.077190] Hardware name: LS1043A RDB Board (DT)
[14839.077193] Call trace:
[14839.079644] [<ffff8000000898f4>] dump_backtrace+0x0/0x154
[14839.079650] [<ffff800000089a5c>] show_stack+0x14/0x1c
[14839.079656] [<ffff8000008f3174>] dump_stack+0x90/0xb0
[14839.079663] [<ffff80000013eea4>] dump_header.isra.10+0x88/0x1b8
[14839.079668] [<ffff80000013f5f8>] oom_kill_process+0x210/0x3d4
[14839.079672] [<ffff80000013faec>] __out_of_memory.isra.15+0x330/0x374
[14839.079676] [<ffff80000013fd5c>] out_of_memory+0x5c/0x80
[14839.079682] [<ffff8000001442d8>] __alloc_pages_nodemask+0x55c/0x7c4
[14839.079687] [<ffff80000013e0cc>] filemap_fault+0x188/0x400
[14839.079693] [<ffff80000015f424>] __do_fault+0x3c/0x98
[14839.079698] [<ffff8000001641c8>] handle_mm_fault+0xc28/0x14f8
[14839.079704] [<ffff800000094c04>] do_page_fault+0x224/0x2b4
[14839.079709] [<ffff8000000822a0>] do_mem_abort+0x40/0xa0
[14839.079713] Exception stack(0xffff80001e47be20 to 0xffff80001e47bf50)
[14839.079719] be20: 00000000 00000000 000001f4 00000000 ffffffff ffffffff a6f90990 0000ffff
[14839.079725] be40: ffffffff ffffffff 3b969772 00000000 dbbcc280 0000ffff 00085db0 ffff8000
[14839.079730] be60: 00000000 00000000 000001f4 00000000 ffffffff ffffffff a6f90990 0000ffff
[14839.079736] be80: 1e47bea0 ffff8000 000895f8 ffff8000 00000008 00000000 00085b90 ffff8000
[14839.079742] bea0: dbbcc280 0000ffff 00085c9c ffff8000 00000000 00000000 0ee34088 00000000
[14839.079747] bec0: 00000000 00000000 00000001 00000000 dbbcc2b0 0000ffff 00000000 00000000
[14839.079752] bee0: 00000000 00000000 00000000 00000000 000f4240 00000000 00000000 00000000
[14839.079758] bf00: 00000049 00000000 0000001c 00000000 0000011b 00000000 00000013 00000000
[14839.079763] bf20: 00000028 00000000 00000000 00000000 a71c1c20 0000ffff 00000000 003b9aca
[14839.079767] bf40: a720a990 0000ffff a6f90918 0000ffff
[14839.082683] Mem-Info:
[14839.082700] active_anon:16910 inactive_anon:6202 isolated_anon:0
active_file:15 inactive_file:0 isolated_file:26
unevictable:62887 dirty:0 writeback:0 unstable:0
slab_reclaimable:944 slab_unreclaimable:8027
mapped:5421 shmem:2349 pagetables:527 bounce:0
free:5120 free_pcp:627 free_cma:0
[14839.082719] DMA free:20480kB min:22528kB low:28160kB high:33792kB active_anon:67640kB inactive_anon:24808kB active_file:60kB inactive_file:0kB unevictable:251548kB isolated(anon):0kB isolated(file):104kB present:1046528kB managed:890652kB mlocked:251548kB dirty:0kB writeback:0kB mapped:21684kB shmem:9396kB slab_reclaimable:3776kB slab_unreclaimable:32108kB kernel_stack:6064kB pagetables:2108kB unstable:0kB bounce:0kB free_pcp:2508kB local_pcp:424kB free_cma:0kB writeback_tmp:0kB pages_scanned:208 all_unreclaimable? no
[14839.082723] lowmem_reserve[]: 0 0 0
[14839.082729] DMA: 755*4kB (EM) 486*8kB (UEM) 617*16kB (UEM) 2*32kB (M) 1*64kB (R) 2*128kB (R) 1*256kB (R) 0*512kB 1*1024kB (R) 1*2048kB (R) 0*4096kB = 20492kB
[14839.082752] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[14839.082755] 7756 total pagecache pages
[14839.082760] 0 pages in swap cache
[14839.082763] Swap cache stats: add 0, delete 0, find 0/0
[14839.082765] Free swap = 0kB
[14839.082768] Total swap = 0kB
[14839.082856] 261632 pages RAM
[14839.082858] 0 pages HighMem/MovableOnly
[14839.082861] 34873 pages reserved
[14839.082863] 4096 pages cma reserved
[14839.082867] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
[14839.082890] [ 1353] 0 1353 876 161 7 3 0 -1000 udevd
[14839.082899] [ 1863] 999 1863 695 48 5 3 0 0 dbus-daemon
[14839.082906] [ 1944] 0 1944 833 23 5 3 0 0 syslogd
[14839.082913] [ 1947] 0 1947 833 18 5 3 0 0 klogd
[14839.082919] [ 1990] 0 1990 2307 686 8 2 0 0 php-fpm
[14839.082925] [ 1991] 65534 1991 2307 857 8 2 0 0 php-fpm
[14839.082932] [ 1992] 65534 1992 2307 857 8 2 0 0 php-fpm
[14839.082938] [ 1999] 0 1999 720 31 5 3 0 0 bash
[14839.083042] [ 2001] 0 2001 1083 393 6 3 0 0 start_appli
[14839.083049] [ 2010] 0 2010 849 26 5 3 0 0 getty
[14839.083055] [ 2115] 0 2115 1262 96 6 4 0 -1000 sshd
[14839.083062] [ 3051] 0 3051 2709 210 6 2 0 0 optf_write
[14839.083068] [ 3052] 0 3052 1719 686 7 2 0 0 launcher
[14839.083074] [ 3055] 0 3055 5056 4196 13 2 0 0 globMW0
[14839.083081] [ 3066] 0 3066 10430 6805 27 2 0 0 confd
[14839.083088] [ 3085] 0 3085 9735 7449 23 2 0 0 hal0
[14839.083095] [ 3086] 0 3086 7781 6642 19 2 0 0 SystemMgr
[14839.083102] [ 3087] 0 3087 7455 6372 20 2 0 0 HWMgr
[14839.083108] [ 3088] 0 3088 8319 7118 20 2 0 0 SWMgr
[14839.083115] [ 3089] 0 3089 7824 6696 19 2 0 0 FaultMgr
[14839.083121] [ 3090] 0 3090 7488 6359 20 2 0 0 TSMgr
[14839.083127] [ 3091] 0 3091 7009 6144 20 2 0 0 SecurityMgr
[14839.083133] [ 3092] 0 3092 7736 6337 20 2 0 0 DHCPRelayMgr
[14839.083225] [ 3093] 0 3093 8747 6555 21 2 0 0 ItfMgr
[14839.083232] [ 3094] 0 3094 8192 6686 21 2 0 0 WlanItfMgr
[14839.083239] [ 3095] 0 3095 7602 6518 20 2 0 0 L2Mgr
[14839.083246] [ 3096] 0 3096 7399 6017 20 2 0 0 QoSMgr
[14839.083252] [ 3097] 0 3097 8647 6486 21 2 0 0 L3Mgr
[14839.083258] [ 3098] 0 3098 7482 6356 17 2 0 0 MulticastMgr
[14839.083264] [ 3099] 0 3099 7783 6609 21 2 0 0 DHCPMgr
[14839.083271] [ 3100] 0 3100 6864 6409 16 2 0 0 CallHomeMgr
[14839.083279] [ 3422] 0 3422 472 23 4 3 0 0 hciattach
[14839.083286] [ 3426] 0 3426 1035 50 6 3 0 0 bluetoothd
[14839.083292] [ 3443] 0 3443 2039 112 8 3 0 0 passkey-agent
[14839.083298] [ 3462] 0 3462 3852 2368 11 3 0 0 dhcpd
[14839.083304] [ 3517] 0 3517 860 161 7 3 0 -1000 udevd
[14839.083393] [ 3518] 0 3518 860 161 7 3 0 -1000 udevd
[14839.083400] [ 3650] 0 3650 1629 132 6 3 0 0 wpa_supplicant
[14839.083406] [ 3720] 0 3720 3134 1711 10 3 0 0 dhclient
[14839.083412] [ 3747] 0 3747 891 149 6 3 0 0 zebra
[14839.083419] [ 3751] 0 3751 834 132 7 3 0 0 ripd
[14839.083425] [ 3949] 0 3949 1037 67 6 4 0 0 ntpd
[14839.083431] [ 8000] 0 8000 721 33 5 3 0 0 sh
[14839.083436] Out of memory: Kill process 3085 (hal0) score 32 or sacrifice child
[14839.083447] Killed process 3085 (hal0) total-vm:38940kB, anon-rss:15236kB, file-rss:14560kB
总共有1G内存,我发现平板占用约35M(3776kB + 32108kB),内核卡死为6064kB,active_anon + inactive_anon约为92M(67640kB + 24808kB),并且用户空间内存消耗和往常一样正常。
那剩下的记忆去了哪里?我该如何检查? 例如,如何检查pcie网卡的专用驱动程序消耗了多少内存?