我在共享主机(uberspace.de)上运行git并在执行git push
或git gc
时达到内存限制。他们报告的内存消耗大约为1500 MB。
所以我按照
中的建议https://stackoverflow.com/a/8761464/288568
它似乎工作了一次,但经过大量的提交后它再也无济于事了,内存消耗再次大约为1500MB,git pack进程被杀死了。
我提交了一个120 MB的SQL转储,只有很少的更改(每次大约5行) - 所以打包的存储库只有大约150 MB,但解压缩大约3 GB(我点击了bitbucket的限制)。 / p>
我看到哪个pmap打开了很多包对象 - 它是否会增加高内存?怎么能限制?
最后我得到" pack-objects死于信号15"和来自主持人的邮件,我的过程被杀了。
sh-4.1$ git config --list|grep pack
pack.windowmemory=25m
pack.packsizelimit=25m
pack.threads=1
pack.deltecachesize=25m
sh-4.1$ git gc
Counting objects: 2586, done.
^Zmpressing objects: 15% (163/1085)
[1]+ Stopped(SIGTSTP) git gc
sh-4.1$ bg
[1] git gc &
sh-4.1$ pmap 14190
14190: git pack-objects --keep-true-parents --honor-pack-keep --non-empty --all --reflog --indexed-objects --unpack-unreachable=2.weeks.ago --local --delta-base-offset .git/objects/pack/.tmp-14187-pack
0000000000400000 1848K r-x-- /home/foo/.toast/pkg/git/v2.11.1/1/root/libexec/git-core/git
00000000007ce000 36K rw--- /home/foo/.toast/pkg/git/v2.11.1/1/root/libexec/git-core/git
00000000007d7000 280K rw--- [ anon ]
0000000000da1000 1680K rw--- [ anon ]
00007f735dc5d000 161648K rw--- [ anon ]
00007f7377555000 149580K rw--- [ anon ]
00007f73915ee000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-be75b796793b738f7566a4be06491cfd3ad9c236.pack
00007f73990b6000 125592K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-33af25946a2fcb1c3e284b3093ac0eafa440cf46.pack
00007f73a0b5c000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-dc37fbdaf5ad7f6c317a7c3d5ffecf53daa045b6.pack
00007f73a8624000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-c7d6e0b1e4da6a2aaa5eff3cad27f6fcbb38a5c9.pack
00007f73b00ec000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bbef922979f633183f0bc207b5e52608b36d6434.pack
00007f73b7bb4000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-1468b8f7d29c999297b2411f9b94d86b0cd9965d.pack
00007f73bf67c000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-3ecc274dd42f4783926224457225968b1fe1d95b.pack
00007f73c7144000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bb77ce8e83062bf2d26a3366b5b4e0f77c3a3810.pack
00007f73cec0c000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-937dae9140ac11d33831a697f9347f98baa25894.pack
00007f73d66d4000 125348K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d497de57bd0e623e5338eb553e0940ea2442274f.pack
00007f73de13d000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-188241ed7cb01328a4c32b09b0aff82311e32a1c.pack
00007f73e5c05000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5fc5649f65f8596a6b3dfd7c71d47d34e7b150e5.pack
00007f73ed6cb000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-aeb01e8db0b4d43181626d8b7a412e813a4a8757.pack
00007f73f5191000 120872K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fcebe7c0d63e44015ecc186ed74d45cb998c22ef.pack
00007f73fc79b000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e84a2620505a10807e64727283f608b9661c39c0.pack
00007f7404263000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-56f5137267429f72a6e57ec70a637dc944cfeec7.pack
00007f740bd29000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-564815c1b986e15fa385daf51d6ecc9f44a07679.pack
00007f74137ef000 120868K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-4c28c08cbf0279f71e3c0a97cc91639fd6a8b308.pack
00007f741adf8000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e2d986a0faf6e464ff240b5bb21bb1b446b329f.pack
00007f74228c0000 484K rw--- [ anon ]
00007f7422939000 96840K r---- /usr/lib/locale/locale-archive
00007f74287cb000 8K r-x-- /lib64/libdl-2.12.so
00007f74287cd000 2048K ----- /lib64/libdl-2.12.so
00007f74289cd000 4K r---- /lib64/libdl-2.12.so
00007f74289ce000 4K rw--- /lib64/libdl-2.12.so
00007f74289cf000 1576K r-x-- /lib64/libc-2.12.so
00007f7428b59000 2048K ----- /lib64/libc-2.12.so
00007f7428d59000 16K r---- /lib64/libc-2.12.so
00007f7428d5d000 8K rw--- /lib64/libc-2.12.so
00007f7428d5f000 16K rw--- [ anon ]
00007f7428d63000 92K r-x-- /lib64/libpthread-2.12.so
00007f7428d7a000 2048K ----- /lib64/libpthread-2.12.so
00007f7428f7a000 4K r---- /lib64/libpthread-2.12.so
00007f7428f7b000 4K rw--- /lib64/libpthread-2.12.so
00007f7428f7c000 16K rw--- [ anon ]
00007f7428f80000 28K r-x-- /lib64/librt-2.12.so
00007f7428f87000 2044K ----- /lib64/librt-2.12.so
00007f7429186000 4K r---- /lib64/librt-2.12.so
00007f7429187000 4K rw--- /lib64/librt-2.12.so
00007f7429188000 1768K r-x-- /usr/lib64/libcrypto.so.1.0.1e
00007f7429342000 2044K ----- /usr/lib64/libcrypto.so.1.0.1e
00007f7429541000 108K r---- /usr/lib64/libcrypto.so.1.0.1e
00007f742955c000 48K rw--- /usr/lib64/libcrypto.so.1.0.1e
00007f7429568000 16K rw--- [ anon ]
00007f742956c000 84K r-x-- /lib64/libz.so.1.2.3
00007f7429581000 2044K ----- /lib64/libz.so.1.2.3
00007f7429780000 4K r---- /lib64/libz.so.1.2.3
00007f7429781000 4K rw--- /lib64/libz.so.1.2.3
00007f7429782000 128K r-x-- /lib64/ld-2.12.so
00007f74297a6000 464K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-004517110da3d0eaeea50b4eace06633eab4294c.pack
00007f742981a000 52K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-ee7d89bc7f8e5cd3e36c3655d967dfc8af3e6c7b.pack
00007f7429827000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-0381d5fab5a4f18e02eb18c3e12c5585702c9564.pack
00007f7429828000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-f26f972bc47c3a18cf47493e680c90fa3d5c9046.pack
00007f7429829000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-30fe739ba7e266ddf6f09a29fc2aa49eedd9e90b.pack
00007f742982a000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fb13617b92c223a9d017935c5ed27c9c8b0e62de.pack
00007f742982b000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d4bc55fff1083c9601bafeb49abf378cdc6f0727.pack
00007f742982c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e104e0bffca85b6346ef588c8c63d6ddadc567e3.pack
00007f742982d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5eeddad5db2c0a44f33a3c41cd9216a7af0417df.pack
00007f742982e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fc2614aa21f9284e0b76b6cc3ad9a90a0bae371e.pack
00007f742982f000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-739869658456c4a7627279fceafa7c7c4c408acb.pack
00007f7429831000 20K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-feabd42e4392c52c8864a0397d10a73d7ada6fc8.pack
00007f7429836000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-26462c3338a2e9302852788bad7b827a98748b65.pack
00007f7429838000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-69b7829b456d813335b339545583ac9585ba9faa.pack
00007f742983a000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e87b7b2f517ff2097026c55e98be6d1cec14587.pack
00007f742983c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-eec773d189c951b5768eb85d33390d3d43db1d9f.pack
00007f742983d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-25232ec34a92ab095486707f73fe5ad0a92bcb14.pack
00007f742983e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5c633980ef7d381a4af2e7bf433c1c86154848c1.pack
00007f742983f000 20K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-2ec52679fc048e2db73cedc5bab30680f9b14bc9.pack
00007f7429875000 4K rw--- [ anon ]
00007f7429876000 400K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d18bf8e1b6a421fac697607e204da80b6c4c0370.pack
00007f74298da000 512K rw--- [ anon ]
00007f742995a000 44K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-004517110da3d0eaeea50b4eace06633eab4294c.idx
00007f7429965000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-4c28c08cbf0279f71e3c0a97cc91639fd6a8b308.idx
00007f7429966000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-ee7d89bc7f8e5cd3e36c3655d967dfc8af3e6c7b.idx
00007f7429967000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fcebe7c0d63e44015ecc186ed74d45cb998c22ef.idx
00007f7429968000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-0381d5fab5a4f18e02eb18c3e12c5585702c9564.idx
00007f7429969000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d497de57bd0e623e5338eb553e0940ea2442274f.idx
00007f742996a000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-f26f972bc47c3a18cf47493e680c90fa3d5c9046.idx
00007f742996b000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-33af25946a2fcb1c3e284b3093ac0eafa440cf46.idx
00007f742996c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-30fe739ba7e266ddf6f09a29fc2aa49eedd9e90b.idx
00007f742996d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-56f5137267429f72a6e57ec70a637dc944cfeec7.idx
00007f742996e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fb13617b92c223a9d017935c5ed27c9c8b0e62de.idx
00007f742996f000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-aeb01e8db0b4d43181626d8b7a412e813a4a8757.idx
00007f7429970000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d4bc55fff1083c9601bafeb49abf378cdc6f0727.idx
00007f7429971000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-564815c1b986e15fa385daf51d6ecc9f44a07679.idx
00007f7429972000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e104e0bffca85b6346ef588c8c63d6ddadc567e3.idx
00007f7429973000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5fc5649f65f8596a6b3dfd7c71d47d34e7b150e5.idx
00007f7429974000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5eeddad5db2c0a44f33a3c41cd9216a7af0417df.idx
00007f7429975000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-937dae9140ac11d33831a697f9347f98baa25894.idx
00007f7429976000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fc2614aa21f9284e0b76b6cc3ad9a90a0bae371e.idx
00007f7429977000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-1468b8f7d29c999297b2411f9b94d86b0cd9965d.idx
00007f7429978000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-739869658456c4a7627279fceafa7c7c4c408acb.idx
00007f7429979000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bb77ce8e83062bf2d26a3366b5b4e0f77c3a3810.idx
00007f742997a000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-feabd42e4392c52c8864a0397d10a73d7ada6fc8.idx
00007f742997b000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-dc37fbdaf5ad7f6c317a7c3d5ffecf53daa045b6.idx
00007f742997c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-26462c3338a2e9302852788bad7b827a98748b65.idx
00007f742997d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-3ecc274dd42f4783926224457225968b1fe1d95b.idx
00007f742997e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-69b7829b456d813335b339545583ac9585ba9faa.idx
00007f742997f000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e2d986a0faf6e464ff240b5bb21bb1b446b329f.idx
00007f7429980000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e87b7b2f517ff2097026c55e98be6d1cec14587.idx
00007f7429981000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bbef922979f633183f0bc207b5e52608b36d6434.idx
00007f7429982000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5c633980ef7d381a4af2e7bf433c1c86154848c1.idx
00007f7429983000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e84a2620505a10807e64727283f608b9661c39c0.idx
00007f7429984000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-eec773d189c951b5768eb85d33390d3d43db1d9f.idx
00007f7429985000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-c7d6e0b1e4da6a2aaa5eff3cad27f6fcbb38a5c9.idx
00007f7429986000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-25232ec34a92ab095486707f73fe5ad0a92bcb14.idx
00007f7429987000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-188241ed7cb01328a4c32b09b0aff82311e32a1c.idx
00007f7429988000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-be75b796793b738f7566a4be06491cfd3ad9c236.idx
00007f7429989000 24K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d18bf8e1b6a421fac697607e204da80b6c4c0370.idx
00007f742998f000 20K rw--- [ anon ]
00007f7429998000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-2ec52679fc048e2db73cedc5bab30680f9b14bc9.idx
00007f7429999000 28K r--s- /usr/lib64/gconv/gconv-modules.cache
00007f74299a0000 4K rw--- [ anon ]
00007f74299a1000 4K r---- /lib64/ld-2.12.so
00007f74299a2000 4K rw--- /lib64/ld-2.12.so
00007f74299a3000 4K rw--- [ anon ]
00007ffe76939000 84K rw--- [ stack ]
00007ffe7698b000 4K r-x-- [ anon ]
ffffffffff600000 4K r-x-- [ anon ]
total 2809104K
sh-4.1$ error: pack-objects died of signal 15
error: failed to run repack
答案 0 :(得分:3)
我有一些大的存储库,我必须压缩,我没有注意到任何与memrory相关的选项帮助。如果git-pack-objects
必须重新包装某些对象,则必须映射大致相同的大内存。管理它的唯一方法是限制它必须重新包装的对象数量。
在你的情况下,对象已经压缩成小包文件(每次推送一个)。所以git gc
必须将那些小包文件重新打包成大包。您可以尝试在上一次成功重新打包创建的大包文件旁边创建空pack-<HASH>.keep
个文件。那么也许下一个git gc
将不会再尝试对这些对象进行分类。
答案 1 :(得分:0)
可能是解决方案
我观察到,当我选择packSizeLimit非常小时,这是非常有效的 - 它增加了.git目录而不是减少它,这也可能导致达到桶位大小限制(未验证)
1)将repo复制到具有更多RAM的另一台计算机并正确打包
2)使用内存限制将其同步回服务器
3)增加到git gc
4)现在在git gc
之后,回购并没有像疯了一样增长
5)提交数据库转储
6)repo将其大小翻倍
7)直接进行api
8)尺寸变好
9)现在推
让我们看看它在几天之后会如何发展......
答案 2 :(得分:0)
Git 2.18(2018年第二季度)应该有助于检测并减轻过多的内存消耗。
&#34;
git gc
&#34;在一个大型存储库中需要花费大量时间 默认情况下将所有对象重新打包到一个包中 该命令已被教导假装最大的现有packfile 标有&#34;.keep
&#34;这样当物体进入时它就不会受到影响 重新包装其他包装和松散包装。
请参阅commit 5af0504,commit 9806f5a,commit 8fc6776,commit 55dfe13,commit ae4e89e,commit ed7e5fc,commit e9e33ab(2018年4月15日) Nguyễn Thái Ngọc Duy (pclouds
)
(由Junio C Hamano -- gitster
--合并于commit 30b015b,2018年5月23日)
gc --auto:如果没有足够的内存,则排除基本包&#34;重新包装-ad&#34;
pack-objects
可能是一个大记忆力,特别是在大型回购, 大家都知道。
为了避免这个问题,将.keep
文件粘贴在巨型基础包上的建议也是众所周知的。最近的补丁添加了一个选项来做到这一点,但它必须是 手动配置或激活。
此补丁允许git gc --auto
在认为repack -ad
将使用大量内存并因交换或刷新操作系统缓存而开始影响系统时自动激活此模式。
gc --auto
决定基于包对象的估计来做到这一点 内存使用情况,至少对于堆部分而言是非常准确的 是否适合系统内存的一半(这里的假设是为了 运行许多其他应用程序的桌面环境。如果未配置
gc.bigBasePackThreshold
,则此机制仅启动。 如果是,则假定用户已经知道他们想要什么。
git gc
man page现在声明:
如果包的数量超过
gc.autoPackLimit
的值, 然后是现有的包(标有.keep
文件的包除外 或超过gc.bigPackThreshold
限制) 使用-A
选项合并为单个包 &#39; git repack&#39;。如果估算的内存量不足
git repack
运行顺利,gc.bigPackThreshold
未设置,最大 包也将被排除(这相当于运行git gc
与--keep-base-pack
)。
将gc.autoPackLimit
设置为0将禁用自动合并 包。