我正在使用EC2实例来运行节点应用程序。一段时间后,我登录到服务器,只是发现服务器磁盘空间不足。调试后,我意识到日志正在占用空间。我删除了3.3Gb日志文件。但是,即使清除后也没有空间。我该怎么办?
这是我运行的命令:
ubuntu@app1:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 488M 0 488M 0% /dev
tmpfs 100M 11M 89M 11% /run
/dev/xvda1 7.7G 7.7G 0 100% /
tmpfs 496M 8.0K 496M 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 496M 0 496M 0% /sys/fs/cgroup
tmpfs 100M 0 100M 0% /run/user/1001
tmpfs 100M 0 100M 0% /run/user/1000
ubuntu@app1:~$ sudo du -h --max-depth=1 / | sort -n
0 /proc
0 /sys
4.0K /lib64
4.0K /media
4.0K /mnt
4.0K /srv
8.0K /dev
8.0K /snap
16K /lost+found
24K /root
800K /tmp
6.4M /etc
11M /run
14M /sbin
16M /bin
246M /boot
331M /home
397M /opt
429M /var
538M /lib
2.1G /usr
3.7G /data
7.7G /
我在/data
中删除了一个3.3G日志文件,然后再次运行du
ubuntu@app1:~$ sudo du -h --max-depth=1 / | sort -h
0 /proc
0 /sys
4.0K /lib64
4.0K /media
4.0K /mnt
4.0K /srv
8.0K /dev
8.0K /snap
16K /lost+found
24K /root
800K /tmp
6.4M /etc
11M /run
14M /sbin
16M /bin
246M /boot
331M /home
352M /data
397M /opt
429M /var
538M /lib
2.1G /usr
4.4G /
ubuntu@app1:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 488M 0 488M 0% /dev
tmpfs 100M 11M 89M 11% /run
/dev/xvda1 7.7G 7.7G 0 100% /
tmpfs 496M 8.0K 496M 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 496M 0 496M 0% /sys/fs/cgroup
tmpfs 100M 0 100M 0% /run/user/1001
tmpfs 100M 0 100M 0% /run/user/1000
尽管/data
目录现在减少到352M,但是df
仍然显示100%的磁盘利用率。我想念什么?
请参考此答案https://unix.stackexchange.com/a/253655/47050,这是strace
的输出
ubuntu@app1:~$ strace -e statfs df /
statfs("/", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=2016361, f_bfree=4096, f_bavail=0, f_files=1024000, f_ffree=617995, f_fsid={2136106470, -680157247}, f_namelen=255, f_frsize=4096, f_flags=4128}) = 0
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda1 8065444 8049060 0 100% /
+++ exited with 0 +++
更新
我跑了
sudo lsof | grep deleted
找到
node\x20/ 22318 deploy 12w REG 202,1 3541729280 791684 /data/app/shared/logs/production.log (deleted)
node\x20/ 22318 deploy 13w REG 202,1 3541729280 791684 /data/app/shared/logs/production.log (deleted)
node\x20/ 22318 deploy 14w REG 202,1 3541729280 791684 /data/app/shared/logs/production.log (deleted)
node\x20/ 22318 deploy 15w REG 202,1 3541729280 791684 /data/app/shared/logs/production.log (deleted)
node\x20/ 22318 deploy 16w REG 202,1 3541729280 791684 /data/app/shared/logs/production.log (deleted)
如何释放这些文件?
更新2
ubuntu@app1:~$ sudo ls -l /proc/22318/fd
total 0
lrwx------ 1 deploy deploy 64 Apr 6 10:05 0 -> socket:[74749956]
lrwx------ 1 deploy deploy 64 Apr 6 10:05 1 -> socket:[74749958]
lr-x------ 1 deploy deploy 64 Apr 6 10:05 10 -> /dev/null
l-wx------ 1 deploy deploy 64 Apr 6 10:05 12 -> /data/app/shared/logs/production.log (deleted)
l-wx------ 1 deploy deploy 64 Apr 6 10:05 13 -> /data/app/shared/logs/production.log (deleted)
l-wx------ 1 deploy deploy 64 Apr 6 10:05 14 -> /data/app/shared/logs/production.log (deleted)
l-wx------ 1 deploy deploy 64 Apr 6 10:05 15 -> /data/app/shared/logs/production.log (deleted)
l-wx------ 1 deploy deploy 64 Apr 6 10:05 16 -> /data/app/shared/logs/production.log (deleted)
l-wx------ 1 deploy deploy 64 Apr 6 10:05 17 -> /data/app/shared/logs/production.log (deleted)
l-wx------ 1 deploy deploy 64 Apr 6 10:05 18 -> /data/app/shared/logs/production.log (deleted)
l-wx------ 1 deploy deploy 64 Apr 6 10:05 19 -> /data/app/shared/logs/production.log (deleted)
lrwx------ 1 deploy deploy 64 Apr 6 10:05 2 -> socket:[74749960]
l-wx------ 1 deploy deploy 64 Apr 6 10:05 20 -> /data/app/shared/logs/production.log (deleted)
lrwx------ 1 deploy deploy 64 Apr 6 10:05 21 -> socket:[74750302]
lrwx------ 1 deploy deploy 64 Apr 6 10:05 22 -> socket:[74750303]
lrwx------ 1 deploy deploy 64 Apr 6 10:05 3 -> socket:[74749962]
lrwx------ 1 deploy deploy 64 Apr 6 10:05 4 -> anon_inode:[eventpoll]
lr-x------ 1 deploy deploy 64 Apr 6 10:05 5 -> pipe:[74749978]
l-wx------ 1 deploy deploy 64 Apr 6 10:05 6 -> pipe:[74749978]
lr-x------ 1 deploy deploy 64 Apr 6 10:05 7 -> pipe:[74749979]
l-wx------ 1 deploy deploy 64 Apr 6 10:05 8 -> pipe:[74749979]
lrwx------ 1 deploy deploy 64 Apr 6 10:05 9 -> anon_inode:[eventfd]
ubuntu@app1:~$ ps aux | grep node
deploy 22318 0.0 12.7 1277192 129844 ? Ssl 2019 173:38 node /data/app/releases/20180904094535/app.js
ubuntu 30665 0.0 0.0 12944 972 pts/0 S+ 10:09 0:00 grep --color=auto node
答案 0 :(得分:0)
文件由节点应用程序保留。确定使用:
sudo lsof | grep deleted
重新启动节点应用程序解决了我的问题。
查找节点进程ID ps aux | grep node
。然后使用kill -9 <process_id>
杀死节点服务器。最后重启节点服务器。在我的情况下,pm2
自动重启了节点。