Question

我开发了一个bash脚本，当循环遍历大量文件时会导致异常内存使用。一段时间后，所有内存都耗尽，系统开始交换，因此无法使用。在代码中放置了一些标记之后，我认为引起该问题的函数是_do_cmd（），它包含在以下简化脚本中。

#!/bin/bash

WORKINGDIR=$(dirname "$0")
SCRIPT=$(basename $0)
LOGFILE=$WORKINGDIR/test-log.txt
FILELIST=$WORKINGDIR/file.list
INDIR=/media/data/incoming
OUTDIR=$WORKINGDIR/Foto/copied

_log() {
  echo -e "[$(date +"%Y-%m-%d %H:%M:%S")]: $*" >> $LOGFILE
}

_do_cmd() {
  local LOGFILE_TMP="$LOGFILE.tmp"
  exec 2> >(tee -a $LOGFILE_TMP)
  _log " ACTION: $@"
  "$@"
  ret=$?
  if [[ $ret -ne 0 ]]; then
      _log "ERROR: Return code $ret"
      grep -v "frame= " "$LOGFILE_TMP" >> $LOGFILE
      rm -f $LOGFILE_TMP
      exit $ret
  fi
  if [ -f $LOGFILE_TMP ]; then rm $LOGFILE_TMP; fi
}

while read F
do
  echo "Before: $(ps -ef | grep $SCRIPT | wc -l)"
  FILE=$(basename $F)
  _do_cmd cp "$INDIR/$FILE" "$OUTDIR"
  echo "After: $(ps -ef | grep $SCRIPT | wc -l)"
done < $FILELIST

当我运行脚本时，我看到如下输出：

$ ./test-mem.sh
Before: 3
After: 4
Before: 4
After: 5
Before: 5
After: 6
Before: 6
After: 7
Before: 7
After: 8
Before: 8
After: 9
Before: 9
After: 10
Before: 10
After: 11
Before: 11
After: 12
Before: 12
After: 13
Before: 13
After: 14
Before: 14
After: 15
Before: 15
After: 16
Before: 16
After: 17
Before: 17
After: 18
Before: 18
After: 19
Before: 19
After: 20
Before: 20
After: 21
^C

在执行期间查看正在运行的进程，我发现在执行期间我的脚本实例数不断增加：

$ watch -n 1 "ps -ef | grep test-mem.sh"
Every 1,0s: ps -ef | grep test-mem.sh                                                          Wed Apr  4 10:23:32 2018

user     4117  4104  0 10:23 pts/1    00:00:00 watch -n 1 ps -ef | grep test-mem.sh
user     4877  1309 11 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     4885  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     4899  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     4913  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     4927  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     4941  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     4955  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     4969  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     4983  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     4997  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5011  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5025  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5043  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5057  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5071  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5085  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5099  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5113  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5127  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5141  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5155  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5169  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5183  4877  0 10:23 pts/0    00:00:00 /bin/bash ./test-mem.sh
user     5304  4117  0 10:23 pts/1    00:00:00 watch -n 1 ps -ef | grep test-mem.sh
user     5305  5304  0 10:23 pts/1    00:00:00 sh -c ps -ef | grep test-mem.sh
user     5307  5305  0 10:23 pts/1    00:00:00 grep test-mem.sh

函数_do_cmd（）的目的是运行命令，捕获其错误输出，并且只有在出现错误的情况下，才将其存储到日志文件并退出到shell。

任何人都可以帮助我理解为什么在每次_do_cmd执行后我都在系统中运行了一个新的test-mem.sh实例？

提前致谢。

Answer 1

好的，我找到了解决方案。使用此审查功能，＆＃34;之前＆＃34;和＃34;之后＆＃34;值保持稳定，内存使用量越来越多地停止增加。

_do_cmd() {
  local LOGFILE_TMP="$LOGFILE.tmp"
  _log " ACTION: $@"
  "$@" 2> >(tee -a $LOGFILE_TMP)
  ret=$?
  if [[ $ret -ne 0 ]]; then
      _log "ERROR: Return code $ret"
      grep -v "frame= " "$LOGFILE_TMP" >> $LOGFILE
      rm -f $LOGFILE_TMP
      exit $ret
  fi
  if [ -f $LOGFILE_TMP ]; then rm $LOGFILE_TMP; fi
}

问题可能是由于exec命令的使用，导致进程替换在后台运行。

bash脚本中的内存使用情况

1 个答案: