Question

我正在尝试扫描一个总共大约65,000个地址的IP块。我们已被指示使用bash专门使用ICMP数据包并找到并行化的方法。以下是我的想法：

#!/bin/bash
ping() {
  if ping -c 1 -W 5 131.212.$i.$j >/dev/null
  then
      ((++s))
      echo -n "*"
  else
      ((++f))
      echo -n "."
  fi
  ((++j))
  #if j has reached 255, set it to zero and increment i
  if [ $j -gt 255 ]; then
      j=0
      ((++i))
      echo "Pinging 131.212.$i.xx IP Block...\n"
  fi
}

s=0 #number of responses recieved
f=0 #number of failures recieved
i=0 #IP increment 1
j=0 #IP increment 2
curProcs=$(ps | wc -l)
maxProcs=$(getconf OPEN_MAX)
while [ $i -lt 256 ]; do
    curProcs=$(ps | wc -l)
    if [ $curProcs -lt $maxProcs ]; then
      ping &
    else
      sleep 10
    fi
done
echo "Found "$s" responses and "$f" timeouts."
echo /usr/bin/time -l
done

但是，我遇到了以下错误（在macOS上）：

redirection error: cannot duplicate fd: Too many open files

我的理解是我正在超越资源限制，如果现有进程数小于指定的最大值，我试图通过仅启动新的ping进程来解决这个问题，但这并不能解决问题。

感谢您的时间和建议。

修改使用预先存在的工具执行此操作后，有很多好的建议。由于受到学术要求的限制，我最终将ping循环拆分为每个12.34.x.x块的不同进程，尽管丑陋但在5分钟内完成了这一操作。这段代码有很多问题，但对于未来的某个人来说这可能是一个很好的起点：

#!/bin/bash

#############################
#      Ping Subfunction     #
#############################
# blocks with more responses will complete first since worst-case scenerio
# is O(n) if no IPs generate a response
pingSubnet() {
  for ((j = 0 ; j <= 255 ; j++)); do
    # send a single ping with a timeout of 5 sec, piping output to the bitbucket
    if ping -c 1 -W 1 131.212."$i"."$j" >/dev/null
    then
        ((++s))
    else
        ((++f))
    fi
  done
  #echo "Recieved $s responses with $f timeouts in block $i..."
  # output number of success results to the pipe opened in at the start
  echo "$s" >"$pipe"
  exit 0
}

#############################
#   Variable Declaration    #
#############################
start=$(date +%s) #start of execution time
startMem=$(vm_stat | awk '/Pages free/ {print $3}' | awk 'BEGIN { FS = "\." }; {print ($1*0.004092)}' | sed 's/\..*$//');
startCPU=$(top -l 1 | grep "CPU usage" | awk '{print 100-$7;}' | sed 's/\..*$//')
s=0 #number of responses recieved
f=0 #number of failures recieved
i=0 #IP increment 1
j=0 #IP increment 2

#############################
#    Pipe Initialization    #
#############################
# create a pipe for child procs to write to
# child procs inherit runtime environment of parent proc, but cannot
# write back to it (like passing by value in C, but the whole env)
# hence, they need somewhere else to write back to that the parent
# proc can read back in
pipe=/tmp/pingpipe
trap 'rm -f $pipe' EXIT
if [[ ! -p $pipe ]]; then
    mkfifo $pipe
    exec 3<> $pipe
fi

#############################
#     IP Block Iteration    #
#############################
# adding an ampersand to the end forks the command to a separate, backgrounded
# child process. this allows for parellel computation but adds logistical
# challenges since children can't write the parent's variables
echo "Initiating scan processes..."
while [ $i -lt 256 ]; do
      #echo "Beginning 131.212.$i.x block scan..."
      #ping subnet asynchronously
      pingSubnet &
      ((++i))
done
echo "Waiting for scans to complete (this may take up to 5 minutes)..."
peakMem=$(vm_stat | awk '/Pages free/ {print $3}' | awk 'BEGIN { FS = "\." }; {print ($1*0.004092)}' | sed 's/\..*$//')
peakCPU=$(top -l 1 | grep "CPU usage" | awk '{print 100-$7;}' | sed 's/\..*$//')
wait
echo -e "done" >$pipe

#############################
#    Concat Pipe Outputs    #
#############################
# read each line from the pipe we created earlier, adding the number
# of successes up in a variable
success=0
echo "Tallying responses..."
while read -r line <$pipe; do
    if [[ "$line" == 'done' ]]; then
      break
    fi
    success=$((line+success))
done

#############################
#    Output Statistics      #
#############################
echo "Gathering Statistics..."
fail=$((65535-success))
#output program statistics
averageMem=$((peakMem-startMem))
averageCPU=$((peakCPU-startCPU))
end=$(date +%s) #end of execution time
runtime=$((end-start))
echo "Scan completed in $runtime seconds."
echo "Found $success active servers and $fail nonresponsive addresses with a timeout of 1."
echo "Estimated memory usage was $averageMem MB."
echo "Estimated CPU utilization was $averageCPU %"

Answer 1

这应该为您提供 GNU Parallel

的一些想法

parallel --dry-run -j 64 -k ping 131.212.{1}.{2} ::: $(seq 1 3) ::: $(seq 11 13)

ping 131.212.1.11
ping 131.212.1.12
ping 131.212.1.13
ping 131.212.2.11
ping 131.212.2.12
ping 131.212.2.13
ping 131.212.3.11
ping 131.212.3.12
ping 131.212.3.13

-j64一次并行执行64次ping
-dry-run意味着什么都不做，只能展示它会做什么
-k表示保持输出顺序 - （只是这样你才能理解）

:::引入了参数，我用不同的数字（1到3，然后是11到13）重复它们，这样你就可以区分这两个计数器，看看是否生成了所有的排列组合。

Answer 2

不要那样做。

请改用fping。它将比你的程序更有效地进行探测。

$ brew install fping

由于brew的魔力，

将使其可用。

Answer 3

当然它并不像你在上面构建的那样最优，但你可以在后台启动最大允许进程数，等待它们结束并开始下一批，就像这样（除了我是使用sleep 1 s）：

for i in {1..20}             # iterate some
do 
    sleep 1 &                # start in the background
    if ! ((i % 5))           # after every 5th (using mod to detect)
    then 
        wait %1 %2 %3 %4 %5  # wait for all jobs to finish
    fi
done

并行化ping扫描

3 个答案: