我尝试制作一个shell脚本,该脚本会读取下载网址列表,以确定它们是否仍处于活动状态。我不确定我当前的剧本有什么问题,(我对此不熟悉),任何指针都会有很大的帮助!
user @ pc:〜/ test#cat sites.list
http://www.google.com/images/srpr/logo3w.png
http://www.google.com/doesnt.exist
notasite
脚本:
#!/bin/bash
for i in `cat sites.list`
do
wget --spider $i -b
if grep --quiet "200 OK" wget-log; then
echo $i >> ok.txt
else
echo $i >> notok.txt
fi
rm wget-log
done
按原样,脚本将所有内容输出到notok.txt - (第一个google网站应该转到ok.txt)。 但如果我跑:
wget --spider http://www.google.com/images/srpr/logo3w.png -b
然后做:
grep "200 OK" wget-log
它没有任何问题地抓住字符串。我用语法做了什么noob错误?谢谢m8s!
答案 0 :(得分:6)
-b选项是将wget发送到后台,所以你在wget完成之前就是在做grep了。
尝试不使用-b选项:
if wget --spider $i 2>&1 | grep --quiet "200 OK" ; then
答案 1 :(得分:4)
您正在做的事情存在一些问题。
for i in
会遇到包含空格的行的问题。最好使用while read
来读取文件的各行。curl
。所以这是处理这个问题的更好方法......
#!/bin/bash
sitelist="sites.list"
curl="/usr/bin/curl"
# Some errors, for good measure...
if [[ ! -f "$sitelist" ]]; then
echo "ERROR: Sitelist is missing." >&2
exit 1
elif [[ ! -s "$sitelist" ]]; then
echo "ERROR: Sitelist is empty." >&2
exit 1
elif [[ ! -x "$curl" ]]; then
echo "ERROR: I can't work under these conditions." >&2
exit 1
fi
# Allow more advanced pattern matching (for case..esac below)
shopt -s globstar
while read url; do
# remove comments
url=${url%%#*}
# skip empty lines
if [[ -z "$url" ]]; then
continue
fi
# Handle just ftp, http and https.
# We could do full URL pattern matching, but meh.
case "$url" in
@(f|ht)tp?(s)://*)
# Get just the numeric HTTP response code
http_code=$($curl -sL -w '%{http_code}' "$url" -o /dev/null)
case "$http_code" in
200|226)
# You'll get a 226 in ${http_code} from a valid FTP URL.
# If all you really care about is that the response is in the 200's,
# you could match against "2??" instead.
echo "$url" >> ok.txt
;;
*)
# You might want different handling for redirects (301/302).
echo "$url" >> notok.txt
;;
esac
;;
*)
# If we're here, we didn't get a URL we could read.
echo "WARNING: invalid url: $url" >&2
;;
esac
done < "$sitelist"
这是未经测试的。仅用于教育目的。可能含有坚果。