Question

这是我的第一个Python项目，所以它非常基础和基础。我经常要为朋友清除病毒，我经常更新我使用的免费程序。我试图创建一种自动化过程的简单方法，而不是手动下载每个程序。由于我也在努力学习python，我认为这是一个练习的好机会。

问题：

我必须找到带有一些链接的.exe文件。我可以找到正确的URL，但是在尝试下载时出现错误。

有没有办法将所有链接添加到列表中，然后创建一个函数来遍历列表并在每个URL上运行该函数？我已经谷歌了很多，我似乎无法使它工作。也许我没有想出正确的方向？

import urllib, urllib2, re, os
from BeautifulSoup import BeautifulSoup

# Website List
sas = 'http://cdn.superantispyware.com/SUPERAntiSpyware.exe'
tds = 'http://support.kaspersky.com/downloads/utils/tdsskiller.exe'
mbam = 'http://www.bleepingcomputer.com/download/malwarebytes-anti-malware/dl/7/?1'
tr = 'http://www.simplysup.com/tremover/download.html'
urllist = [sas, tr, tds, tr]
urrllist2 = []

# Find exe files to download

match = re.compile('\.exe')
data = urllib2.urlopen(urllist)
page = BeautifulSoup(data)

# Check links
#def findexe():
for link in page.findAll('a'):
    try:
        href = link['href']
        if re.search(match, href):
            urllist2.append(href)

    except KeyError:
        pass

os.chdir(r"C:\_VirusFixes")
urllib.urlretrieve(urllist2, os.path.basename(urllist2))

正如您所看到的，我已将功能注释掉，因为我无法使其正常工作。

我应该放弃列表并单独下载吗？我试图提高效率。

任何建议或者如果你能指出我正确的方向，我们将不胜感激。

Answer 1

urllib2.urlopen是用于访问单个URL的函数。如果要访问多个，则应循环遍历列表。你应该这样做：

for url in urllist:
    data = urllib2.urlopen(url)
    page = BeautifulSoup(data)

    # Check links
    for link in page.findAll('a'):
        try:
            href = link['href']
            if re.search(match, href):
                urllist2.append(href)

        except KeyError:
            pass

    os.chdir(r"C:\_VirusFixes")
    urllib.urlretrieve(urllist2, os.path.basename(urllist2))

Answer 2

除了mikez302's answer之外，还有一种更易读的编写代码的方式：

import os
import re
import urllib
import urllib2

from BeautifulSoup import BeautifulSoup

websites = [
    'http://cdn.superantispyware.com/SUPERAntiSpyware.exe'
    'http://support.kaspersky.com/downloads/utils/tdsskiller.exe'
    'http://www.bleepingcomputer.com/download/malwarebytes-anti-malware/dl/7/?1'
    'http://www.simplysup.com/tremover/download.html'
]

download_links = []

for url in websites:
    connection = urllib2.urlopen(url)
    soup = BeautifulSoup(connection)
    connection.close()

    for link in soup.findAll('a', {href: re.compile(r'\.exe$')}):
        download_links.append(link['href'])

for url in download_links:
    urllib.urlretrieve(url, r'C:\_VirusFixes', os.path.basename(url))

Answer 3

上面的代码对我不起作用，在我的情况下，这是因为页面通过脚本组装链接而不是将其包含在代码中。当我遇到这个问题时，我使用了下面的代码，它只是一个刮刀：

import os
import re
import urllib
import urllib2

from bs4 import BeautifulSoup

url = ''

connection = urllib2.urlopen(url)
soup = BeautifulSoup(connection) #Everything the same up to here 
regex = '(.+?).zip'       #Here we insert the pattern we are looking for
pattern = re.compile(regex)
link = re.findall(pattern,str(soup)) #This finds all the .zip (.exe) in the text
x=0
for i in link:
    link[x]=i.split(' ')[len(i.split(' '))-1] 
# When it finds all the .zip, it usually comes back with a lot of undesirable 
# text, luckily the file name is almost always separated by a space from the 
# rest of the text which is why we do the split
    x+=1  

os.chdir("F:\Documents")
# This is the filepath where I want to save everything I download

for i in link:
    urllib.urlretrieve(url,filename=i+".zip") # Remember that the text we found doesn't include the .zip (or .exe in your case) so we want to reestablish that.

这不如之前答案中的代码有效，但它几乎适用于任何网站。

从多个网站下载文件。

3 个答案: