Question

我希望找出在html文件中请求外部资源的网址。

我目前使用scr和img标记中的script属性，以及href标记中的link属性（用于标识css）。

我是否应该检查其他标签以识别其他资源？

作为参考，我目前的Python代码是：

html = read_in_file(file)
soup = BeautifulSoup(html)
image_scr = [x['src'] for x in soup.findAll('img')]
css_link = [x['href'] for x in soup.findAll('link')]
scipt_src = []   ## Often times script doesn't have attributes 'src' hence need for try/except
for x in soup.findAll('script'):
    try:
        scipt_src.append(x['src'])
    except KeyError:
        pass

Answer 1

更新了我的代码，以捕获html代码中最常见的资源。显然，这并没有查看CSS或Javascript中请求的资源。如果我遗失了标签，请发表评论。

from bs4 import BeautifulSoup 
def find_list_resources (tag, attribute,soup):
   list = []
   for x in soup.findAll(tag):
       try:
           list.append(x[attribute])
       except KeyError:
           pass
   return(list)

html = read_in_file(file)
soup = BeautifulSoup(html)

image_scr = find_list_resources('img',"src",soup)   
scipt_src = find_list_resources('script',"src",soup)    
css_link = find_list_resources("link","href",soup)
video_src = find_list_resources("video","src",soup)         
audio_src = find_list_resources("audio","src",soup) 
iframe_src = find_list_resources("iframe","src",soup)
embed_src = find_list_resources("embed","src",soup)
object_data = find_list_resources("object","data",soup)         
soruce_src = find_list_resources("source","src",soup)

Beutifulsoup从html中提取所有外部资源

1 个答案: