需要从脚本标签HTML Python中提取所有链接

Question

基本上，我需要解析HTML中所有<script>标签中的所有src =“”链接。

<script src="path/to/example.js" type="text/javascript"></script>

不幸的是，bs4无法做到这一点。有什么想法可以实现吗？

Answer 1

import requests
import bs4
text = requests.get('http://example.com').text
soup = bs4.BeautifulSoup(text, features='html.parser')
scripts = soup.find_all('script')
srcs = [link['src'] for link in scripts if 'src' in link.attrs]
print(srcs)

Answer 2

我会压缩并使用script[src]以确保脚本具有src属性

import requests
from bs4 import BeautifulSoup as bs
r = requests.get('http://example.com').content
soup = bs(r, 'lxml') # 'html.parser' if lxml not installed
srcs = [item['src'] for item in soup.select('script[src]')]