Question

我正在使用python请求库和BeautifulSoup。当请求无效时，只有一个URL，它将返回弹出alert()的HTML。 Beautifulsoup中的问题是我无法获得window.alert弹出文本。

我尝试使用this answer中的regex方法，但似乎不起作用。

这样做时：

for script in soup.find_all("script"):
    alert = re.findall(r'(?<=alert\(\").+(?=\")', script.text)

脚本永远不会获得执行的脚本。

这是我正在提取的脚本：

<script language="JavaScript">
if(top.frames.length != 0) {
    location.href="frame_break.jsp"
}
</script>

<html>
<body>

</body>
</html>


<script>
    var err='User ID';
    alert(err);
    iBankForm.action='login.jsp';
    iBankForm.submit();
  </script>

我期望收到警报文本User ID。我注意到我是否有标签，无法在下面抓取脚本如果我将脚本删除或移动到body标签中，则可以获取

<script>
    var err='User ID';
    alert(err);
    iBankForm.action='login.jsp';
    iBankForm.submit();
  </script>

Answer 1

对您的数据运行BeautifulSoup的diagnose()时，我会获得以下信息：

data = '''
<script language="JavaScript">
if(top.frames.length != 0) {
    location.href="frame_break.jsp"
}
</script>

<html>
<body>

</body>
</html>


<script>
    var err='User ID';
    alert(err);
    iBankForm.action='login.jsp';
    iBankForm.submit();
  </script>'''

from bs4.diagnose import diagnose

diagnose(data)

打印：

Diagnostic running on Beautiful Soup 4.7.1
Python version 3.6.8 (default, Jan 14 2019, 11:02:34) 
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]]
Found lxml version 4.3.3.0
Found html5lib version 1.0.1

Trying to parse your markup with html.parser
Here's what html.parser did with the markup:
<script language="JavaScript">
 if(top.frames.length != 0) {
    location.href="frame_break.jsp"
}
</script>
<html>
 <body>
 </body>
</html>
<script>
 var err='User ID';
    alert(err);
    iBankForm.action='login.jsp';
    iBankForm.submit();
</script>
--------------------------------------------------------------------------------
Trying to parse your markup with html5lib
Here's what html5lib did with the markup:
<html>
 <head>
  <script language="JavaScript">
   if(top.frames.length != 0) {
    location.href="frame_break.jsp"
}
  </script>
 </head>
 <body>
  <script>
   var err='User ID';
    alert(err);
    iBankForm.action='login.jsp';
    iBankForm.submit();
  </script>
 </body>
</html>
--------------------------------------------------------------------------------
Trying to parse your markup with lxml
Here's what lxml did with the markup:
<html>
 <head>
  <script language="JavaScript">
   if(top.frames.length != 0) {
    location.href="frame_break.jsp"
}
  </script>
 </head>
 <body>
 </body>
</html>

--------------------------------------------------------------------------------
Trying to parse your markup with lxml-xml
Here's what lxml-xml did with the markup:
<?xml version="1.0" encoding="utf-8"?>
<script language="JavaScript">
 if(top.frames.length != 0) {
    location.href="frame_break.jsp"
}
</script>
--------------------------------------------------------------------------------

由此可见，lxml解析器将不会解析最后一个<script>，因此您永远不会通过BeautifulSoup到达它。解决方案是不同的解析器，例如html.parser：

import re
from bs4 import BeautifulSoup
soup = BeautifulSoup(data, 'html.parser')


for script in soup.select('script:contains(alert)'):
    alert = re.findall(r'(?<=alert\().+(?=\))', script.text)
    print(alert)

打印：

['err']

Answer 2

通过使用html5lib解析器库来解决如果您阅读文档https://www.crummy.com/software/BeautifulSoup/bs4/doc/，则其解析网页的方式与网络浏览器的解析方式相同这样就可以在body标签之外获取脚本

soup = BeautifulSoup(payload, 'html5lib')
        errors = None
        for scr in soup.find_all("script"):
            scrExtract = scr.extract()
            alert = re.findall('err="(.*\w)', scrExtract.text)
            if len(alert) > 0:
                errors = alert[0]

        print(errors)

显示警报时从窗口警报中删除警报文本

2 个答案: