Question

我正在尝试在页上刮擦桌子。

我可以从浏览器调试器中看到，我想要的表存在于HTML中。例如您可以看到肽名：{{3}}

我编写了以下代码来提取该表：

system.data

但是输出的输出是：

for i in range(1001,1003):
#    try:
        res = requests.get("https://webs.iiitd.edu.in/raghava/antitbpdb/display.php?details=" + str(i))
        soup = BeautifulSoup(res.content, 'html.parser')
        table = soup.find_all('table')
        print table

有人可以解释为什么find_all无法找到所有表（特别是我想要的表）以及如何解决此问题吗？

Answer 1

不确定为什么不显示。

因为它也是一张桌子，所以我继续使用熊猫做.read_html

import pandas as pd

url = 'https://webs.iiitd.edu.in/raghava/antitbpdb/display.php?details=antitb_1001'

tables = pd.read_html(url)
table = tables[-1]

输出：

print (table)
                           0                                                  1
0        Primary information                                                NaN
1                         ID                                        antitb_1001
2               Peptide Name                                          Polydim-I
3                   Sequence                             AVAGEKLWLLPHLLKMLLTPTP
4    N-terminal Modification                                               Free
5    C-terminal Modification                                               Free
6      Chemical Modification                                               None
7             Linear/ Cyclic                                             Linear
8                     Length                                                 22
9                  Chirality                                                  L
10                    Nature                                        Amphipathic
11                    Source                                            Natural
12                    Origin  Isolated from the venom of the Neotropical was...
13                   Species         Mycobacterium abscessus subsp. massiliense
14                    Strain  Mycobacterium abscessus subsp. massiliense iso...
15  Inhibition Concentartion                                  MIC = 60.8 Î¼g/mL
16          In vitro/In vivo                                               Both
17                 Cell Line  Peritoneal macrophages, J774 macrophages cells...
18  Inhibition Concentartion  Treatment of infected macrophages with 7.6 Î¼g...
19              Cytotoxicity  Non-cytotoxic, 10% cytotoxicity on J774 cells ...
20             In vivo Model  6 to 8 weeks old BALB/c and IFN-Î³KO (Knockout...
21               Lethal Dose  2 mg/kg/mLW shows 90% reduction in bacterial load
22           Immune Response                                                NaN
23       Mechanism of Action                               Cell wall disruption
24                    Target                                          Cell wall
25       Combination Therapy                                               None
26          Other Activities                                                NaN
27                 Pubmed ID                                           26930596
28       Year of Publication                                               2016
29             3-D Structure                 View in Jmol or Download Structure

Answer 2

仅供参考（如果您想知道问题的根本原因），目标table具有无效的标记：

<table class ="tab" cellpadding= "5" ... STYLE="border-spacing: 0px;border-style: line ;
 <tr bgcolor="#DAD5BF"></tr>

请注意，开始标记未关闭：<table ...（应为<table ...>），祖先是<div>，而结束标记是</p>

这就是为什么BeautifulSoup无法将其识别为table，因此soup.find_all('table')不会返回它的原因

但是，现代的浏览器具有内置的工具来“修复”损坏的标签，因此在浏览器table中看起来并不“损坏”：关闭</div>已添加到祖先div中而p标签转换为空节点<p></p>

BeautifulSoup无法解析完整的HTML-这是因为动态HTML？

2 个答案: